# BentoML — Build AI Model Serving APIs

> BentoML builds model inference REST APIs and multi-model serving systems from Python scripts. 8.6K+ GitHub stars. Auto Docker, dynamic batching, any ML framework. Apache 2.0.

## Install

Save as a script file and run:

## Quick Use

```bash
# Install
pip install -U bentoml

# Create a service (service.py)
cat > service.py << 'EOF'
import bentoml

@bentoml.service
class Summarizer:
    def __init__(self):
        from transformers import pipeline
        self.pipeline = pipeline("summarization")

    @bentoml.api
    def summarize(self, text: str) -> str:
        result = self.pipeline(text, max_length=100)
        return result[0]["summary_text"]
EOF

# Serve locally
bentoml serve service:Summarizer

# Build Docker container
bentoml build && bentoml containerize summarizer:latest
```

---

## Intro

BentoML is a Python framework for building online serving systems optimized for AI apps and model inference. With 8,600+ GitHub stars and Apache 2.0 license, it turns model inference scripts into production REST APIs using Python type hints, automatically generates Docker containers with dependency management, provides performance optimization through dynamic batching and model parallelism, and supports any ML framework and inference runtime. Deploy to Docker or BentoCloud for production.

**Best for**: ML engineers deploying models as production APIs with minimal boilerplate
**Works with**: Claude Code, OpenAI Codex, Cursor, Gemini CLI, Windsurf
**Frameworks**: PyTorch, TensorFlow, HuggingFace, ONNX, XGBoost, any runtime

---

## Key Features

- **Python-first**: Type hints auto-generate REST API schema
- **Auto Docker**: One command to containerize with all dependencies
- **Dynamic batching**: Automatically batch requests for throughput
- **Model parallelism**: Multi-GPU and multi-model serving
- **Any framework**: PyTorch, TensorFlow, HuggingFace, ONNX, XGBoost
- **BentoCloud**: Managed deployment with auto-scaling

---

### FAQ

**Q: What is BentoML?**
A: BentoML is a Python framework with 8.6K+ stars for turning ML models into production REST APIs. Auto Docker, dynamic batching, any framework. Apache 2.0.

**Q: How do I install BentoML?**
A: `pip install -U bentoml`. Decorate your class with `@bentoml.service`, methods with `@bentoml.api`, then `bentoml serve`.

---

## Source & Thanks

> Created by [BentoML](https://github.com/bentoml). Licensed under Apache 2.0.
> [bentoml/BentoML](https://github.com/bentoml/BentoML) — 8,600+ GitHub stars

---
Source: https://tokrepo.com/en/workflows/8885a870-2236-43c7-b948-5c0d330e17de
Author: Script Depot