CLI

bentoml

BentoML CLI tool

bentoml [OPTIONS] COMMAND [ARGS]...

Options

-q, --quiet

Hide process logs and only print command results

--verbose

Print verbose debugging information for BentoML developer

--version

Show the version and exit.

<API_NAME>

Run a API defined in saved BentoService bundle from command line

bentoml <API_NAME> [OPTIONS] API_NAME BUNDLE_PATH

Options

--with-conda

Run API server in a BentoML managed Conda environment

Arguments

API_NAME

Required argument

BUNDLE_PATH

Required argument

config

Configure BentoML configurations and settings

bentoml config [OPTIONS] COMMAND [ARGS]...

reset

Reset all local BentoML configs to default

bentoml config reset [OPTIONS]

set

Set config value in local BentoML configuration file

bentoml config set [OPTIONS] [UPDATES]...

Arguments

UPDATES

Optional argument(s)

unset

Unset config in local BentoML configuration file

bentoml config unset [OPTIONS] [UPDATES]...

Arguments

UPDATES

Optional argument(s)

view

View local BentoML configurations

bentoml config view [OPTIONS]

view-effective

View effective BentoML configs, including default config values and local config overrides

bentoml config view-effective [OPTIONS]

deployment

Commands for creating and managing BentoService deployments on cloudcomputing platforms or kubernetes cluster

bentoml deployment [OPTIONS] COMMAND [ARGS]...

apply

Apply model service deployment from yaml file

bentoml deployment apply [OPTIONS]

Options

-f, --file <deployment_yaml>

[required]

-o, --output <output>
Options

json|yaml

--wait, --no-wait

Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

create

Create a BentoService model serving deployment

bentoml deployment create [OPTIONS] NAME

Options

-b, --bento <bento>

Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0” [required]

-p, --platform <platform>

Which cloud platform to deploy this BentoService to [required]

Options

aws-lambda|gcp-function|aws-sagemaker|kubernetes|custom

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “default” whichcan be changed in BentoML configuration file

-l, --labels <labels>

Key:value pairs that are attached to deployments and intended to be usedto specify identifying attributes of the deployments that are meaningful to users

--annotations <annotations>

Used to attach arbitary metadata to BentoService deployments, BentoML library and other plugins can then retrieve this metadata.

--region <region>

Directly mapping to cloud provider region. Option applicable to platform:AWS Lambda, AWS SageMaker, GCP Function

--instance-type <instance_type>

Type of instance will be used for inference. Option applicable to platform: AWS SageMaker, AWS Lambda, GCP Function

--instance-count <instance_count>

Number of instance will be used. Option applicable to platform: AWS SageMaker

--api-name <api_name>

User defined API function will be used for inference. Option applicableto platform: AWS SageMaker

--kube-namespace <kube_namespace>

Namespace for kubernetes deployment. Option applicable to platform: Kubernetes

--replicas <replicas>

Number of replicas. Option applicable to platform: Kubernetes

--memory-size <memory_size>

Maximum Memory Capacity for AWS Lambda function, you can set the memory size in 64MB increments from 128MB to 3008MB. The default value is 1024 MB.

--timeout <timeout>

The amount of time that AWS Lambda allows a function to run before stopping it. The default is 3 seconds. The maximum allowed value is 900 seconds

--service-name <service_name>

Name for service. Option applicable to platform: Kubernetes

--service-type <service_type>

Service Type. Option applicable to platform: Kubernetes

-o, --output <output>
Options

json|yaml

--wait, --no-wait

Wait for cloud resources to complete creation or until an error is encountered. When set to no-wait, CLI will return immediately after sendingrequest to cloud platform.

Arguments

NAME

Required argument

delete

Delete deployment

bentoml deployment delete [OPTIONS] NAME

Options

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “default” whichcan be changed in BentoML configuration file

--force

force delete the deployment record in database and ignore errors when deleting cloud resources

Arguments

NAME

Required argument

describe

View the detailed state of the deployment

bentoml deployment describe [OPTIONS] NAME

Options

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “default” whichcan be changed in BentoML configuration file

-o, --output <output>
Options

json|yaml

Arguments

NAME

Required argument

get

Get deployment current state

bentoml deployment get [OPTIONS] NAME

Options

-n, --namespace <namespace>

Deployment namespace

-o, --output <output>
Options

json|yaml

Arguments

NAME

Required argument

list

List active deployments

bentoml deployment list [OPTIONS]

Options

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “default” whichcan be changed in BentoML configuration file

--all-namespaces
--limit <limit>

Limit how many deployments will be retrieved

--filters <filters>

List deployments containing the filter string in name or version

-l, --labels <labels>

List deployments matching the giving labels

-o, --output <output>
Options

json|yaml|table

info

List all APIs defined in the BentoService loaded from saved bundle

bentoml info [OPTIONS] BUNDLE_PATH

Arguments

BUNDLE_PATH

Required argument

open-api-spec

Display API specification JSON in Open-API format

bentoml open-api-spec [OPTIONS] BUNDLE_PATH

Arguments

BUNDLE_PATH

Required argument

serve

Start REST API server hosting BentoService loaded from saved bundle

bentoml serve [OPTIONS] BUNDLE_PATH

Options

--port <port>

The port to listen on for the REST api server, default is 5000.

--with-conda

Run API server in a BentoML managed Conda environment

Arguments

BUNDLE_PATH

Required argument

serve-gunicorn

Start REST API server from saved BentoService bundle with gunicorn

bentoml serve-gunicorn [OPTIONS] BUNDLE_PATH

Options

-p, --port <port>
-w, --workers <workers>

Number of workers will start for the gunicorn server

--timeout <timeout>
--with-conda

Run API server in a BentoML managed Conda environment

Arguments

BUNDLE_PATH

Required argument