Logging#
time expected: 6 minutes
Server Logging#
BentoML provides a powerful and detailed logging pattern out of the box. Request logs for webservices are logged along with requests to each of the model runner services.
The request log format is as follows:
time [LEVEL] [component] ClientIP:ClientPort (scheme,method,path,type,length) (status,type,length) Latency (trace,span,sampled)
For example, a log message might look like:
2022-06-28T18:07:35-0700 [INFO] [api_server] 127.0.0.1:37386 (scheme=http,method=POST,path=/classify,type=application/json,length=20) (status=200,type=application/json,length=3) 0.005ms (trace=67131233608323295915755120473254509377,span=4151694932783368069,sampled=0)
OpenTelemetry Compatible#
The BentoML logging system implements the OpenTelemetry standard for HTTP throughout the call stack to provide for maximum debuggability. Propogation of the OpenTelemetry parameters follows the standard provided here
The following are parameters which are provided in the logs as well for correlation back to particular requests.
trace_id is the id of a trace which tracks “the progression of a single request, as it is handled by services that make up an application.” [1]
span_id is the id of a span which is contained within a trace.
“A span is the building block of a trace and is a named, timed operation that represents a piece of the workflow in the distributed system. Multiple spans are pieced together to create a trace.” [2]
sampled_id is the number of times this trace has been sampled.
“Sampling is a mechanism to control the noise and overhead introduced by OpenTelemetry by reducing the number of samples of traces collected and sent to the backend.” [3]
Logging Configuration#
Access logs can be configured by setting the appropriate flags in the bento configuration file for both web requests and model serving requests. Read more about how to use a bento configuration file here in the - Configuration Guide
To configure other logs, please use the default Python logging configuration. All BentoML logs are logged under the bentoml
namespace.
API Service Request Logging#
For requests made to the API server, logging can be enabled and disabled using the api_server.logging.access
parameter at the
top level of the bentoml_configuration.yml
.
api_server:
logging:
access:
enabled: true
# whether to log the size of the request body
request_content_length: true
# whether to log the content type of the request
request_content_type: true
# whether to log the content length of the response
response_content_length: true
# whether to log the content type of the response
response_content_type: true
Model Server Request Logging#
Depending on how you’ve configured BentoML, the API server and runner server can be run
separately. In either case, BentoML also provides a logging configuration under
runners
to allow user to configure the runner server output logs.
You may configure the runner access logs under the runners parameter at the top level of your bentoml_configuration.yml
:
runners:
logging:
access:
enabled: true
...
The available configuration options are the same to the one for API service request logging options above.
Access Logging Format#
You may configure the format of the Trace and Span IDs in the access logs in bentoml_configuration.yml
.
The default configuration is shown below, where the opentelemetry trace_id
and span_id
are logged in
hexadecimal format, consistent with OpenTelemetry logging instrumentation. You may also configure other format
specs, such as decimal d
.
api_server:
logging:
access:
format:
trace_id: 032x
span_id: 016x
Library Logging#
When using BentoML as a library, BentoML does not configure any logs. By default, Python will configure a root logger that logs at level WARNING and higher. If you want to see BentoML’s DEBUG or INFO logs, register a log handler to the bentoml
namespace:
import logging
ch = logging.StreamHandler()
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
ch.setFormatter(formatter)
bentoml_logger = logging.getLogger("bentoml")
bentoml_logger.addHandler(ch)
bentoml_logger.setLevel(logging.DEBUG)
Note
Important: bentoml serve
will fork the given service.py into the child process. This means any type of rotation FileHandler are not supported within the service definition. See notes from Python logging module for more information.
Notes