So, you’ve trained your machine learning model and now you want to put it into production? Possibly, you even would like to deploy a microservice, where you can expose its prediction logic through an HTTP interface. If you’ve ever gone through this process, you’ve probably encountered two problems:

• Moving Python-based machine learning models around is difficult, as the model is not fully self-contained and needs to be packed with all of its dependencies.
• Making a prediction for a single input from an HTTP request can be slow using a batch-optimized Python-based model, possibly too slow.

In this talk, you will learn how we solved this problem in a dynamic-pricing project using the Open Neural Network Exchange (ONNX) framework, where we complied our model into a statically typed, fully self-contained computational graph that we could invoke using an optimized runtime, which allowed us to reduce single-row inference time by 99%, while also drastically simplifying our model management.

You will be introduced to the ONNX ecosystem, consisting of the specification itself, as well as converters, runtimes, optimizers, and visualizers. You will also learn how to convert scikit-learn-based modeling pipelines to the ONNX specification, using sklearn-onnx and onnxmltools. Finally, you will learn how to implement ONNX converters for your custom machine learning transformers and estimators.

Jan-Benedikt Jagusch

Affiliation: QuantCo

Hi there! I'm a data engineer at QuantCo and organizer of PyData Südwest. I'm located in Heidelberg, Germany.

visit the speaker at: Github

Christian Bourjau

Affiliation: QuantCo

Ever since having done a Masters and a PhD in experimental particle physics Christian has a passion to combine cutting edge data science with modern software engineering. The goal of that endeavor is to create projects that are efficient to work in for data scientists but also have a clean and maintainable path towards production. Recently, Christian has focused much effort to that end on ONNX ecosystem during his work at QuantCo.

visit the speaker at: Github