Orchestration is the automated management and coordination of systems that consist of various steps and subroutines. Python developers often need to manage many applications and data pipelines. Doing this manually is not a scalable solution. Especially if pipelines have interdependencies and need to be executed in minute intervals. The more complex the application system, the more complicated it is to manage and the need for an automated solution increases. This is where orchestration comes into play. However, it is easy to lose the overview of the rapidly evolving environment of orchestration tools. This talk is appropriate for users new to the topic as well as users that used a specific orchestration tool extensively. It gives a broad overview of many different orchestration tools and goes into depth of three carefully selected frameworks. Namely Kubeflow, Airflow and Prefect.

Jannis Grönberg

Affiliation: Kineo.ai

I am a passionate computational engineering graduate and highly motivated to develop innovative solutions help shape a better future.

visit the speaker at: Github