Uber's approach to deploying machine learning

Continuous Integration and Continuous Deployment (CI/CD) are widely used in software development, but doing it for ML can be a bit more tricky.

Machine learning models are halfway between data and code. Trained models have similar functionality to code, but are learnt from and represented by data. Therefore standard approaches to version control, testing and CI/CD have to be tweaked a bit to work.

A recent blog post from Uber describes how they approach this. Uber have many ML models in production, providing critical services. Therefore they need to be both reliable and maintainable. ML engineers need to easily be able to add new features to models or tweak architectures without damaging old models and keeping the service running. Here Uber describe how they use dynamic model loading using a model artefact and config store to decouple model and server development cycles. Doing this allows ML engineers to easily replace models without having to change all of the serving code. To do this you also need extensive and reliable testing frameworks and it seems like they have this too.

🛎️ Why this matters: Useful primer on CI/CD for ML, although somewhat light on fully usable details

📖 Read more (2251 words) 📖