Continuous Deployment for Azure ML Pipelines
Hi friends,
We all know that machine learning projects are quite different from standard software development projects. For example, estimating an ML project is surprisingly challenging, to the point of being impossible at times. Maintaining an ML project, specifically deciding when to update a model or a pipeline or a dataset is quite different too when compared to maintaining a standard dev project.
This of course comes as no surprise, since developing a classic system requires code, code, and a bit more code sprinkled away as microservices, whereas a machine learning system is based on data, lots of data, all the data if possible. One is deterministic, the other less so.
But I digress.
One thing I really enjoy when doing work in, say, C#, is how easy it is to use best practices such as automatic builds. Creating a build that runs every time I push something to main, compiles my code, and deploys it to Azure in a blaze of glory, is quite straightforward. It's not as straightforward for machine learning projects though, so I've written a guide on doing the same with Azure ML Pipelines. Hope you find it useful!
5 things to read
I'm also including five of the most interesting things I've read/listened to on the web lately, which I think you'll enjoy:
- Erik Bernhardsson's story on building a data team. It's well written, it's funny, and most of all it rings true.
- Uber Engineering on tuning model performance
- A summary of Cal Newport's So Good They Can't Ignore You that's better than the book imho
- Derek Sivers on letting go of your projects & ideas and seeing which ones come back
- And finally, how Team Wiz hacked thousands of Azure customers’ databases. Don't miss their post on protecting your environments, too.
In other news
Sandro Mancuso and I are talking about craftsmanship in AI project at next week's Codecamp on Architecture & Design. It's free for everyone, so I'm counting on you to join us!
Yours truly,
Vlad
Vlad Iliescu
Click here to read or share this on the web. Hi friends, This was a lighter month in terms of writing articles, but don't fret - I've still got some interesting things for you to read. For starters, if you want to understand more about craftsmanship and about honing your machine learning skills, you might want to read this thread 🧶, the distillation of my conversation with Sandro Mancuso from earlier this month. I've also a couple of tips on Azure Functions, stuff like how to configure...
Hi friends, The future is coming and there's reason to be excited about it. I've been lucky enough to be able to test GitHub Copilot for the past few weeks and even though it has its quirks, it looks very promising. Not promising enough to fear losing our jobs, but promising enough to warrant daydreaming about the day when we'll all be using Copilot-powered compilers to compile English to Python/C#/JavaScript/etc. Sigh. I've even written an article about my experience. It has significantly...
Hi friends, I've just published a new Azure ML-focused article called 3 Ways to Pass Data Between Azure ML Pipeline Steps. As the title implies, it goes beyond what I've written in my previous article on Azure ML pipelines, and takes an in-depth look at the various ways of passing data in Azure ML pipelines. I quite like it. Also, if you're in the mood for something lightweight and absolutely unrelated to ML, take a look at my article on caching, Cloudflare and Netlify - How I Got Caching...