Machine learning lab: your open-source machine learning platform

3 years ago, when Machine Learning became more widely used in the industry, we realised that there were very few tools that could help to experiment and prototype ML models quickly. Not only that, but in the industry, ML was facing issues that didn’t exist in research, such as authorisations, security, or all the caveats that come to light when trying to productize a proof-of-concept. That’s why we developed Machine Learning Lab and we’re happy to announce that since September 2020, it is available as an open-source project for everybody to enjoy!

ML Lab started as a simple, light-weight solution to cover these needs. The approach we took was to bring the tool to the data and not the other way around. To achieve this, ML Lab leveraged containerized technologies on the basis of Docker and Kubernetes(K8s). This allows us to deploy the platform quickly and easily in any server or cluster where Docker is available. Going hand-in-hand with containerization, the backend and web app were written with having cloud-native principles in mind from the beginning.

ML Lab user page

The Workspace

ML Lab launches a modified version of the open-source project ML Workspace for each user. With the workspace, the user has some tools and features at hand to support with the data science process in an isolated compute environment. For example, it has tools such as VNC or VS Code installed as well as Python or R for programming or ssh for remote access. The Lab-flavor has some Lab-specifics installed. As an example, it comes pre-installed with the ML Lab Python client with which data, models, and experiments of an ML Lab project can be accessed, up-or downloaded, or tracked, respectively.

ML Lab Workspace

Experiments

One of the biggest problems in ML is reproducibility of the experiments. ML Lab offers an experiments’ dashboard to help you keep track of the different runs. By using the python client library for ML Lab, data scientists are able to keep track of metrics, artifacts, and dependencies of their experiments, making it easy to reproduce results and collaborate between different team members working on a project.

Experiments

Models and datasets

Machine Learning Lab also offers a simple storage system to keep track of your datasets and Machine Learning models. Every file is versioned and you’re able to integrate into different systems and tools by using the provided API.

More-over, ML Lab comes with a custom python library: UnifiedModel which allows you to have one-click deployments from your model.

Models

Service deployment

You have experimented with different architectures, you have done some hyperparameter optimization, and now you want to expose your model as a REST API, but how you can do this? ML Lab offers the possibility of spinning-up Docker containers, so you can create your image and then use ML Lab to serve this image exposing your model as a web service.

Moreover, we facilitate the step of creating the docker image and the web service by providing some helpful tools such as the unified model library, which will allow you to go from a Python script into a web service in a couple of commands!

What’s next?

We’re continuously improving the code-base and we already have some features in the making, just to mention two:

• Ability to spawn different workspaces with customisable compute resources and base images. This will allow data scientists to use their own docker images as a base for their workspaces with the necessary libraries already pre-installed.
• Enablement of different authentication mechanisms so that users can configure them such as login with Google or GitHub.

Are you curious? Do you want to try it out?

Check out the repository: https://github.com/SAP/machine-learning-lab PR’s are welcome!

The documentation: https://sap.github.io/machine-learning-lab/

Shout out to Benjamin Raethlein, who was contributing to this blog post!