3 ms·
The first thing we did when we started building our ml platform was to go with a plugin architecture. All applications are simply plugins. Data connectors are p
by Jugurtha 7y ago
The first thing we did when we started building our ml platform was to go with a plugin architecture. All applications are simply plugins. Data connectors are plugins.
We do consulting work and have to build custom solutions for our clients. Once they were satisfied with the models, they'd ask for an application to use these models: user management, data uploads, etc.
We used to build a custom solution for each client. It was very hard to change these solutions when client's demands changed, and it was almost impossible to reuse that solution with another client.
We took advantage of a decrease in activity to build a platform where things would be plugins. The time series forecasting application is a plugin, with endpoints you can tap into with authorization tokens.
The recommender system is a plugin, the notebook application is a plugin, etc.
Now our notebooks are "Published" in one click, which generates an AppBook a business person can interact with by changing the parameters. The runs are tracked (params, metrics, and models) in case domain experts tweak parameters that result in a better model than we have. The models can be deployed to endpoints you can invoke with generated tokens.
The major benefit is that when a new member joined the team, they had to understand the whole code base to be able to contribute or add something. Now? They don't have to understand anything, only the application they're building. If they want to render it on the sidebar, they just have to add a `sidebar.html` with the proper icon and route. If they want to show it in our appstore, they just have to add a file with the proper icon, and our platform handles the rest. It will discover the application, load it, etc.
An administrator can even load an application as a zip file. The platform will consume it, install its dependencies, load it, and activate it.
We want the product to be easy to add to and modify its behavior with config, instead of changing the code and we work to do just that.