3 ms·
I'll do you one better. Productionizing a data science prototype is exactly the kind of grunt work AI is able to take over. I think its a much better result to
by dcreater 1y ago
I'll do you one better. Productionizing a data science prototype is exactly the kind of grunt work AI is able to take over.
I think its a much better result to have data science prototype translated to a performant production version rather than have a databricks type approach or what bauplan is proposing.
- stingraycharles 1y agoMaybe, but it would still need to work within a well defined framework. Usually the data science part is “solve the problem”, the data engineering part is “make it work reliably, fast, at scale”. What that looks like is highly dependent upon the environment at hand, and letting AI take that over may be one of those “now you have 2 problems” things.
- jtagliabuetooso 1y agoWe are not proposing or advocating for any approach to development (I personally almost never use notebooks these days and run Bauplan with preview). The blog together with our marimo friends is to showcase that you can have notebook development if you like it AND cloud scaling (which u need) without code changes, thanks to the fact that both marimo and Bauplan are basically Python (maybe a small thing, but there is nothing else in the market remotely close). On the AI part, we agree: the fact that bauplan is just Python, including data management and infra-as-code, makes it trivial for AI to build pipelines in Bauplan, which is not something that can be said about other data platforms - if you follow our blog, we are releasing in a few weeks or so a full "agentic" implementation with Bauplan API of production ETL workloads, which you may find interesting.