4 ms·

I am slightly confused by your reply. Can you elaborate a bit, was it good or bad?

• wodenokoto 3y ago
  You can develop a database schema for your application on dummy data, and test your business logic on dummy data. But you can't develop a machine learning model on dummy data. It is best practice to keep live data out of your development environment for normal software development, but it is impossible for machine learning projects.
  • fifilura 3y ago
    Yes, this is what I thought and exactly what I am experiencing right now. It is somewhat frustrating to see the engineering/platform team spending time on this kind of abstraction that is essentially useless for doing the work we need to do.
  • tomrod 3y ago
    Yep. I advise my clients to have a controlled, sandbox partition of prod. Call it the whiteboard or what have you, things can be erased or you can cache model version runs. Your data transforms can happen in dev but training should happen against real data. Synthetic data can be a compromise but that can sharply bias your model and, since it a harder lift, often multiplies data prep time