4 ms·
Well, simplest way to put it is - you get an Data Engineering (ETL) product - that'll move your 10s of thousands of ETL workflows from legacy ETL product to Spa
by ibains 7y ago
Well, simplest way to put it is - you get an Data Engineering (ETL) product - that'll move your 10s of thousands of ETL workflows from legacy ETL product to Spark, give you a full stack ETL product on Spark, and move you from on-premise to cloud.
Plus it has improvements along multiple dimensions such as interfaces - that developers care a lot about!
- mlevental 7y agowhat is "legacy ETL" exactly? you have a transpiler but what code does it transpile? what if all my ETL is written in Haskell and SQL (it's not - I don't have any ETL processes - I'm just trying to figure out what you're transpiling)
- ibains 7y agoThis is the workflows written in legacy formats - Ab Initio, Informatica, IBM DataStage - this is a $10B+ industry - so lots of usage out there. They typically have graph formats and Domain Specific Languages to specify transformations. We reverse engineer them. Enterprises have 10s of thousands of these workflows and getting out of legacy lock-in is hard!