6 ms·

Lots and lots of SQL (a very good thing IMO). This can come in the form of Oracle, Redshift, or any of the commonly available RDS databases (probably not SQL Se

Lots and lots of SQL (a very good thing IMO). This can come in the form of Oracle, Redshift, or any of the commonly available RDS databases (probably not SQL Server though). This is augmented with a lot of Big Data stuff, which used to be pretty diverse (Pig, Hive, raw Hadoop, etc.) but is moving very quickly towards a Spark-centric platform. There is occasionally some commercial software like Tableau/Microstrategy. Apart from that, it is a whole lot of homegrown technology. As far as architecture is concerned, every team is different. At least on the retail side, we tend to center around a large data warehouse that is currently transitioning from oracle to redshift, and with daily log parsing ETL jobs from most services to populate tables in that data warehouse. Downstream of the data warehouse is fair game for pretty much anything.

drited 11y ago
Would you know why sql server generally isn't used?
- deleted 11y ago
  [deleted]
- sshykes 11y ago
  Probably because: * SQL Server is proprietary * SQL Server licenses are expensive * SQL Server runs only on Windows (or least used to?)
  - hunterjrj 11y ago
    I chuckled a bit because the GP mentioned Oracle. I'm sure you've heard this one: Customer: "How much does an Oracle database license cost?" Oracle Rep: "Well, how much do you have?"
    - true_religion 11y ago
      That's actually not a joke. Oracle liked to argue you should accept server licences as being a fraction of your budget, so when it went up you'd automatically pay them more.
  - avon 11y ago
    Amazon actually does allow SQL Server. The poster saying it probably didn't was influenced by the fact that Azure, Microsoft's own cloud solution is an AWS competitor. To your own points: SQL Server is as proprietary as Oracle SQL Server is cheaper than Oracle SQL Server is being ported to Linux in 2017 :)
    - newjersey 11y ago
      Amazon.com seems to strongly encourage the companies it acquires to use aws. Apparently, woot.com is fixing to rewrite the website in Java or something just so they do not have to use Windows. Apparently, it makes sense to them.
arikfr 11y ago
Are you using Re:dash[1]? Seems like a good fit if you're doing lots of SQL and working with many different databases. [1] http://redash.io/ http://redash.io/
kakoni 11y ago
Do you have somesort of framework for those ETL jobs? How do you handle ETL dependencies/workflow?(=thinking spotify's luigi[1] here) [1] https://github.com/spotify/luigi https://github.com/spotify/luigi

drited 11y ago

Would you know why sql server generally isn't used?

deleted 11y ago
[deleted]
sshykes 11y ago
Probably because: * SQL Server is proprietary * SQL Server licenses are expensive * SQL Server runs only on Windows (or least used to?)
- hunterjrj 11y ago
  I chuckled a bit because the GP mentioned Oracle. I'm sure you've heard this one: Customer: "How much does an Oracle database license cost?" Oracle Rep: "Well, how much do you have?"
  - true_religion 11y ago
    That's actually not a joke. Oracle liked to argue you should accept server licences as being a fraction of your budget, so when it went up you'd automatically pay them more.
- avon 11y ago
  Amazon actually does allow SQL Server. The poster saying it probably didn't was influenced by the fact that Azure, Microsoft's own cloud solution is an AWS competitor. To your own points: SQL Server is as proprietary as Oracle SQL Server is cheaper than Oracle SQL Server is being ported to Linux in 2017 :)
  - newjersey 11y ago
    Amazon.com seems to strongly encourage the companies it acquires to use aws. Apparently, woot.com is fixing to rewrite the website in Java or something just so they do not have to use Windows. Apparently, it makes sense to them.

arikfr 11y ago

Are you using Re:dash[1]? Seems like a good fit if you're doing lots of SQL and working with many different databases. [1] http://redash.io/ http://redash.io/

kakoni 11y ago

Do you have somesort of framework for those ETL jobs? How do you handle ETL dependencies/workflow?(=thinking spotify's luigi[1] here) [1] https://github.com/spotify/luigi https://github.com/spotify/luigi