3 ms·

This is really cool but I'm curious why Bloomberg would need this. Ie, what special needs does Bloomberg have that would lead to a primarily non-engineering com

This is really cool but I'm curious why Bloomberg would need this. Ie, what special needs does Bloomberg have that would lead to a primarily non-engineering company investing the resources to create this. Was there nothing off the shelf that would have fit their needs? I don't mean that in a derogatory way, I'm just curious what motivated making this.

shusson 9y ago
The work started 12 years ago when there were probably no `NewSQL` [1] solutions out there. https://en.wikipedia.org/wiki/NewSQL https://en.wikipedia.org/wiki/NewSQL
zeusk 9y ago
Bloomberg, after all, is a high-tech news and information delivery network.
adenadel 9y ago
Bloomberg started as a technology company. Sure, they have expanded into media, but I'm pretty sure they are still first and foremost a provider of software (plus hardware) and data.
- raldi 9y ago
  I would bet that the Terminals are responsible for something like 140% of their profits, and that the media empire is a huge loss leader / vanity project.
  - gaius 9y ago
    The whole point of Reuters news is to inform traders about things that will lead to more trades being done over the Reuters platform, so it's entirely likely Bloomberg operates the same model.
deleted 9y ago
[deleted]
jroblak 9y ago
https://bloomberg.github.io/comdb2/overview_home.html https://bloomberg.github.io/comdb2/overview_home.html >> We had several goals in mind. One was being wire-format compatible with an older internal product to allow developers to migrate applications easier. Another was to have a database that could be updated anywhere and stay in sync everywhere. The first goal could only be satisfied in-house. The second was either in its infancy for open source products, or available at high cost from commercial sources.
joshu 9y ago
Bloomberg is a massive tech company.
magnawave 9y ago
I definitely wouldn't call Bloomberg a "non-engineering" company if you've ever worked with any of their trading / market data products. Pretty much true of any trading side finance company nowadays.
raldi 9y ago
"Non-engineering company" -- ha! But to answer your question: Mike Bloomberg's autobiography talks about this. When they started in the 80's there wasn't as much great off-the-shelf software as there is today. Their customers were (and always have been) insanely demanding when it comes to reliability and speed. The last thing Mike wanted to do was be caught sheepishly explaining to the CIO of Merrill Lynch, "Well, gee, our Oracle database has this bug they can't fix for the next two weeks..." or even "...this optimization they can't make for the next six months" This is a company that invented its own layer-2 network protocols 35 years ago just to squeeze every last drop of performance and reliability out of the hardware. Of course they wrote their own database (actually two -- there was a comdb 1 of of course, too)
foobarbazetc 9y ago
I've worked at midsize "non-engineering" companies that do boring stuff like logistics where they wrote their own RDBMS and even virtual machines for legacy hardware. It's not that uncommon especially if you've been around a long time.
keeptrying 9y ago
People forget that Bloomberg had computer networks before the Internet. I remember when we didn't even have comdb2.
zerr 9y ago
If even advertising companies are engineering shops, what makes you think fintech companies aren't?
- gaius 9y ago
  Right, it's like everyone's forgot what Google's business model actually is!
nemothekid 9y ago
>Was there nothing off the shelf that would have fit their needs? I can't think of any solid distributed RDBMS that would have been around in 2004. Does anyone with more knowledge have any idea (open or not)?
- Keyframe 9y ago
  IBM DB2 and Oracle 9i were around for sure. Other solutions as well. DB landscape was well-established back then.
  - dsparkman 9y ago
    They were, but rediculously expensive at the time. $100k+ at the time to have a non-replicated version on Oracle running on a multicore server. Dealing with Oracle was like dealing with the mob.
    - Keyframe 9y ago
      Indeed, but that's a whole other issue.
- gaius 9y ago
  DEC RDB might have done it, if it was still available.
alexjscotti 9y ago
So. We did a few things better than the industry to this day. HA is a spectrum. Transparent masking of all failures in any state of arbitary SQL transaction is on the far right side of that spectrum. To directly answer your question: go look at where 'replication' was on rdbms 14 years ago. PostgreSQL? MySQL?
- ericpearl 9y ago
  14 years ago Sybase had strong replication. Sybase was wall street's darling for a long time starting in the late 80's.
apaprocki 9y ago
Just for some scale, we have 4k+ engineering employees and if we were a public company, we'd be somewhere around the 5th largest software company in the world by revenue. (Strange stat, but our primary product is software and accounts for over 3/4 revenue)
- raldi 9y ago
  Looking forward to seeing a fastsend / PRCCOM paper one day.
  - apaprocki 9y ago
    PRCCOM is no more! Don't need to keep bringing that one up :)
    - raldi 9y ago
      Oh, in that case I hope you guys see fit to write it up one day. It's a legitimately interesting and clever solution, even if obsolete.
addicted 9y ago
Bloomberg was doing cloud computing decades before the term was even invented. Edit: As someone else points out, they were doing cloud computing before the internet existed.
75dvtwin 9y ago
>>what special needs does Bloomberg have that would lead to a primarily non-engineering company investing the resources to create this They probably open sourced the database, and would do more -- to dispel the above notion, and therefore to attract more qualified recruits. Financial services companies, and technology suppliers around them, have came up much of the 'modern day' tech, some times decade(s), before google's and facebook's of the world -- have had NoSQL with stored procs(eg Goldman S, early to mid 90s ), -- in-house grown programming lang (APL+ based in Morgan Stanley) emphasizing vector-based operations -- Smart contracts https://www.lexifi.com/product/technology/contract-description-language (where a contract is represented as algorithm specified in a domain specific language) ). This was way before etherium, started, I believe, in Credit Swiss (but not sure) -- one of the fastest time series database (kdb+) -- Data science and machine learning (modeling risk and valuations) Today's hedge funds manage petabytes of data. So do many of the big investment banks...

shusson 9y ago

The work started 12 years ago when there were probably no `NewSQL` [1] solutions out there. https://en.wikipedia.org/wiki/NewSQL https://en.wikipedia.org/wiki/NewSQL

zeusk 9y ago

Bloomberg, after all, is a high-tech news and information delivery network.

adenadel 9y ago

Bloomberg started as a technology company. Sure, they have expanded into media, but I'm pretty sure they are still first and foremost a provider of software (plus hardware) and data.

raldi 9y ago
I would bet that the Terminals are responsible for something like 140% of their profits, and that the media empire is a huge loss leader / vanity project.
- gaius 9y ago
  The whole point of Reuters news is to inform traders about things that will lead to more trades being done over the Reuters platform, so it's entirely likely Bloomberg operates the same model.

deleted 9y ago

[deleted]

jroblak 9y ago

https://bloomberg.github.io/comdb2/overview_home.html https://bloomberg.github.io/comdb2/overview_home.html >> We had several goals in mind. One was being wire-format compatible with an older internal product to allow developers to migrate applications easier. Another was to have a database that could be updated anywhere and stay in sync everywhere. The first goal could only be satisfied in-house. The second was either in its infancy for open source products, or available at high cost from commercial sources.

joshu 9y ago

Bloomberg is a massive tech company.

magnawave 9y ago

I definitely wouldn't call Bloomberg a "non-engineering" company if you've ever worked with any of their trading / market data products. Pretty much true of any trading side finance company nowadays.

raldi 9y ago

"Non-engineering company" -- ha! But to answer your question: Mike Bloomberg's autobiography talks about this. When they started in the 80's there wasn't as much great off-the-shelf software as there is today. Their customers were (and always have been) insanely demanding when it comes to reliability and speed. The last thing Mike wanted to do was be caught sheepishly explaining to the CIO of Merrill Lynch, "Well, gee, our Oracle database has this bug they can't fix for the next two weeks..." or even "...this optimization they can't make for the next six months" This is a company that invented its own layer-2 network protocols 35 years ago just to squeeze every last drop of performance and reliability out of the hardware. Of course they wrote their own database (actually two -- there was a comdb 1 of of course, too)

foobarbazetc 9y ago

I've worked at midsize "non-engineering" companies that do boring stuff like logistics where they wrote their own RDBMS and even virtual machines for legacy hardware. It's not that uncommon especially if you've been around a long time.

keeptrying 9y ago

People forget that Bloomberg had computer networks before the Internet. I remember when we didn't even have comdb2.

zerr 9y ago

If even advertising companies are engineering shops, what makes you think fintech companies aren't?

gaius 9y ago
Right, it's like everyone's forgot what Google's business model actually is!

nemothekid 9y ago

>Was there nothing off the shelf that would have fit their needs? I can't think of any solid distributed RDBMS that would have been around in 2004. Does anyone with more knowledge have any idea (open or not)?

Keyframe 9y ago
IBM DB2 and Oracle 9i were around for sure. Other solutions as well. DB landscape was well-established back then.
- dsparkman 9y ago
  They were, but rediculously expensive at the time. $100k+ at the time to have a non-replicated version on Oracle running on a multicore server. Dealing with Oracle was like dealing with the mob.
  - Keyframe 9y ago
    Indeed, but that's a whole other issue.
gaius 9y ago
DEC RDB might have done it, if it was still available.

alexjscotti 9y ago

So. We did a few things better than the industry to this day. HA is a spectrum. Transparent masking of all failures in any state of arbitary SQL transaction is on the far right side of that spectrum. To directly answer your question: go look at where 'replication' was on rdbms 14 years ago. PostgreSQL? MySQL?

ericpearl 9y ago
14 years ago Sybase had strong replication. Sybase was wall street's darling for a long time starting in the late 80's.

apaprocki 9y ago

Just for some scale, we have 4k+ engineering employees and if we were a public company, we'd be somewhere around the 5th largest software company in the world by revenue. (Strange stat, but our primary product is software and accounts for over 3/4 revenue)

raldi 9y ago
Looking forward to seeing a fastsend / PRCCOM paper one day.
- apaprocki 9y ago
  PRCCOM is no more! Don't need to keep bringing that one up :)
  - raldi 9y ago
    Oh, in that case I hope you guys see fit to write it up one day. It's a legitimately interesting and clever solution, even if obsolete.

addicted 9y ago

Bloomberg was doing cloud computing decades before the term was even invented. Edit: As someone else points out, they were doing cloud computing before the internet existed.

75dvtwin 9y ago

>>what special needs does Bloomberg have that would lead to a primarily non-engineering company investing the resources to create this They probably open sourced the database, and would do more -- to dispel the above notion, and therefore to attract more qualified recruits. Financial services companies, and technology suppliers around them, have came up much of the 'modern day' tech, some times decade(s), before google's and facebook's of the world -- have had NoSQL with stored procs(eg Goldman S, early to mid 90s ), -- in-house grown programming lang (APL+ based in Morgan Stanley) emphasizing vector-based operations -- Smart contracts https://www.lexifi.com/product/technology/contract-description-language (where a contract is represented as algorithm specified in a domain specific language) ). This was way before etherium, started, I believe, in Credit Swiss (but not sure) -- one of the fastest time series database (kdb+) -- Data science and machine learning (modeling risk and valuations) Today's hedge funds manage petabytes of data. So do many of the big investment banks...