19 ms·

Uber has started many projects that ended up getting open sourced. And many of them are now either abandoned or on life support. H3 comes to mind as something w

Uber has started many projects that ended up getting open sourced. And many of them are now either abandoned or on life support. H3 comes to mind as something we almost ended up using but luckily avoided. These open-sourcings seem a bit like PR pieces with no guarantees of any support or evolution after being published.

gtirloni 7y ago
It's opensource. Why should Uber give any guarantees? They are not in the business of selling software. Unless Uber is actively blocking contributions, it's not Uber's fault if no community formed around something they opensourced. As for this being a PR piece, they could have achieved the same with just a detailed blog post and no code. It looks like a expensive PR piece if they have to opensource work that took probably hundreds of development hours.
- missosoup 7y ago
  I agree with everything you say. But without any certainty around the roadmap, support, and longterm commitment by Uber to maintain these projects, they're nothing more than interesting repos amongst a sea of interesting repos. The way Uber brands them suggests that they're suitable for use in production environments, but so far that hasn't been the case with anything they open sourced outside a narrow envelope that resembles their own operating model. Maybe this project will set a new trend, but so far nothing they put out gained any traction or became suitable for general purpose production use. In that regard, H3 and their other projects have remained at the level of decent 'show HN' pieces rather than something you'd ever use professionally. In other words, marketing. Based on previous news coming out e.g. https://news.ycombinator.com/item?id=20931644 https://news.ycombinator.com/item?id=20931644 It seems like Uber had too big of an engineering department with too little work to do, so they started reinventing wheels. Which is cool if they're willing to support them in the long term, but so far that hasn't proven to be the case.
  - mamon 7y ago
    Isn't that kind of the point of open-sourcing your internal tools? You don't want to be bothered with maintenance and support, so you're hoping that some anonymous volounteers will do that for you :)
    - cfors 7y ago
      Maybe I’m a cynic of all big corp companies but if you’ve ever worked with a big corp open source department that’s almost the entire point is to build PR for the engineering department. Same goes for tech blogs. These things will be PR pieces first, and valid production tools/frameworks second (mostly).
      - closeparen 7y ago
        In my experience software gets written in the first place for the usual internal reasons. Corporate or individual prestige may be the driving factor in open sourcing, though, rather than a genuine interest in having it used externally.
    - lopsidedBrain 7y ago
      Pretty much every successful open source project that people pay attention to is one that has had long-term support behind it. Linux, Mozilla, gcc, clang, git. Almost always, that support begins with the original author. Protects that don't do that are therefore unlikely to remain interesting for long.
  - carlisle_ 7y ago
    >It seems like Uber had too big of an engineering department with too little work to do, so they started reinventing wheels. Which is cool if they're willing to support them in the long term, but so far that hasn't proven to be the case. Former Uber engineer here. I can assure you that while our engineering team was massive, there was anything but too little work. If anything most engineers were massively overtaxed. Whether or not the work we were undertaking was meritious and valuable is an entire branch of philosophy I'm pretty sure. Part of the struggle at big companies is that a lot of existing solutions just don't work. Let me use an example with chat. A few years ago Slack was evaluated as a replacement for HipChat, since Atlassian's outages had finally started affecting us during our own outages. Everybody wanted to go to Slack, but the cost of Slack was tremendously prohibitive and the state of the service then (as I was told) was such that it could not support a company of Uber's size. Tremendous effort would have been undertaken by Slack to support Uber and they didn't want to expend that effort for a single customer. This was late 2015 early 2016. There were tons of options, but ultimately an in-house chat software was created. At the time it seemed required to make our own highly reliable chat, considering how distributed engineering teams were. I think if you talk to anybody without the background of how chat evolved at Uber they would think the in-house chat project would have been a boondoggle. Not all over-scoped engineering projects are actually so noble. There was certainly a ton of "reinventing wheels" going on. There was significantly more "these problems are really hard and I only have bad solutions." Though if the result is ultimately, "nothing more than interesting repos amongst a sea of interesting repos" sign me up.
- _jal 7y ago
  > Why should Uber give any guarantees? First, "guarantee" is the wrong word to take too literally here. Depending on how you want to look at it, there are no guarantees, even with guarantees. But looked at more loosely, answering that is really Uber's responsibility. Why did they release it? If it is just a PR release, fire-and-forget works fine for that. If they want to see wider adoption outside of their firm, there are some fairly obvious things they should do to foster that. Sometimes you release just the right thing at just the right time and everyone else does your evangelism and support work for you, but it is much more normal for your next great thing to take a while to build a user base.
  - tylerl 7y ago
    There are lots of reasons to open source internal software, and only a minority of them involve establishing a serious community and driving significant adoption. But the PR claim you're making isn't particularly credible. The ROI is abysmal if that's all you're after, and there are easier ways to get it. Based on your specific complaints, it sounds like your opinion doesn't matter in this case; you're not the audience. You want support and a predictable future: you're looking for a product, not for technology. This isn't a product, and it's not a platform. If instead you represented another company looking into solving this same problem yourself, and are looking at starting points, then you're the perfect audience. In that case, you'd have time and motivation to contact the developers directly rather than gripe on HN. You'd be less interested in whether there was an organized community, and more interested in how to directly influence the roadmap. You'd care about what the code looks like, how they solved Problem X and Problem Y, that kind of thing.
- deleted 7y ago
  [deleted]
- ForHackernews 7y ago
  Everything you say is true, but tossing useless code releases over the wall isn't really participating in the open source community, either. It looks to me like maybe their engineers internally are fans of the idea of "open source", and the PR department is happy to try and get some good press out of it, but the company culture isn't really set up to develop in public or maintain these things they've nominally "released". Sadly, this isn't unusual among tech companies, but it'd be more obvious what's happening if they just put up a bare-bones FTP with a README: "Here's some code under <LICENSE>. Use it at your own risk."
- papito 7y ago
  Even I have half a mind about releasing something that I cannot commit to for a little bit - the initial bugfix stage, at least. I would say a giant like Uber hurts themselves more in terms of PR when not being conservative enough about putting source out there. People inevitably gravitate towards big player "open sauce" as it implies some commitment. A company should first put their own system through hell and decide that "yeah, this is good, we are sticking with this", before luring people to use it.
- grogenaut 7y ago
  Open source for the steward is limping the project along is the worst type of open source because the steward usually isn't going to make any real decisions around the project the people are reticent to fork it and drive it because there is a Steward doing some activity. Selenium was in the state for years.
- Udik 7y ago
  > They are not in the business of selling software. But they are probably in the business of selling themselves as a software/ tech company. People value tech companies, so you'd better be one, even if you do taxi services, produce and distribute tv series, or rent office space.
reichardt 7y ago
Why do you consider H3 to be on life support? It's basically a finished spec with actively developed implementations. https://github.com/uber/h3 https://github.com/uber/h3
- deleted 7y ago
  [deleted]
- ForHackernews 7y ago
  So is Gopher, but almost nobody uses it.
richieartoul 7y ago
Chronosphere, a startup founded by two of the early M3 engineers, just raised 11 million dollars to build a monitoring platform based around M3DB: https://techcrunch.com/2019/11/05/chronosphere-launches-with-11m-series-a-to-build-scalable-cloud-native-monitoring-tool https://techcrunch.com/2019/11/05/chronosphere-launches-with... Uber also uses M3DB extensively internally and the project is nowhere near being abandoned or on life support: https://github.com/m3db/m3/commits/master https://github.com/m3db/m3/commits/master
- bfung 7y ago
  Yep, another way to pre-market/pre-signal to investors. A pretty common story these days: 1. I built an X to solve Unicorn U’s problem. 2. Open source it, give talks on it 3. Leave Unicorn U and start a company based on X 4. ... 5. Profit(???) Also, debatable whether or not Unicorn U actually needed a freshly built X instead of using existing tools/tech.
  - sieabahlpark 7y ago
    Seems genius imo
- StreamBright 7y ago
  >> While the founders, CEO Martin Mao and CTO Rob Skillington, were working at Uber, they recognized a gap in the monitoring industry, particularly around cloud-native technologies like containers and microservices What is the actual gap that is not addressed with one or all of the following? - Prometheus / Grafana [1] - Datadog [2] - Cloudwatch [3] 1. https://docs.docker.com/config/thirdparty/prometheus/ https://docs.docker.com/config/thirdparty/prometheus/ 2. https://www.datadoghq.com/blog/introducing-live-container-monitoring/ https://www.datadoghq.com/blog/introducing-live-container-mo... 3. https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-metrics-EKS.html https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitori...
  - opsunit 7y ago
    Wavefront: https://www.wavefront.com/ https://www.wavefront.com/
api 7y ago
Many large companies push this stuff out for publicity and recruitment. Sometimes employees are encouraged to spend a little bit of their time on it or to brand extracurricular activities with the company name for publicity. The test for open source is if it keeps getting maintained and supported for years. That only happens when the project is a core business effort, has some direct means of support (e.g. dual licensing or SaaS), or happens to be one of the few genuinely volunteer driven large scale open source projects.
scrappyjoe 7y ago
Huh? The last commit to the H3 github repo was 3 days ago. In what sense is it abandoned? Genuinely interested as we are considering using it as a core library.
ajfriend 7y ago
We're definitely still working on H3. We just got a nice, new domain: h3geo.org We're also basically done with a new Python wrapper written in Cython. https://github.com/uber/h3-py/tree/cython https://github.com/uber/h3-py/tree/cython We could probably use some help with the last step of packaging, if anyone is interested!
excerionsforte 7y ago
https://github.com/facebookarchive https://github.com/facebookarchive - 11 Pages of unsupported open source software. Beringei, a TSDB, (https://github.com/facebookarchive/beringei https://github.com/facebookarchive/beringei) in particular with what you are saying was a PR piece (https://engineering.fb.com/core-data/beringei-a-high-performance-time-series-storage-engine/ https://engineering.fb.com/core-data/beringei-a-high-perform...) since it was never really used by anyone outside of FB. I really don't see the negative part of free code that you can learn from and/or incorporate at all.
- Fellshard 7y ago
  Free code is good, yes. When you rely on the systems themselves, and do so with expectation of support from the originating company, your expectations will almost certainly be broken. I think that's the simplest takeaway - not to run away from any open-sourced project, but to take into proper consideration if/how they plan on supporting the tool, and how much you would be capable of adapting and owning yourself if the worst happened.
  - excerionsforte 7y ago
    Yeah exactly. If one wants support they can pay for it i.e. SaaS if available. If an open source project has not created a contract with any user then there is no guarantee of support. I don't believe any company creates a contract with users automatically because they made source code available. That is unsustainable. Chronosphere is the SaaS part for M3DB in this case. The negativity around someone open sourcing code for PR is nuts especially when all of the code is available. I love reading the code and getting ideas about how things work.
iblaine 7y ago
Open sourcing projects is the new merit badge for engineers. But at least there’s more good than bad from it. Hudi is at least one Uber project I can point to off the top of my head that is a great idea.
parentheses 7y ago
The issue here is not the company but the fact that the owners of the original library did not figure out how to “disown” the library. Open sourcing something is naturally more expensive than not. It’s seldom the case that impact to the community triggers contributions that outweigh that cost. The fallacy we hold is that companies will prop up software that is open source for everyone to use despite the lacking community contribution. We as engineers should push ourselves to contribute when we find issues - rather than simply create tickets that represent work were want to have done for free. This is how open source software dies. There is a minority that does this.
throwaway5752 7y ago
Lots of good open source projects fail. But not every company is willing to open source code like this, though, and I'm very happy that Uber did so in this case. I get your frustration, but everyone should remember there are never any promises of support with open source software, regardless of how well supported it is at a particular time.

gtirloni 7y ago

It's opensource. Why should Uber give any guarantees? They are not in the business of selling software. Unless Uber is actively blocking contributions, it's not Uber's fault if no community formed around something they opensourced. As for this being a PR piece, they could have achieved the same with just a detailed blog post and no code. It looks like a expensive PR piece if they have to opensource work that took probably hundreds of development hours.

missosoup 7y ago
I agree with everything you say. But without any certainty around the roadmap, support, and longterm commitment by Uber to maintain these projects, they're nothing more than interesting repos amongst a sea of interesting repos. The way Uber brands them suggests that they're suitable for use in production environments, but so far that hasn't been the case with anything they open sourced outside a narrow envelope that resembles their own operating model. Maybe this project will set a new trend, but so far nothing they put out gained any traction or became suitable for general purpose production use. In that regard, H3 and their other projects have remained at the level of decent 'show HN' pieces rather than something you'd ever use professionally. In other words, marketing. Based on previous news coming out e.g. https://news.ycombinator.com/item?id=20931644 https://news.ycombinator.com/item?id=20931644 It seems like Uber had too big of an engineering department with too little work to do, so they started reinventing wheels. Which is cool if they're willing to support them in the long term, but so far that hasn't proven to be the case.
- mamon 7y ago
  Isn't that kind of the point of open-sourcing your internal tools? You don't want to be bothered with maintenance and support, so you're hoping that some anonymous volounteers will do that for you :)
  - cfors 7y ago
    Maybe I’m a cynic of all big corp companies but if you’ve ever worked with a big corp open source department that’s almost the entire point is to build PR for the engineering department. Same goes for tech blogs. These things will be PR pieces first, and valid production tools/frameworks second (mostly).
    - closeparen 7y ago
      In my experience software gets written in the first place for the usual internal reasons. Corporate or individual prestige may be the driving factor in open sourcing, though, rather than a genuine interest in having it used externally.
  - lopsidedBrain 7y ago
    Pretty much every successful open source project that people pay attention to is one that has had long-term support behind it. Linux, Mozilla, gcc, clang, git. Almost always, that support begins with the original author. Protects that don't do that are therefore unlikely to remain interesting for long.
- carlisle_ 7y ago
  >It seems like Uber had too big of an engineering department with too little work to do, so they started reinventing wheels. Which is cool if they're willing to support them in the long term, but so far that hasn't proven to be the case. Former Uber engineer here. I can assure you that while our engineering team was massive, there was anything but too little work. If anything most engineers were massively overtaxed. Whether or not the work we were undertaking was meritious and valuable is an entire branch of philosophy I'm pretty sure. Part of the struggle at big companies is that a lot of existing solutions just don't work. Let me use an example with chat. A few years ago Slack was evaluated as a replacement for HipChat, since Atlassian's outages had finally started affecting us during our own outages. Everybody wanted to go to Slack, but the cost of Slack was tremendously prohibitive and the state of the service then (as I was told) was such that it could not support a company of Uber's size. Tremendous effort would have been undertaken by Slack to support Uber and they didn't want to expend that effort for a single customer. This was late 2015 early 2016. There were tons of options, but ultimately an in-house chat software was created. At the time it seemed required to make our own highly reliable chat, considering how distributed engineering teams were. I think if you talk to anybody without the background of how chat evolved at Uber they would think the in-house chat project would have been a boondoggle. Not all over-scoped engineering projects are actually so noble. There was certainly a ton of "reinventing wheels" going on. There was significantly more "these problems are really hard and I only have bad solutions." Though if the result is ultimately, "nothing more than interesting repos amongst a sea of interesting repos" sign me up.
_jal 7y ago
> Why should Uber give any guarantees? First, "guarantee" is the wrong word to take too literally here. Depending on how you want to look at it, there are no guarantees, even with guarantees. But looked at more loosely, answering that is really Uber's responsibility. Why did they release it? If it is just a PR release, fire-and-forget works fine for that. If they want to see wider adoption outside of their firm, there are some fairly obvious things they should do to foster that. Sometimes you release just the right thing at just the right time and everyone else does your evangelism and support work for you, but it is much more normal for your next great thing to take a while to build a user base.
- tylerl 7y ago
  There are lots of reasons to open source internal software, and only a minority of them involve establishing a serious community and driving significant adoption. But the PR claim you're making isn't particularly credible. The ROI is abysmal if that's all you're after, and there are easier ways to get it. Based on your specific complaints, it sounds like your opinion doesn't matter in this case; you're not the audience. You want support and a predictable future: you're looking for a product, not for technology. This isn't a product, and it's not a platform. If instead you represented another company looking into solving this same problem yourself, and are looking at starting points, then you're the perfect audience. In that case, you'd have time and motivation to contact the developers directly rather than gripe on HN. You'd be less interested in whether there was an organized community, and more interested in how to directly influence the roadmap. You'd care about what the code looks like, how they solved Problem X and Problem Y, that kind of thing.
deleted 7y ago
[deleted]
ForHackernews 7y ago
Everything you say is true, but tossing useless code releases over the wall isn't really participating in the open source community, either. It looks to me like maybe their engineers internally are fans of the idea of "open source", and the PR department is happy to try and get some good press out of it, but the company culture isn't really set up to develop in public or maintain these things they've nominally "released". Sadly, this isn't unusual among tech companies, but it'd be more obvious what's happening if they just put up a bare-bones FTP with a README: "Here's some code under <LICENSE>. Use it at your own risk."
papito 7y ago
Even I have half a mind about releasing something that I cannot commit to for a little bit - the initial bugfix stage, at least. I would say a giant like Uber hurts themselves more in terms of PR when not being conservative enough about putting source out there. People inevitably gravitate towards big player "open sauce" as it implies some commitment. A company should first put their own system through hell and decide that "yeah, this is good, we are sticking with this", before luring people to use it.
grogenaut 7y ago
Open source for the steward is limping the project along is the worst type of open source because the steward usually isn't going to make any real decisions around the project the people are reticent to fork it and drive it because there is a Steward doing some activity. Selenium was in the state for years.
Udik 7y ago
> They are not in the business of selling software. But they are probably in the business of selling themselves as a software/ tech company. People value tech companies, so you'd better be one, even if you do taxi services, produce and distribute tv series, or rent office space.

reichardt 7y ago

Why do you consider H3 to be on life support? It's basically a finished spec with actively developed implementations. https://github.com/uber/h3 https://github.com/uber/h3

deleted 7y ago
[deleted]
ForHackernews 7y ago
So is Gopher, but almost nobody uses it.

richieartoul 7y ago

Chronosphere, a startup founded by two of the early M3 engineers, just raised 11 million dollars to build a monitoring platform based around M3DB: https://techcrunch.com/2019/11/05/chronosphere-launches-with-11m-series-a-to-build-scalable-cloud-native-monitoring-tool https://techcrunch.com/2019/11/05/chronosphere-launches-with... Uber also uses M3DB extensively internally and the project is nowhere near being abandoned or on life support: https://github.com/m3db/m3/commits/master https://github.com/m3db/m3/commits/master

bfung 7y ago
Yep, another way to pre-market/pre-signal to investors. A pretty common story these days: 1. I built an X to solve Unicorn U’s problem. 2. Open source it, give talks on it 3. Leave Unicorn U and start a company based on X 4. ... 5. Profit(???) Also, debatable whether or not Unicorn U actually needed a freshly built X instead of using existing tools/tech.
- sieabahlpark 7y ago
  Seems genius imo
StreamBright 7y ago
>> While the founders, CEO Martin Mao and CTO Rob Skillington, were working at Uber, they recognized a gap in the monitoring industry, particularly around cloud-native technologies like containers and microservices What is the actual gap that is not addressed with one or all of the following? - Prometheus / Grafana [1] - Datadog [2] - Cloudwatch [3] 1. https://docs.docker.com/config/thirdparty/prometheus/ https://docs.docker.com/config/thirdparty/prometheus/ 2. https://www.datadoghq.com/blog/introducing-live-container-monitoring/ https://www.datadoghq.com/blog/introducing-live-container-mo... 3. https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-metrics-EKS.html https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitori...
- opsunit 7y ago
  Wavefront: https://www.wavefront.com/ https://www.wavefront.com/

api 7y ago

Many large companies push this stuff out for publicity and recruitment. Sometimes employees are encouraged to spend a little bit of their time on it or to brand extracurricular activities with the company name for publicity. The test for open source is if it keeps getting maintained and supported for years. That only happens when the project is a core business effort, has some direct means of support (e.g. dual licensing or SaaS), or happens to be one of the few genuinely volunteer driven large scale open source projects.

scrappyjoe 7y ago

Huh? The last commit to the H3 github repo was 3 days ago. In what sense is it abandoned? Genuinely interested as we are considering using it as a core library.

ajfriend 7y ago

We're definitely still working on H3. We just got a nice, new domain: h3geo.org We're also basically done with a new Python wrapper written in Cython. https://github.com/uber/h3-py/tree/cython https://github.com/uber/h3-py/tree/cython We could probably use some help with the last step of packaging, if anyone is interested!

excerionsforte 7y ago

https://github.com/facebookarchive https://github.com/facebookarchive - 11 Pages of unsupported open source software. Beringei, a TSDB, (https://github.com/facebookarchive/beringei https://github.com/facebookarchive/beringei) in particular with what you are saying was a PR piece (https://engineering.fb.com/core-data/beringei-a-high-performance-time-series-storage-engine/ https://engineering.fb.com/core-data/beringei-a-high-perform...) since it was never really used by anyone outside of FB. I really don't see the negative part of free code that you can learn from and/or incorporate at all.

Fellshard 7y ago
Free code is good, yes. When you rely on the systems themselves, and do so with expectation of support from the originating company, your expectations will almost certainly be broken. I think that's the simplest takeaway - not to run away from any open-sourced project, but to take into proper consideration if/how they plan on supporting the tool, and how much you would be capable of adapting and owning yourself if the worst happened.
- excerionsforte 7y ago
  Yeah exactly. If one wants support they can pay for it i.e. SaaS if available. If an open source project has not created a contract with any user then there is no guarantee of support. I don't believe any company creates a contract with users automatically because they made source code available. That is unsustainable. Chronosphere is the SaaS part for M3DB in this case. The negativity around someone open sourcing code for PR is nuts especially when all of the code is available. I love reading the code and getting ideas about how things work.

iblaine 7y ago

Open sourcing projects is the new merit badge for engineers. But at least there’s more good than bad from it. Hudi is at least one Uber project I can point to off the top of my head that is a great idea.

parentheses 7y ago

The issue here is not the company but the fact that the owners of the original library did not figure out how to “disown” the library. Open sourcing something is naturally more expensive than not. It’s seldom the case that impact to the community triggers contributions that outweigh that cost. The fallacy we hold is that companies will prop up software that is open source for everyone to use despite the lacking community contribution. We as engineers should push ourselves to contribute when we find issues - rather than simply create tickets that represent work were want to have done for free. This is how open source software dies. There is a minority that does this.

throwaway5752 7y ago

Lots of good open source projects fail. But not every company is willing to open source code like this, though, and I'm very happy that Uber did so in this case. I get your frustration, but everyone should remember there are never any promises of support with open source software, regardless of how well supported it is at a particular time.