6 ms·

Is it just me or is every big release from Uber just a custom rewrite of an existing technology? It seems their engineering department has a large not-invented-

by spyspy 8y ago

Is it just me or is every big release from Uber just a custom rewrite of an existing technology? It seems their engineering department has a large not-invented-here attitude. I could be wrong - they're certainly large enough to have custom requirements that aren't met with what's on the market but the pattern is just becoming suspect.

throwawaymath 8y ago
I wouldn't say this is a rewrite of existing technology. They borrowed concepts from other well-known open source projects, but this is substantially a wrapper around Mesos, not a competing project. The technical overview of Peloton[1] is more clear about this than the open source announcement, which is what's featured here. _________________ 1. https://eng.uber.com/peloton/ https://eng.uber.com/peloton/
- sciurus 8y ago
  Thanks for sharing! That is definitely a better link than the one posted.
pj_mukh 8y ago
Anyone who had to take an Uber after they switched away from Google Maps and onto their in-house half-baked mapping/navigation solution knows this is a huge problem. I'm assuming like all mapping solutions it'll get better but for now, it's just full of bad routes, over-optimizing turns, out-of-date detours (for MONTHS!) and non-sensical U-turns
- 0xDEFC0DE 8y ago
  Not to defend Uber/NIH syndrome, but if Google wanted to specifically charge Uber more for maps because of usage, could they legally do that? Maybe the rewrites are risk management?
  - giancarlostoro 8y ago
    Google increased maps cost for everybody though, not just Uber. https://gadgets.ndtv.com/apps/features/google-maps-apis-new-pricing-impact-1907242 https://gadgets.ndtv.com/apps/features/google-maps-apis-new-...
    - 0xDEFC0DE 8y ago
      I'm asking whether only Uber could be charged more.
  - anitil 8y ago
    You've hit the nail on the head. Uber can't risk being one management decision from Google away from shutting down.
    - jen20 8y ago
      Were they not recently embroiled in a lawsuit, too?
    - baroffoos 8y ago
      They are already forced to be like that by being in the app store.
  - Nextgrid 8y ago
    They could just use a third-party (and at their scale, they can definitely negotiate a custom deal where they feed back usage data to improve the third-party’s service) or even use open source solutions like OpenStreetMaps. Even the latter (with the overhead of hosting it themselves) makes total sense at their scale.
    - 0xDEFC0DE 8y ago
      That'd be in a contract though, so what if Google decided to not renew and only let Uber know like a month before the contract ended? If these questions seem dumb it's because I know pretty little about legal battles in software.
warp_factor 8y ago
I think you are right. I think that what is going there is also a bit political. They started to grow their Engineering department so fast that they need to justify the headcounts now. So each team is trying to invent new projects all the time. Anecdotally, this was partially confirmed to me by a friend working there. I said this before, but I still cannot understand why a service like Uber need so many engineers in the backend (multiple thousands). It is a complex distributed application, but nowhere near the scale or complexity of a Facebook or Google.
- deleted 8y ago
  [deleted]
- jrockway 8y ago
  I appreciate their efforts on open-source projects. Jaeger is wonderful and the effort they put into both making something great, and supporting the open standards (Opentracing and the legacy Zipkin propagation) is greatly appreciated. I recently had the need to write a service in Typescript (most everything else is Go), and I felt very at home using the Jaeger node bindings. It felt like I wasn't losing any features for using a less-popular language and everything just worked. Sure, they just reinvented Dapper from Google... but unlike Dapper I can download and use Jaeger. That counts for a lot. Do I use their ride sharing service? Nope. But I do like their open source projects.
- theon144 8y ago
  >I said this before, but I still cannot understand why a service like Uber need so many engineers in the backend (multiple thousands). It is a complex distributed application, but nowhere near the scale or complexity of a Facebook or Google. Thank you so much, I thought I was going crazy. I understand the demands of running a service on the level Uber has, but well, for instance I can't imagine what kind of computational workload / infrastructure requirements would make developing your own resource scheduler a reasonable option - for a Taxi app? With non-essential (to the core product) machine learning? Forgive me if I'm ignorant, but what exactly does Uber engineering team do? edit: On their blog I was able to find that they namely "forecast rider demand", from a relatively small [0] article - that is, comapred to the article [1] about what essentially is "just" data visualization, which doesn't help my confusion much. 0 - https://eng.uber.com/neural-networks/ https://eng.uber.com/neural-networks/ 1 - https://eng.uber.com/maze/ https://eng.uber.com/maze/
ummonk 8y ago
Their urban gps locating technology based off satellite masking is pretty unique as far as I know.
jjoonathan 8y ago
Does anyone know an open source Uber Michelangelo?
- eggie5 8y ago
  there's pipeline.ai, airbnb said they'd open source theirs this year and also, TFX suite is getting there. Platforms are becoming popular
mindcrime 8y ago
Is it just me or is every big release from Uber just a custom rewrite of an existing technology? I'm a little sad that this is the top comment here. I mean, maybe you're right. But so what? Some people find this useful, and some won't. Same as anything else. At the end of the day, every line of code added to the world's pool of OSS code is a Good Thing™ as far as I'm concerned. Even if it's something I personally don't have a use for. I think we should encourage companies to release code as open source, and give Uber at least some small measure of "props" for the stuff they release. Maybe none of their stuff is a game changer like Linux, but it doesn't need to be.
malandrew 8y ago
The problem with a comment like this is that it only takes into account when something was released publicly and not when the problem was first worked on and implemented internally. Many internal projects that eventually become open-source often are not NIH projects because when the project was proposed their may have been no public open source projects or at least none that is mature enough. Even if something exists but it still in its early stages, it presents a lot of risk because your company isn't in the driver seat building and maintaining it. Claiming something is NIH based on when it first became polished enough to be open-source ignores all the history behind the state of the world when a project was first worked on.

by spyspy 8y ago

throwawaymath 8y ago

I wouldn't say this is a rewrite of existing technology. They borrowed concepts from other well-known open source projects, but this is substantially a wrapper around Mesos, not a competing project. The technical overview of Peloton[1] is more clear about this than the open source announcement, which is what's featured here. _________________ 1. https://eng.uber.com/peloton/ https://eng.uber.com/peloton/

sciurus 8y ago
Thanks for sharing! That is definitely a better link than the one posted.

pj_mukh 8y ago

Anyone who had to take an Uber after they switched away from Google Maps and onto their in-house half-baked mapping/navigation solution knows this is a huge problem. I'm assuming like all mapping solutions it'll get better but for now, it's just full of bad routes, over-optimizing turns, out-of-date detours (for MONTHS!) and non-sensical U-turns

0xDEFC0DE 8y ago
Not to defend Uber/NIH syndrome, but if Google wanted to specifically charge Uber more for maps because of usage, could they legally do that? Maybe the rewrites are risk management?
- giancarlostoro 8y ago
  Google increased maps cost for everybody though, not just Uber. https://gadgets.ndtv.com/apps/features/google-maps-apis-new-pricing-impact-1907242 https://gadgets.ndtv.com/apps/features/google-maps-apis-new-...
  - 0xDEFC0DE 8y ago
    I'm asking whether only Uber could be charged more.
- anitil 8y ago
  You've hit the nail on the head. Uber can't risk being one management decision from Google away from shutting down.
  - jen20 8y ago
    Were they not recently embroiled in a lawsuit, too?
  - baroffoos 8y ago
    They are already forced to be like that by being in the app store.
- Nextgrid 8y ago
  They could just use a third-party (and at their scale, they can definitely negotiate a custom deal where they feed back usage data to improve the third-party’s service) or even use open source solutions like OpenStreetMaps. Even the latter (with the overhead of hosting it themselves) makes total sense at their scale.
  - 0xDEFC0DE 8y ago
    That'd be in a contract though, so what if Google decided to not renew and only let Uber know like a month before the contract ended? If these questions seem dumb it's because I know pretty little about legal battles in software.

warp_factor 8y ago

I think you are right. I think that what is going there is also a bit political. They started to grow their Engineering department so fast that they need to justify the headcounts now. So each team is trying to invent new projects all the time. Anecdotally, this was partially confirmed to me by a friend working there. I said this before, but I still cannot understand why a service like Uber need so many engineers in the backend (multiple thousands). It is a complex distributed application, but nowhere near the scale or complexity of a Facebook or Google.

deleted 8y ago
[deleted]
jrockway 8y ago
I appreciate their efforts on open-source projects. Jaeger is wonderful and the effort they put into both making something great, and supporting the open standards (Opentracing and the legacy Zipkin propagation) is greatly appreciated. I recently had the need to write a service in Typescript (most everything else is Go), and I felt very at home using the Jaeger node bindings. It felt like I wasn't losing any features for using a less-popular language and everything just worked. Sure, they just reinvented Dapper from Google... but unlike Dapper I can download and use Jaeger. That counts for a lot. Do I use their ride sharing service? Nope. But I do like their open source projects.
theon144 8y ago
>I said this before, but I still cannot understand why a service like Uber need so many engineers in the backend (multiple thousands). It is a complex distributed application, but nowhere near the scale or complexity of a Facebook or Google. Thank you so much, I thought I was going crazy. I understand the demands of running a service on the level Uber has, but well, for instance I can't imagine what kind of computational workload / infrastructure requirements would make developing your own resource scheduler a reasonable option - for a Taxi app? With non-essential (to the core product) machine learning? Forgive me if I'm ignorant, but what exactly does Uber engineering team do? edit: On their blog I was able to find that they namely "forecast rider demand", from a relatively small [0] article - that is, comapred to the article [1] about what essentially is "just" data visualization, which doesn't help my confusion much. 0 - https://eng.uber.com/neural-networks/ https://eng.uber.com/neural-networks/ 1 - https://eng.uber.com/maze/ https://eng.uber.com/maze/

ummonk 8y ago

Their urban gps locating technology based off satellite masking is pretty unique as far as I know.

jjoonathan 8y ago

Does anyone know an open source Uber Michelangelo?

eggie5 8y ago
there's pipeline.ai, airbnb said they'd open source theirs this year and also, TFX suite is getting there. Platforms are becoming popular

mindcrime 8y ago

Is it just me or is every big release from Uber just a custom rewrite of an existing technology? I'm a little sad that this is the top comment here. I mean, maybe you're right. But so what? Some people find this useful, and some won't. Same as anything else. At the end of the day, every line of code added to the world's pool of OSS code is a Good Thing™ as far as I'm concerned. Even if it's something I personally don't have a use for. I think we should encourage companies to release code as open source, and give Uber at least some small measure of "props" for the stuff they release. Maybe none of their stuff is a game changer like Linux, but it doesn't need to be.

malandrew 8y ago

The problem with a comment like this is that it only takes into account when something was released publicly and not when the problem was first worked on and implemented internally. Many internal projects that eventually become open-source often are not NIH projects because when the project was proposed their may have been no public open source projects or at least none that is mature enough. Even if something exists but it still in its early stages, it presents a lot of risk because your company isn't in the driver seat building and maintaining it. Claiming something is NIH based on when it first became polished enough to be open-source ignores all the history behind the state of the world when a project was first worked on.