3 ms·

Great, another service that ex googlers will advocate for at startups with 10 customers and 2 Mb/day transfers. Because it scales!

xnx 2y ago
I'm not sure ex Googlers are even necessary for ill suited technologies to be used. There definitely weren't enough ex-Googlers to explain the spread of microservices or NoSQL. Reading about it on Hacker News might be enough.
- pjmlp 2y ago
  Microservices are just SUN RPC, CORBA, DCOM, EJBs, .NET Remoting, SOA, WCF, rebranded for a newer generation, which apparently hasn't learnt much about them. Hence now the modular monolith hype, as they rediscover programming languages have modules, and placing a network in the middle doesn't sort out spaghetti dependencies!
  - thecleaner 2y ago
    Its applicable if the services have different teams and different infra requirements. Its hard to create a ton of sphaghetti when integration is atleast a network hop away. There's some friction to being dependent on too much stuff.
    - liveoneggs 2y ago
      https://en.wikipedia.org/wiki/Conway%27s_law https://en.wikipedia.org/wiki/Conway%27s_law
    - pjmlp 2y ago
      Tell me you never wrote a call graph across distributed services architecture, without telling me. At least now Open Telemetry based tools can render those in beautiful ways.
    - nottorp 2y ago
      > There's some friction to being dependent on too much stuff. You ... think? I think there's a promotion in that in some organizations...
the_broken 2y ago
Well it definitely scales, but also has a constant cost of entry, which probably would make it unsuitable for "small startups" - all details are in the paper. However, if you read through the bibliography, you might notice that every so called "hyperscaler" and many network operators of similar scale (e.g. academia) uses something similar.
- nottorp 2y ago
  > but also has a constant cost of entry, which probably would make it unsuitable for "small startups" The cost of entry will be worth it just in case the startup turns into the next Google overnight :)
- aseipp 2y ago
  > all details are in the paper. > However, if you read Yeah, no, that's not going to happen on this site. Why would people waste their time actually reading the paper? That takes too much effort and that's time you could have spent getting internet upvote points. That's why it's much better to post idiotic dismissals which have zero substance or value (and have been regurgitated 1000 times, reminiscent of when a toddler learns a new word).
  - nottorp 2y ago
    What do the implementation details have to do with the general characteristics of any developer/backend software coming from google? They're always designed for infinite scaling, they always have trade offs like the initial setup costs, they always don't belong in a small operation. There are always people with a fetish for those solutions because it feels good to have the potential to handle "big data".
chubot 2y ago
It does remind me bit of "Big Data is Dead", which I believe was written by someone who worked on BigQuery (external version of Dremel), and then realized it doesn't actually add much value for customers: Big Data is Dead (2023) - https://news.ycombinator.com/item?id=34694926 https://news.ycombinator.com/item?id=34694926 My opinion is that Google culture enforces scale because it can lead to doing something competitors can't, or if you want to put it less charitably, "monopoly" For example, there are stories YouTube being a few months from falling over when it was acquired, and being saved by Google scale At the time, I do think it was true that few other companies could scale to what YouTube users demanded, or at least absorb that cost --- So the company has a culture of scale, for sure, because it has business value FOR GOOGLE I remember Marissa Mayer sent out "product guidance", and said that technical advantage is one of the criteria, i.e. being able to do something that a competitor can't do The market is very hostile -- profits are eroded away by competition. But yeah for most companies, like approximately all Google Cloud customers, scale is not the reason they exist. The product is the reason they exist By definition, there can only be one company that uses scale as a competitive weapon :) i.e. there is only one "most scalable" company Of course that's not strictly true, because there are different dimensions of scale, but it's perhaps a useful way of thinking about why Google culture is obsessed with scale. And conversely, why Google seemingly ignores and sunsets products that don't scale.
- nasretdinov 2y ago
  Yeah, I think people just don't fully realise how powerful hardware has become. You don't need complicated tools if you need to transfer even significantly large amounts of data, since you can have hundreds of gigabits of network capacity on a single server, so unless you're truly large you probably can get away with using vanilla rsync for file transfers for example :).
  - waych 2y ago
    Managed async bulk transfers as-a-service seems relevant to anyone writing lots of data doing multisite including DR. You can easily have this problem and not have yet hit any other scale problems that are addressed by the alluded-to tools people love to hate (k8s, tf, containers etc). What is being managed here with Effingo is the relatively tiny links between sites, not the bandwidth on the fat local links. Bandwidth needs to be managed and prioritized across all the various copies that are happening across different services apps and teams simultaneously across the company, and across the systems themselves as aggregate for the links. Managing it gives you control on cost but also ensures optimal use within the constraints imposed and consistent and reliable handling of the copy use case.
JSDevOps 2y ago
I laughed

xnx 2y ago

I'm not sure ex Googlers are even necessary for ill suited technologies to be used. There definitely weren't enough ex-Googlers to explain the spread of microservices or NoSQL. Reading about it on Hacker News might be enough.

pjmlp 2y ago
Microservices are just SUN RPC, CORBA, DCOM, EJBs, .NET Remoting, SOA, WCF, rebranded for a newer generation, which apparently hasn't learnt much about them. Hence now the modular monolith hype, as they rediscover programming languages have modules, and placing a network in the middle doesn't sort out spaghetti dependencies!
- thecleaner 2y ago
  Its applicable if the services have different teams and different infra requirements. Its hard to create a ton of sphaghetti when integration is atleast a network hop away. There's some friction to being dependent on too much stuff.
  - liveoneggs 2y ago
    https://en.wikipedia.org/wiki/Conway%27s_law https://en.wikipedia.org/wiki/Conway%27s_law
  - pjmlp 2y ago
    Tell me you never wrote a call graph across distributed services architecture, without telling me. At least now Open Telemetry based tools can render those in beautiful ways.
  - nottorp 2y ago
    > There's some friction to being dependent on too much stuff. You ... think? I think there's a promotion in that in some organizations...

the_broken 2y ago

Well it definitely scales, but also has a constant cost of entry, which probably would make it unsuitable for "small startups" - all details are in the paper. However, if you read through the bibliography, you might notice that every so called "hyperscaler" and many network operators of similar scale (e.g. academia) uses something similar.

nottorp 2y ago
> but also has a constant cost of entry, which probably would make it unsuitable for "small startups" The cost of entry will be worth it just in case the startup turns into the next Google overnight :)
aseipp 2y ago
> all details are in the paper. > However, if you read Yeah, no, that's not going to happen on this site. Why would people waste their time actually reading the paper? That takes too much effort and that's time you could have spent getting internet upvote points. That's why it's much better to post idiotic dismissals which have zero substance or value (and have been regurgitated 1000 times, reminiscent of when a toddler learns a new word).
- nottorp 2y ago
  What do the implementation details have to do with the general characteristics of any developer/backend software coming from google? They're always designed for infinite scaling, they always have trade offs like the initial setup costs, they always don't belong in a small operation. There are always people with a fetish for those solutions because it feels good to have the potential to handle "big data".

chubot 2y ago

It does remind me bit of "Big Data is Dead", which I believe was written by someone who worked on BigQuery (external version of Dremel), and then realized it doesn't actually add much value for customers: Big Data is Dead (2023) - https://news.ycombinator.com/item?id=34694926 https://news.ycombinator.com/item?id=34694926 My opinion is that Google culture enforces scale because it can lead to doing something competitors can't, or if you want to put it less charitably, "monopoly" For example, there are stories YouTube being a few months from falling over when it was acquired, and being saved by Google scale At the time, I do think it was true that few other companies could scale to what YouTube users demanded, or at least absorb that cost --- So the company has a culture of scale, for sure, because it has business value FOR GOOGLE I remember Marissa Mayer sent out "product guidance", and said that technical advantage is one of the criteria, i.e. being able to do something that a competitor can't do The market is very hostile -- profits are eroded away by competition. But yeah for most companies, like approximately all Google Cloud customers, scale is not the reason they exist. The product is the reason they exist By definition, there can only be one company that uses scale as a competitive weapon :) i.e. there is only one "most scalable" company Of course that's not strictly true, because there are different dimensions of scale, but it's perhaps a useful way of thinking about why Google culture is obsessed with scale. And conversely, why Google seemingly ignores and sunsets products that don't scale.

nasretdinov 2y ago
Yeah, I think people just don't fully realise how powerful hardware has become. You don't need complicated tools if you need to transfer even significantly large amounts of data, since you can have hundreds of gigabits of network capacity on a single server, so unless you're truly large you probably can get away with using vanilla rsync for file transfers for example :).
- waych 2y ago
  Managed async bulk transfers as-a-service seems relevant to anyone writing lots of data doing multisite including DR. You can easily have this problem and not have yet hit any other scale problems that are addressed by the alluded-to tools people love to hate (k8s, tf, containers etc). What is being managed here with Effingo is the relatively tiny links between sites, not the bandwidth on the fat local links. Bandwidth needs to be managed and prioritized across all the various copies that are happening across different services apps and teams simultaneously across the company, and across the systems themselves as aggregate for the links. Managing it gives you control on cost but also ensures optimal use within the constraints imposed and consistent and reliable handling of the copy use case.

JSDevOps 2y ago

I laughed