5 ms·

> Honestly when I read comments like this I have to wonder, do you really believe that thousands of companies spend trillions of dollars a year for something th

> Honestly when I read comments like this I have to wonder, do you really believe that thousands of companies spend trillions of dollars a year for something that doesn't work? Joking/not-joking. Have you ever been to the Bay Area? Yes. Emphatically yes it is the case companies spend trillions of dollars unnecessarily. We've seen this with people who didn't know how to build microservices and farcical "LMNOP" [1] type services that might as well be a joke. We've seen it with gigantically-valued unicorns that over-engineered tons of crap and hired too many people and still can't make a profit. We've seen it with CMOs and massively overpriced marketing technology because budgets and statuses are related. We'll see it with tons more iterations of this exact same affluenza. The history of our industry is that the margins on software are so good that people can afford to do crazy nonsense. [1] https://www.youtube.com/watch?reload=9&v=y8OnoxKotPQ https://www.youtube.com/watch?reload=9&v=y8OnoxKotPQ

dcgudeman 6y ago
Just so I don't misunderstand your point: "Yes, companies without coordinating are collectively spending vast sums of precious resources on a product/service, BUT they are all misguided and wasting their money because look at these unrelated events where individuals misplaced their efforts/ money". Is that correct?
- abernard1 6y ago
  > wasting their money because look at these unrelated events where individuals misplaced their efforts/ money They're not unrelated. They're all related to fat 60-80% profit margins on SaaS. And no coordination is necessary to spend money on silly, make-work activities if you have margins like that. Software has extremely low, borderline zero, variable costs. A lot of these companies that spend money on examples like I've given probably could hire nobody at all and still have crazy growth because of the unit economics. (Not coincidentally, the companies most in want of "Big Data" solutions tend to be past this point.) I can get 1M QPS on a silly Aurora setup with replicas. Best tool for all jobs: no. But don't tell me that dollar-for-dollar a data architecture with like 25 different components is dramatically superior to an OLTP db, OLAP offline store + batch jobs, and a streaming system.
mgraczyk 6y ago
Yeah I mean this misses the main argument I'm making. I have vast amounts of firsthand evidence from randomized controlled trials that non-financial data can be used to create value. This is enough evidence for anyone in the industry. Presumably the commenter doesn't have access to this evidence. Instead he has to rely on other heuristics, like the weaker argument that companies spend trillions on data and analytics. Companies sometimes waste money, and maybe microservices are an example of this. But companies collectively spend 3-4 orders of magnitude more money on "all data that does not have USD units" than on microservices, so the commenter should take that as strong evidence that data can be used to create value.
- abernard1 6y ago
  His quote was: > Is there any evidence that the vast amounts of clicks and user interactions companies have been collecting are worth anything at all? And for the huge majority of companies, it is not. Even many 1B+ dollar companies. Most value produced by businesses still could exist with boring ETL or BI concepts that have been around forever, because the hardware powering it is so fast. Many if not most of those businesses probably would be better off. So yes, companies blow a lot of money on stuff with questionable ROI. I don't discount your experiences and that there are cases where the complexity might have a payoff. But honestly, we've seen periods of excess complexity and waste in software before: it's the norm.
Sevii 6y ago
That video is great, I've actually worked on a project where we tried to federate customer databases across 3+ mergers. Started before I joined the company and after 3-4 iterations they shipped something (that worked) around 3 years later right before I left.

dcgudeman 6y ago

Just so I don't misunderstand your point: "Yes, companies without coordinating are collectively spending vast sums of precious resources on a product/service, BUT they are all misguided and wasting their money because look at these unrelated events where individuals misplaced their efforts/ money". Is that correct?

abernard1 6y ago
> wasting their money because look at these unrelated events where individuals misplaced their efforts/ money They're not unrelated. They're all related to fat 60-80% profit margins on SaaS. And no coordination is necessary to spend money on silly, make-work activities if you have margins like that. Software has extremely low, borderline zero, variable costs. A lot of these companies that spend money on examples like I've given probably could hire nobody at all and still have crazy growth because of the unit economics. (Not coincidentally, the companies most in want of "Big Data" solutions tend to be past this point.) I can get 1M QPS on a silly Aurora setup with replicas. Best tool for all jobs: no. But don't tell me that dollar-for-dollar a data architecture with like 25 different components is dramatically superior to an OLTP db, OLAP offline store + batch jobs, and a streaming system.

mgraczyk 6y ago

Yeah I mean this misses the main argument I'm making. I have vast amounts of firsthand evidence from randomized controlled trials that non-financial data can be used to create value. This is enough evidence for anyone in the industry. Presumably the commenter doesn't have access to this evidence. Instead he has to rely on other heuristics, like the weaker argument that companies spend trillions on data and analytics. Companies sometimes waste money, and maybe microservices are an example of this. But companies collectively spend 3-4 orders of magnitude more money on "all data that does not have USD units" than on microservices, so the commenter should take that as strong evidence that data can be used to create value.

abernard1 6y ago
His quote was: > Is there any evidence that the vast amounts of clicks and user interactions companies have been collecting are worth anything at all? And for the huge majority of companies, it is not. Even many 1B+ dollar companies. Most value produced by businesses still could exist with boring ETL or BI concepts that have been around forever, because the hardware powering it is so fast. Many if not most of those businesses probably would be better off. So yes, companies blow a lot of money on stuff with questionable ROI. I don't discount your experiences and that there are cases where the complexity might have a payoff. But honestly, we've seen periods of excess complexity and waste in software before: it's the norm.

Sevii 6y ago

That video is great, I've actually worked on a project where we tried to federate customer databases across 3+ mergers. Started before I joined the company and after 3-4 iterations they shipped something (that worked) around 3 years later right before I left.