4 ms·

Is there any evidence that the vast amounts of clicks and user interactions companies have been collecting are worth anything at all? Let’s say I deleted every

Is there any evidence that the vast amounts of clicks and user interactions companies have been collecting are worth anything at all? Let’s say I deleted every time series whose Y axis isn’t measuring US dollars in every tech company’s database everywhere. Maybe for all those time series you just store the most recent value. Describe to me what would be lost. You’re onto something but you’re not going far enough! Most, if not all, historic metadata, analytics and behavioral data collection - when it is not measuring literal dollar amounts - is completely worthless.

mgraczyk 6y ago
This is completely wrong, and nobody who works with data at any scale could possibly believe anything like this. We literally run long term A/B tests with thousands of variations of what you're describing. The purpose of these tests is to measure the effect of losing some data. The tests show (to nobody's surprise) that each piece of data is useful. These tests tell us exactly how useful each piece of data is. Honestly when I read comments like this I have to wonder, do you really believe that thousands of companies spend trillions of dollars a year for something that doesn't work? Maybe talk to somebody who works on this stuff a bit?
- abernard1 6y ago
  > Honestly when I read comments like this I have to wonder, do you really believe that thousands of companies spend trillions of dollars a year for something that doesn't work? Joking/not-joking. Have you ever been to the Bay Area? Yes. Emphatically yes it is the case companies spend trillions of dollars unnecessarily. We've seen this with people who didn't know how to build microservices and farcical "LMNOP" [1] type services that might as well be a joke. We've seen it with gigantically-valued unicorns that over-engineered tons of crap and hired too many people and still can't make a profit. We've seen it with CMOs and massively overpriced marketing technology because budgets and statuses are related. We'll see it with tons more iterations of this exact same affluenza. The history of our industry is that the margins on software are so good that people can afford to do crazy nonsense. [1] https://www.youtube.com/watch?reload=9&v=y8OnoxKotPQ https://www.youtube.com/watch?reload=9&v=y8OnoxKotPQ
  - dcgudeman 6y ago
    Just so I don't misunderstand your point: "Yes, companies without coordinating are collectively spending vast sums of precious resources on a product/service, BUT they are all misguided and wasting their money because look at these unrelated events where individuals misplaced their efforts/ money". Is that correct?
    - abernard1 6y ago
      > wasting their money because look at these unrelated events where individuals misplaced their efforts/ money They're not unrelated. They're all related to fat 60-80% profit margins on SaaS. And no coordination is necessary to spend money on silly, make-work activities if you have margins like that. Software has extremely low, borderline zero, variable costs. A lot of these companies that spend money on examples like I've given probably could hire nobody at all and still have crazy growth because of the unit economics. (Not coincidentally, the companies most in want of "Big Data" solutions tend to be past this point.) I can get 1M QPS on a silly Aurora setup with replicas. Best tool for all jobs: no. But don't tell me that dollar-for-dollar a data architecture with like 25 different components is dramatically superior to an OLTP db, OLAP offline store + batch jobs, and a streaming system.
  - mgraczyk 6y ago
    Yeah I mean this misses the main argument I'm making. I have vast amounts of firsthand evidence from randomized controlled trials that non-financial data can be used to create value. This is enough evidence for anyone in the industry. Presumably the commenter doesn't have access to this evidence. Instead he has to rely on other heuristics, like the weaker argument that companies spend trillions on data and analytics. Companies sometimes waste money, and maybe microservices are an example of this. But companies collectively spend 3-4 orders of magnitude more money on "all data that does not have USD units" than on microservices, so the commenter should take that as strong evidence that data can be used to create value.
    - abernard1 6y ago
      His quote was: > Is there any evidence that the vast amounts of clicks and user interactions companies have been collecting are worth anything at all? And for the huge majority of companies, it is not. Even many 1B+ dollar companies. Most value produced by businesses still could exist with boring ETL or BI concepts that have been around forever, because the hardware powering it is so fast. Many if not most of those businesses probably would be better off. So yes, companies blow a lot of money on stuff with questionable ROI. I don't discount your experiences and that there are cases where the complexity might have a payoff. But honestly, we've seen periods of excess complexity and waste in software before: it's the norm.
- gfodor 6y ago
  yes consider RPC: CORBA EJBs SOAP Microservices <--- You are here
- an_opabinia 6y ago
  I’m not trying to get into a flame war with you, there’s no reason to be hostile. It sounds like you “work on this stuff a bit,” you’re welcome to share concrete examples, it would be really interesting! I think it’s an intriguing thought exercise. For example, does one need the entire history of interactions with e.g., an Instagram post, or just aggregated measurements? I’m not like, against measuring. Just against warehousing of non financial timeseries.
  - mgraczyk 6y ago
    Yeah don't mean to argue, here are some examples. It's difficult for any company to be competitive at scale without sufficient logging to support all of these things. * Timestamps of events related to content loading and rendering. This is crucial for debugging and improving load times. * Backfilling aggregated data so that ML models can be trained without waiting weeks for new streaming aggregation. * Answering product questions of almost any kind that weren't asked when logging was built. Concrete example from my recent experience, you may want to know how often people like a post then later look at comments, vs look at comments then later like a post. This gives you information about cause and effect.
    - abernard1 6y ago
      The first one doesn't even need much historical data. Unless you have some very unoptimized periodic jobs, the last few days or something is plenty. The second can be done simply on something like Dynamo, CosmosDB, or your cloud-hosted NoSQL of choice. Heck, it can even be done on Aurora or vanilla Postgres + partitioning if it's <64TB. The third can be done with any off the shelf cloud data warehouse software, at many petabyte scale. And even then, I'm sorry, but I just don't believe you that the product clicks over some large timeframe are historically relevant if your software and UI changes often. All of these things mentioned have had extremely simple, boring solutions at petabyte scale for >10 years, and in some cases more than that. If you add a batch workflow manager and a streaming solution like Spark, that's like 3-4 technologies total to cover all these cases (and many more!)
Supermancho 6y ago
> Is there any evidence that the vast amounts of clicks and user interactions companies have been collecting are worth anything at all? Yes. Every advertising platform ever uses this information. In Europe, you have to have regulation that makes account costing (what the US might call forensic accounting) possible. The presentations on A/B tests by FANG companies might also interest you. They are on Youtube.

mgraczyk 6y ago

This is completely wrong, and nobody who works with data at any scale could possibly believe anything like this. We literally run long term A/B tests with thousands of variations of what you're describing. The purpose of these tests is to measure the effect of losing some data. The tests show (to nobody's surprise) that each piece of data is useful. These tests tell us exactly how useful each piece of data is. Honestly when I read comments like this I have to wonder, do you really believe that thousands of companies spend trillions of dollars a year for something that doesn't work? Maybe talk to somebody who works on this stuff a bit?

abernard1 6y ago
> Honestly when I read comments like this I have to wonder, do you really believe that thousands of companies spend trillions of dollars a year for something that doesn't work? Joking/not-joking. Have you ever been to the Bay Area? Yes. Emphatically yes it is the case companies spend trillions of dollars unnecessarily. We've seen this with people who didn't know how to build microservices and farcical "LMNOP" [1] type services that might as well be a joke. We've seen it with gigantically-valued unicorns that over-engineered tons of crap and hired too many people and still can't make a profit. We've seen it with CMOs and massively overpriced marketing technology because budgets and statuses are related. We'll see it with tons more iterations of this exact same affluenza. The history of our industry is that the margins on software are so good that people can afford to do crazy nonsense. [1] https://www.youtube.com/watch?reload=9&v=y8OnoxKotPQ https://www.youtube.com/watch?reload=9&v=y8OnoxKotPQ
- dcgudeman 6y ago
  Just so I don't misunderstand your point: "Yes, companies without coordinating are collectively spending vast sums of precious resources on a product/service, BUT they are all misguided and wasting their money because look at these unrelated events where individuals misplaced their efforts/ money". Is that correct?
  - abernard1 6y ago
    > wasting their money because look at these unrelated events where individuals misplaced their efforts/ money They're not unrelated. They're all related to fat 60-80% profit margins on SaaS. And no coordination is necessary to spend money on silly, make-work activities if you have margins like that. Software has extremely low, borderline zero, variable costs. A lot of these companies that spend money on examples like I've given probably could hire nobody at all and still have crazy growth because of the unit economics. (Not coincidentally, the companies most in want of "Big Data" solutions tend to be past this point.) I can get 1M QPS on a silly Aurora setup with replicas. Best tool for all jobs: no. But don't tell me that dollar-for-dollar a data architecture with like 25 different components is dramatically superior to an OLTP db, OLAP offline store + batch jobs, and a streaming system.
- mgraczyk 6y ago
  Yeah I mean this misses the main argument I'm making. I have vast amounts of firsthand evidence from randomized controlled trials that non-financial data can be used to create value. This is enough evidence for anyone in the industry. Presumably the commenter doesn't have access to this evidence. Instead he has to rely on other heuristics, like the weaker argument that companies spend trillions on data and analytics. Companies sometimes waste money, and maybe microservices are an example of this. But companies collectively spend 3-4 orders of magnitude more money on "all data that does not have USD units" than on microservices, so the commenter should take that as strong evidence that data can be used to create value.
  - abernard1 6y ago
    His quote was: > Is there any evidence that the vast amounts of clicks and user interactions companies have been collecting are worth anything at all? And for the huge majority of companies, it is not. Even many 1B+ dollar companies. Most value produced by businesses still could exist with boring ETL or BI concepts that have been around forever, because the hardware powering it is so fast. Many if not most of those businesses probably would be better off. So yes, companies blow a lot of money on stuff with questionable ROI. I don't discount your experiences and that there are cases where the complexity might have a payoff. But honestly, we've seen periods of excess complexity and waste in software before: it's the norm.
gfodor 6y ago
yes consider RPC: CORBA EJBs SOAP Microservices <--- You are here
an_opabinia 6y ago
I’m not trying to get into a flame war with you, there’s no reason to be hostile. It sounds like you “work on this stuff a bit,” you’re welcome to share concrete examples, it would be really interesting! I think it’s an intriguing thought exercise. For example, does one need the entire history of interactions with e.g., an Instagram post, or just aggregated measurements? I’m not like, against measuring. Just against warehousing of non financial timeseries.
- mgraczyk 6y ago
  Yeah don't mean to argue, here are some examples. It's difficult for any company to be competitive at scale without sufficient logging to support all of these things. * Timestamps of events related to content loading and rendering. This is crucial for debugging and improving load times. * Backfilling aggregated data so that ML models can be trained without waiting weeks for new streaming aggregation. * Answering product questions of almost any kind that weren't asked when logging was built. Concrete example from my recent experience, you may want to know how often people like a post then later look at comments, vs look at comments then later like a post. This gives you information about cause and effect.
  - abernard1 6y ago
    The first one doesn't even need much historical data. Unless you have some very unoptimized periodic jobs, the last few days or something is plenty. The second can be done simply on something like Dynamo, CosmosDB, or your cloud-hosted NoSQL of choice. Heck, it can even be done on Aurora or vanilla Postgres + partitioning if it's <64TB. The third can be done with any off the shelf cloud data warehouse software, at many petabyte scale. And even then, I'm sorry, but I just don't believe you that the product clicks over some large timeframe are historically relevant if your software and UI changes often. All of these things mentioned have had extremely simple, boring solutions at petabyte scale for >10 years, and in some cases more than that. If you add a batch workflow manager and a streaming solution like Spark, that's like 3-4 technologies total to cover all these cases (and many more!)

Supermancho 6y ago

> Is there any evidence that the vast amounts of clicks and user interactions companies have been collecting are worth anything at all? Yes. Every advertising platform ever uses this information. In Europe, you have to have regulation that makes account costing (what the US might call forensic accounting) possible. The presentations on A/B tests by FANG companies might also interest you. They are on Youtube.