3 ms·

My friend, do I have a rant lined up for you..: When I worked at a utility company, we had a purchased dataset on customers with things like inane, ridiculousl

My friend, do I have a rant lined up for you..: When I worked at a utility company, we had a purchased dataset on customers with things like inane, ridiculously specific features such as "average number of game consoles per household", on street level. Now, a. the data quality was absolute dog shit, and b. the idiotic amount of features made it super easy to overfit, leak or otherwise do untoward shit when training a model, and c. often the resulting models (churn damping, targeted marketing) didn't perform significantly better than random sampling... But the business users/POs ate up that shit like fine cuisine italian sandwiches, because "we have comprehensive 360' multichannel whateverthefuck" insight into our customer base and we make meaningful business decisions based on this. And this is in GDPR-crazy europe.

majormajor 3y ago
Yeah... I've seen similar in the US. I see LLMs rapidly increasing the data quality of those sorts of datasets by enabling full-text crawling of all sorts of other publicaly-available or purchasable streams of text. You said on HN once that you had 5 consoles? Well, we matched your hn username to your username on this other site, and there was a breach that matched usernames to emails over there, and there was a different breach that let us match emails to full names, and bam, now we have an accurate number.
- mrguyorama 3y ago
  >now we have an accurate number. No, now we THINK we have an accurate number because everyone in the entire chain is bullshitting about how good machine learning and their data is. That's so much worse.
- nradov 3y ago
  People lie online. I have 5 Ferraris. I am 7 ft tall. I once scored 4 touchdowns in a single game. Now the power company AI can use those "facts" to target their marketing message to me, one of their wealthiest and most attractive customers.
deleted 3y ago
[deleted]

majormajor 3y ago

Yeah... I've seen similar in the US. I see LLMs rapidly increasing the data quality of those sorts of datasets by enabling full-text crawling of all sorts of other publicaly-available or purchasable streams of text. You said on HN once that you had 5 consoles? Well, we matched your hn username to your username on this other site, and there was a breach that matched usernames to emails over there, and there was a different breach that let us match emails to full names, and bam, now we have an accurate number.

mrguyorama 3y ago
>now we have an accurate number. No, now we THINK we have an accurate number because everyone in the entire chain is bullshitting about how good machine learning and their data is. That's so much worse.
nradov 3y ago
People lie online. I have 5 Ferraris. I am 7 ft tall. I once scored 4 touchdowns in a single game. Now the power company AI can use those "facts" to target their marketing message to me, one of their wealthiest and most attractive customers.

deleted 3y ago

[deleted]