3 ms·
I miss the pre-LLM days when you could make a decent argument that having any unnecessary data was just a liability. Now all anybody thinks is “more data for th
by wlesieutre 6mo ago
I miss the pre-LLM days when you could make a decent argument that having any unnecessary data was just a liability. Now all anybody thinks is “more data for the AI!”
- CincinnatiMan 6mo agoWere you not around for the Big Data heyday a decade ago?
- varispeed 6mo agoUntil thumb drives became large enough to fit most datasets it stopped becoming Big Data. Just normal data.
- ffsm8 6mo agoWe have thumb drives that can store petabytes of data? Or did you mean the "big data" crowd which thought 500GB was noteworthy? I don't think anyone took those serious, neither in 2010s nor now. That was always "small" data
- varispeed 6mo agoMost companies using term "big data" had datasets in TB region. One company I had a gig at had full Hadoop cluster setup and their whole dataset was 40GB. Their marketing had all the big data adjacent keywords over the brochures for clients.
- gf000 6mo agoThat's a decent quality 3 hours movie :D
- butlike 6mo ago> We have thumb drives that can store petabytes of data We do?
- ffsm8 6mo agoPlease provide a link.
- BizarroLand 6mo agoYou would need 4 and change of these 245tb Kioxias to hold 1 petabyte, and an entire server grade computer to run them. https://www.tomshardware.com/pc-components/ssds/kioxia-unveils-245tb-ssd-the-worlds-highest-capacity-storage-device-could-store-12-500-4k-movies https://www.tomshardware.com/pc-components/ssds/kioxia-unvei... Or 250 of these ~$400 4tb flash drives and an insane number of dongles to connect them all: https://www.slashgear.com/1847725/largest-usb-thumb-drive-highest-capacity-can-buy/ https://www.slashgear.com/1847725/largest-usb-thumb-drive-hi...
- vunuxodo 6mo agoPlus one more for your parity drive.
- dylan604 6mo agoIt was a question that you've edited out the punctuation. You're asking the exact same thing as the person you've replied
- 0x457 6mo agoMy rule of thumb was "can it fit in RAM on a server?" If it can, then it's not big data. 500GB is in the "fits" category.
- jmalicki 6mo agoTo some degree IMO big data is still a mindset when it might take a day to process your data in a normal SQL query. Some tech doesn't scale to the data size for all use cases, and you need different solutions.
- ToucanLoucan 6mo agoHell you mean a decade ago? I still see businesses running losses left right and center saying that they're gonna monetize user data, any day now. Related "monetizing user data" seems to just mean ads. Ads on everything, forever, until the userbase gets fed up and moves to a new service that definitely won't do that, and the cycle repeats about every 3 years.
- citrin_ru 6mo agoData hoarding predates LLMs. There where other machine learning methods which also needed data for training.
- Forgeties79 6mo ago“Before LLM’s there was_____” I see this whenever an LLM’s impact is assessed. We know. The issue is scale and the ability for smaller and smaller groups (down to individuals) to execute at scale. Fake news always existed. Now one dude in India can flood multiple sock puppet media accounts with right wing content/images (actual example) at a scale previously unimaginable.
- dpoloncsak 6mo agoDo LLMs require that much more data than the tradional ML approaches we've seen over the years?
- sigmoid10 6mo agoYes. This is pretty well established. Neural networks in general are considerably less sample-efficient than traditional ML methods. The reason they became so successful is that they scale better as you increase training data and model size. But only with modern compute power they became useful outside of academic toy model applications.
- Forgeties79 6mo agoThat’s not the issue I’m hitting here primarily but yes. My concern is that I can open up chatGPT and even with a free, “anonymous” account run an assembly line generating tens of thousands of words a day to pump to Twitter that are good enough to prop up multiple fake accounts and cause mayhem. Now make it thousands of people like me doing it. Now add funding and political orgs. Add company leadership that turns a blind eye so long as it drives engagement. This scale and pipeline wasn’t possible 5 years ago, even if we clearly see the throughline. I’m not even getting into fake images either. That used to require some know how. There are basically no hurdles and even if most people learn it’s fake, millions likely won’t. If you’re a little lucky, less scrupulous “news” outlets will amplify it for you as well for free.
- hdndjsbbs 6mo ago10+ years ago companies were hoovering up data for ML - trying to find correlations in high-dimensionality data. Mostly the results were garbage but occasionally you hit on a real, unexpected phenomenon. Nowadays you just throw all the data into a black box and believe whatever it says blindly.