3 ms·
AI and 'Big Data' (as trends) aren't really overlapping in my view. Of course training these LLM models requires a huge amount of data but that's very different
by physicsguy 2y ago
AI and 'Big Data' (as trends) aren't really overlapping in my view. Of course training these LLM models requires a huge amount of data but that's very different from the prospect of spinning up a Spark cluster and writing really badly performing Python code to process something that could have easily been done in a reasonable time anyway on a decent workstation with 128gb of RAM and a large hard drive/SSD, which was a large part of what the hype train was a few years ago.