4 ms·
There's no limit to the algorithms. People dont understand yet. They can learn the whole universe with a big enough compute cluster. We built a generalizable le
by mikert89 6mo ago
There's no limit to the algorithms. People dont understand yet. They can learn the whole universe with a big enough compute cluster. We built a generalizable learning machine
- totaa 6mo agothe question is will we experience resource constraints before we get there? what if the step up to post-scarcity is gated by a compute level just out of our reach?
- teaearlgraycold 6mo agoNot sure if this is satire. Edit: What we have built is a natural language interface to existing, textually recorded, information. Transformers cannot learn the whole universe because the universe has not yet been recorded into text.
- supliminal 6mo agoIt’s more than likely not.
- erelong 6mo agoPoe's (c)law?
- bryogenic 6mo agoPoe’s (C)law: The more absurd AI-generated content becomes, the more likely people are to believe it is real.
- 0x3f 6mo agoBased on a glance at their other comments: not satire.
- alfalfasprout 6mo ago100% agreed. Sadly, lots of people out there with the "trust me bro, just need more compute". Hopefully we don't consume all the planet's resources trying.
- xvector 6mo agoI reevaluated my priors long ago when I saw that scaling laws show no sign of stopping, no sign of plateau. Strangely some people on HN seem to desperately cling to the notion that it's all going to come to a halt. This is unscientific. What evidence do you have - any evidence - that the scaling laws are due to come to an end?
- deleted 6mo ago[deleted]
- rishabhaiover 6mo agoI suspect it's not that people do not see the progress, they fail to fully trust laws not truly backed by physics like the transistor laws. We empirically see that scaling works and continue to work.
- esafak 6mo agohttps://en.wikipedia.org/wiki/Neural_scaling_law https://en.wikipedia.org/wiki/Neural_scaling_law
- 0x3f 6mo agoAll the curves have been levelling off as expected. Not really sure what you're talking about.
- solenoid0937 6mo agoThey have not, every successful pre-train as of late has had performance increases greater than what the scaling laws predict.
- lukeschlather 6mo agoTransformers operate on images and a variety of sensor data. They can also operate completely on non-textual inputs and outputs. I don't know what the ceiling on their capabilities is, but the complaint that they only operate on text seems just obviously wrong. There are numerous examples but one is meteorological forecasting which takes in a variety of time series sensor inputs and outputs e.g. time-series temperature maps. https://www.nature.com/articles/s41598-025-07897-4 https://www.nature.com/articles/s41598-025-07897-4
- firecall 6mo agoAFAIK the data does not need to be text.
- teaearlgraycold 6mo agoWell diffusers are trained unsupervised on raw pictures. I don't know how they train multi-modal LLMs on images, but yes obviously they are consuming other media than just text. I don't think, but would be happy to be corrected, that models glean much of their "knowledge" from non-textual training data.
- mikert89 6mo agoyou couldnt be more wrong
- teaearlgraycold 6mo agoPlease tell me more. When I ask an LLM a question, and get a text response, can that response incorporate non-textual information from visual training data?
- IsTom 6mo agoThere are limits to algorithms. AI won't solve the halting problem nor will it solve EXPTIME problems in polynomial time.