5 ms·

I'm not sure, it's an observation considering how AI improvement is related to Moore's law. [1](https://techcrunch.com/2025/01/07/nvidia-ceo-says-his-ai-chips-

I'm not sure, it's an observation considering how AI improvement is related to Moore's law. [1](https://techcrunch.com/2025/01/07/nvidia-ceo-says-his-ai-chips-are-improving-faster-than-moores-law/ https://techcrunch.com/2025/01/07/nvidia-ceo-says-his-ai-chi...)

high_na_euv 2y ago
But some say that Moore Law is dead :) Anyway, the mumber of tiktok users coorelates with advancements in AI too! Before tiktok the progress was slower, then when tiktok appeared it progressed as hell!
- nerder92 2y ago
  Yes I see your point, correlation is not causation. Again, this is my best guess and observation based on my personal view of the world and my understanding of data on hands at t0 (today). This doesn't spare if from being incorrect or extremely wrong, as always when dealing with predictions of a future outcome.
somenameforme 2y ago
That's an assumption. Most/all neural network based tech faces a similar problem of exponentially diminishing returns. You get from 0 to 80 in no time. A bit of effort and you eventually ramp it up to 85, and it really seems the goal is imminent. Yet suddenly each percent, and then each fraction of a percent starts requiring exponentially more work. And then you can even get really fun things like you double your training time and suddenly the resultant software starts scoring worse on your metrics, usually due to overfitting. And it seems, more or less, clear that the rate of change in the state of the art has already sharply decreased. So it's likely LLMs have already entered into this window.
kykeonaut 2y ago
However, an increase in computing quality doesn't necessarily mean an increase in output quality, as you need compute power + data to train these models. Just increasing compute power will increase the performance/training speed of these models, but you also need to increase the quality of the data that you are training these models on. Maybe... the reason why these models show a high school level of understanding is because most of the data on the internet that these models have been trained on is of high school graduate quality.
deleted 2y ago
[deleted]

high_na_euv 2y ago

But some say that Moore Law is dead :) Anyway, the mumber of tiktok users coorelates with advancements in AI too! Before tiktok the progress was slower, then when tiktok appeared it progressed as hell!

nerder92 2y ago
Yes I see your point, correlation is not causation. Again, this is my best guess and observation based on my personal view of the world and my understanding of data on hands at t0 (today). This doesn't spare if from being incorrect or extremely wrong, as always when dealing with predictions of a future outcome.

somenameforme 2y ago

That's an assumption. Most/all neural network based tech faces a similar problem of exponentially diminishing returns. You get from 0 to 80 in no time. A bit of effort and you eventually ramp it up to 85, and it really seems the goal is imminent. Yet suddenly each percent, and then each fraction of a percent starts requiring exponentially more work. And then you can even get really fun things like you double your training time and suddenly the resultant software starts scoring worse on your metrics, usually due to overfitting. And it seems, more or less, clear that the rate of change in the state of the art has already sharply decreased. So it's likely LLMs have already entered into this window.

kykeonaut 2y ago

However, an increase in computing quality doesn't necessarily mean an increase in output quality, as you need compute power + data to train these models. Just increasing compute power will increase the performance/training speed of these models, but you also need to increase the quality of the data that you are training these models on. Maybe... the reason why these models show a high school level of understanding is because most of the data on the internet that these models have been trained on is of high school graduate quality.

deleted 2y ago

[deleted]