3 ms·

While there is certainly a lot of room for progress in architecture to improve training time, model size and inference speed, high quality data and curated data

While there is certainly a lot of room for progress in architecture to improve training time, model size and inference speed, high quality data and curated data sets are the ultimate bottleneck in terms of what AI can actually do. Low quality data can be useful to a point, but once you have enough data all it's doing is teaching the model to produce low quality output. The next wave of major progress in AI will come when we create image/text data sets that have been heavily annotated with aesthetic judgment metadata, and train models to make aesthetic judgments, then condition generative models on aesthetic judgment scores during training.

eternalban 4y ago
"The key roadblock to achieving human-level language understanding in AI systems is the lack of a comprehensive model of human language and cognition. While large amounts of data and powerful algorithms can help improve the performance of language models, ultimately a better understanding of how human language works and how it is processed by the brain is necessary to build truly intelligent language systems. In addition, the ability to integrate multiple modalities such as vision, speech, and gesture, and to handle the complexity and variability of real-world language use are also important challenges to be addressed." -- you know who