4 ms·
> Anything loosely connected to AI + LLMs received the bulk of the attention (rightly so, as it is a new chapter in computing) Beyond LLMs and vector search, t
by refset 3y ago
> Anything loosely connected to AI + LLMs received the bulk of the attention (rightly so, as it is a new chapter in computing)
Beyond LLMs and vector search, the scope for applying AI / machine learning _within_ databases is enormous: join planning, learned indexes, compression, workload prediction, configuration tuning etc.
Given the current pace of advances perhaps a dedicated "AI in Databases in 2024" review will be on the cards this time next year.
- chatmasta 3y agoAgreed this is an underexplored topic, which is funny because a few years ago prior to LLMs, if you asked someone how they'd apply ML to a database, you'd get an answer more like this than about any vector indexing or LLM shenanigans.
- matt_d 3y agoLikely worth mentioning that Andy (the author) has been organizing an ML⇄DB Seminar Series (Machine Learning for Databases + Databases for Machine Learning) for the past few months (Fall 2023); materials & lectures: https://db.cs.cmu.edu/seminar2023/ https://db.cs.cmu.edu/seminar2023/, https://www.youtube.com/playlist?list=PLSE8ODhjZXjYVdJKka5g3xTKfPBITrxOu https://www.youtube.com/playlist?list=PLSE8ODhjZXjYVdJKka5g3...
- 392 3y agoI actually just read a paper where some new file format achieved far faster and smaller writes than the Parquet used in all of these modern big datalakehouses, and it was done by guessing like 20 different simple compressions on a sample to figure out what will work best for each chunk of data. Sounds like manual feature detection to me!