5 ms·
Databases in 2023: A Year in Review
- hobs 3y agoInteresting article, what's with the weird sucking up to Larry Ellison at the end though? It was so weird I thought it was sarcasm but it seemed like it might be genuine?
- refset 3y agoDefinitely sarcasm, it's a running gag - see the 2021 and 2022 posts also :)
- apavlo 3y ago> it seemed like it might be genuine It is genuine. Larry and I go way back. Source: https://youtu.be/_UGUDVeSQSI?t=872 https://youtu.be/_UGUDVeSQSI?t=872
- jszymborski 3y agoSorry to hear that.
- hobs 3y agoOk, you were way too subtle for my first reading or I should have gotten more sleep!
- refset 3y ago> Anything loosely connected to AI + LLMs received the bulk of the attention (rightly so, as it is a new chapter in computing) Beyond LLMs and vector search, the scope for applying AI / machine learning _within_ databases is enormous: join planning, learned indexes, compression, workload prediction, configuration tuning etc. Given the current pace of advances perhaps a dedicated "AI in Databases in 2024" review will be on the cards this time next year.
- chatmasta 3y agoAgreed this is an underexplored topic, which is funny because a few years ago prior to LLMs, if you asked someone how they'd apply ML to a database, you'd get an answer more like this than about any vector indexing or LLM shenanigans.
- matt_d 3y agoLikely worth mentioning that Andy (the author) has been organizing an ML⇄DB Seminar Series (Machine Learning for Databases + Databases for Machine Learning) for the past few months (Fall 2023); materials & lectures: https://db.cs.cmu.edu/seminar2023/ https://db.cs.cmu.edu/seminar2023/, https://www.youtube.com/playlist?list=PLSE8ODhjZXjYVdJKka5g3xTKfPBITrxOu https://www.youtube.com/playlist?list=PLSE8ODhjZXjYVdJKka5g3...
- 392 3y agoI actually just read a paper where some new file format achieved far faster and smaller writes than the Parquet used in all of these modern big datalakehouses, and it was done by guessing like 20 different simple compressions on a sample to figure out what will work best for each chunk of data. Sounds like manual feature detection to me!
- redskyluan 3y ago[dead]
- emmanueloga_ 3y agoDuckPGQ is a super interesting project, since it would mean we would now have a good option for OLAP + graph queries in the same package. But afaict at the moment is a research project... I wonder if DuckDB has it in its roadmap to actually ship this feature.
- kwillets 3y agoThanks for the writeup; in my opinion WCOJ is the most interesting development in core technology. I'll have to disagree about database customers becoming more savvy; cloud has brought in a whole new crackpipe for managers to smoke.
- emmanueloga_ 3y agoA bit random but I wonder if anybody knows what happened to Tigris? It looked like a pretty promising product [1], then one day all activity just halted without any sort of announcement. -- 1: https://hn.algolia.com/?q=tigrisdata https://hn.algolia.com/?q=tigrisdata , a FoundationDB-based document DB, alternative to Mongo and DynamoDB.
- ww520 3y agoIs there any progress or news on the main memory based databases?
- octacat 3y ago> Basho went bankrupt in 2017, and now Riak is maintained by one person who works for the UK's NHS Let Riak go...
- lucysienna 3y ago[dead]