3 ms·

(Replaced my original comment here which was a little unkind.) Question for OP, who created Memvid (the .mv2 file format that's used to distribute this data).

by simonw 11mo ago

(Replaced my original comment here which was a little unkind.) Question for OP, who created Memvid (the .mv2 file format that's used to distribute this data). Are you still taking text, chunking it and then storing those chunks as QR codes in a video file? That seems like an inherently inefficient storage mechanism to me compared with something like SQLite or Parquet - do you have concrete numbers or a demo that shows that your file format really is more effective for storing data for "AI agents" than those existing solutions?

tossit444 11mo ago
Look at memvid's closed issues. The entire thing is a farce. https://github.com/memvid/memvid/issues?q=is%3Aissue%20state%3Aclosed https://github.com/memvid/memvid/issues?q=is%3Aissue%20state...
minimaxir 11mo ago
As a side note: the dataset is referenced in the paper as being from Hugging Face (https://huggingface.co/datasets/julien040/hacker-news-posts https://huggingface.co/datasets/julien040/hacker-news-posts), which does host it as a 426 MB Parquet, while the .mv2 being distributed is 847 MB, for some reason.
simonw 11mo ago
Found a relevant comment here: https://github.com/memvid/memvid/issues/86#issuecomment-3560234272 https://github.com/memvid/memvid/issues/86#issuecomment-3560... > We’ve rebuilt everything from the ground up for Memvid v2, new format, new core, new benchmarks, no QR hacks. It’s a real storage engine now, crash-safe, deterministic, and fully verified with a proper TOC, WAL, Merkle tree, and time index. So I guess the QR code hack isn't a thing any more.