adsharma

6 ms·

61.
by adsharma 6mo ago
Experimenting with the pglite way, this time using vlang here: https://github.com/adsharma/vpg v run cmd/vpg_test.v
62.
by adsharma 6mo ago
The first one. It's forked from pgserver. Yes, what you get is a multi-server postgres under the covers. But for many users, the convenience of "uv pip install...", auto clean up via context manager is the higher order bit th
63.
by adsharma 6mo ago
Pgserver is not maintained. Had to fork it as pgembed, compile recent versions and bundle BM25 and vector extensions.
64.
by adsharma 6mo ago
Pgembed is pglite for native code.
65.
by adsharma 6mo ago
This is why transpilers exist. py2many can compile static python to mojo, apart from rust and golang. Is it comprehensive? No. But it's deterministic. In the age of LLMs, with sufficient GPU you can either: * Get the LLM to enhance
66.
by adsharma 6mo ago
There is a question of what benefit would it bring even if its open sourced? Static python can transpile to mojo. I haven't seen an argument on what concepts can only be expressed in mojo and not static python? Borrow checker? For sure
67.
by adsharma 6mo ago
Amazing people still keep discovering it. And google search fails to surface working implementations. "Python to rust transpiler" -> pyrs (py2many is a successor) "Python to go transpiler" -> pytago Grumpy was writ
68.
by adsharma 6mo ago
Cython uses C-API. This one doesn't.
69.
by adsharma 6mo ago
Static python as described in this skill. https://github.com/py2many/static-python-skill
70.
by adsharma 6mo ago
Do you have LongMemEval numbers for pgvector vs pgvector+ hybrid search?
71.
by adsharma 7mo ago
Thank you for the shout out! I looked into your benchmark setup a bit. Two things going on: - Ladybug by default allocates 80% of the physical memory to the buffer pool. You can limit it. This wasn't the main reason. - Much of the RSS
72.
by adsharma 7mo ago
Makes it a good match for columnar databases which already operate on the read-only, read-mostly part of the spectrum. Perhaps people can invent LSM like structures on top of them. But at least establish that CSR on disk is a basic requirem
73.
by adsharma 7mo ago
I didn't dismiss the language. I called it a north star. Rust is still the best option if you desire memory safety. But rewriting a complex working piece of software in Rust is not trivial. Having an incremental path (where only parts
74.
by adsharma 7mo ago
Also an important test is the check on whether it's WCOJ on top of relational storage or is the compressed sparse row (CSR) actually persisted to disk. The PGQ implementations don't. There are second order optimizations that LLMs
75.
by adsharma 7mo ago
I maintain LadybugDB which implements WCOJ (inherited from the KuzuDB days). So I don't disagree with the idea. Just that it's a graph database with relational internals and some internal warts that makes it hard to compose querie
76.
by adsharma 7mo ago
This is not just a random idea. AlexNet -> Tansformers -> ChatGPT -> Claude Code -> Small LMs serving KBs Large LLMs could have a role in efficiently producing such KBs.
77.
by adsharma 7mo ago
So this thing is based on Kiwix, which is based on the ZIM file format. In the meanwhile, wikipedia ships wikidata, which uses RDF dumps (and probably 8x less compressed than it should be). https://www.wikidata.org/wiki&#x2F
78.
by adsharma 7mo ago
This is the same topic I had an intense argument with my coworkers at the company formerly called FB a decade ago. There is a belief that most joins are 1-2 deep. And that many hop queries with reasoning are rare and non-existent. I wonder
79.
by adsharma 7mo ago
I maintain a fork of pgserver (pglite with native code). It's called pgembed. Comes with many vector and BM25 extensions. Just in case folks here were wondering if I'm some type of a graphdb bigot.
80.
by adsharma 7mo ago
It comes from people who develop LLMs. Anthropic and Google. References below. My other favorite quote: transformers are GNNs which won the hardware lottery. Longer form at blog.ladybugmem.ai You want to believe that everything probabilisti
81.
by adsharma 7mo ago
For starters, LLMs themselves are a graph database with probabilistic edge traversal. Some apps want it to be deterministic. I'm surprised this question comes up so often. It's mainly from the vector embedding camp, who rightfully
82.
by adsharma 7mo ago
> many millions of dollars to anyone who can demonstrate a graph database that can handle a sparse trillion-edge graph. I wonder why no one has claimed it. It's possible to compress large graphs to 1 byte per edge via Graph reorderi
83.
by adsharma 7mo ago
That importing is expensive and prevents you from handling billion scale graphs. It's possible to run cypher against duckdb (soon postgres as well via duckdb's postgres extension) without having to import anything. That's a g
84.
by adsharma 7mo ago
What is open source and what is a graph database are both hotly debated topics. Author of ArcadeDB critiques many nominally open source licenses here: https://www.linkedin.com/posts/garulli_why-arcadedb-will-nev... Wha
85.
by adsharma 7mo ago
What people perceive as "Facebook production graph" is not just TAO. There is an ecosystem around it and I wrote one piece of it. Full history here: https://www.linkedin.com/pulse/brief-history-graphs-facebook
86.
by adsharma 7mo ago
Source: https://www.theregister.com/2023/03/08/great_graph_debate_we... > There are some additional optimizations that are specific to graphs that a relational DBMS needs to incorporate: [...] This is esse
87.
by adsharma 7mo ago
Are you talking about the query plan for scanning the rel table? Kuzu used a hash index and a join. Trying to make it optional. Try explain match (a)-[b]->(c) return a.rowid, b.rowid, c.rowid;
88.
by adsharma 7mo ago
Are you talking about Andy Pavlo bet here? https://news.ycombinator.com/item?id=29737326 Kuzu folks took some of these discussions and implemented them. SIP, ASP joins, factorized joins and WCOJ. Internally it's struct
89.
by adsharma 7mo ago
LadybugDB is backed by this tech (I didn't write it) https://vldb.org/cidrdb/2023/kuzu-graph-database-management-... You can judge for yourself what work has been done in the last 5 months. Many short videos
90.
by adsharma 7mo ago
There are 25 graph databases all going me too in the AI/LLM driven cycle. Writing it in Rust gets visibility because of the popularity of the language on HN. Here's why we are not doing it for LadybugDB. Would love to explore a mo

More ›