7 ms·

Show HN: TinkerBird – A Chrome-native vector database

107 points by wizenheimer 2y ago

meiraleal 2y ago
Great project! Thanks for sharing, I'm looking for something like this. One thing I'm curious about tho is the evolution of the project. Github says the last update in the code was 7 months ago. Is it because it is ready/done?
oefrha 2y ago
It’s a vector db on top of IndexedDB. I fail to see what’s “Chrome” (or “native”) about it. Is there anything Chrome-specific that prevents it from working in any other modern browser?
- keepamovin 2y ago
  [flagged]
  - oefrha 2y ago
    I’m not responding to the title. The project is introduced as “TinkerBird is a Chrome-native vector database designed for efficient storage and retrieval of high-dimensional vectors (embeddings).” The condescension against a valid question is also not appreciated. Edit: Can’t respond to reply since it’s been flagged dead: > needlessly ornery focus on something irrelevant For a web developer, browser support is one of the most important considerations. I can’t use the most amazing library in the world if that means 10%-50% of my users are locked out (not exaggerating, a mobile-centric site for a U.S. audience could see ~50% traffic from Safari). And to expand on the question, IndexedDB implementations can be quirky (at least historically), so without digging into the code it’s entirely conceivable to me that e.g. Safari support could be somehow troublesome.
    - keepamovin 2y ago
      [flagged]
- wizenheimer 2y ago
  That's right, it should. If it supports indexeddb, it supports tinkerbird. Ref: https://caniuse.com/indexeddb https://caniuse.com/indexeddb PS: Pretty early nonetheless, hacked it together during last Christmas. Gotta make it usable. PPS: I guess the README is dated. Built out a sample app for Tinkerbird. Feel free to take a look: https://github.com/wizenheimer/tinkerboard https://github.com/wizenheimer/tinkerboard
  - oefrha 2y ago
    Cool, tried the demo on Safari. Might want to consider rewording to avoid misconception about browser support.
    - andybak 2y ago
      Or misconceptions about what it even is. I didn't realise it was a javascript library.
- deleted 2y ago
  [deleted]
keepamovin 2y ago
That's a pretty well written bug report: https://github.com/wizenheimer/tinkerbird/issues/5 https://github.com/wizenheimer/tinkerbird/issues/5 I hope you get the help you need for that!
- wizenheimer 2y ago
  Thanks, mate. Skill issue :D
samspenc 2y ago
I haven't played with it yet, but very impressive :) a full vector DB built on top of IndexedDB. That means it should run on all modern browsers, and should be able to handle large volumes of data (at least, proportional to disk space).
- vmfunction 2y ago
  How is the state of IndexedDB on Safari IO now days? Last heard that it is pretty bad with unexpected wip if it is not used for a while.
- wizenheimer 2y ago
  Thanks, mate. I guess the README is dated. Here's a sample app built around tinkerbird. Make it yours :D Playground: https://tinkerboard.vercel.app/ https://tinkerboard.vercel.app/ Repo: https://github.com/wizenheimer/tinkerboard https://github.com/wizenheimer/tinkerboard
nostrebored 2y ago
Avoiding the server call seems to only be time-efficient if calls to embed are time-efficient. The nice thing about server side inference is that you can use embedding models that can handle web-scale data independent of hardware
- kovezd 2y ago
  Time-efficiency is not the only metric to consider. Privacy, and offline mode are also important. Great tool for federated learning.
alexliu518 2y ago
Cool! Keep it up!
- wizenheimer 2y ago
  Thanks, mate. Incase, you feel like playing around. https://tinkerboard.vercel.app/ https://tinkerboard.vercel.app/
deleted 2y ago
[deleted]
hoten 2y ago
Is there something that makes this specific to Chrome or is this a web compatible library?
- andybak 2y ago
  Yes - I don't understand the "Chrome" part at all. I presumed it was a Chrome extension but it doesn't seem to be.
  - hoten 2y ago
    They just meant it as a JavaScript library, apparently. OP- the term "chrome native" is pretty inaccurate and is giving people the wrong idea about what you're showing off.
    - wizenheimer 2y ago
      Appreciate the feedback, mate
      - Tagbert 2y ago
        yes, that makes it sound like it is Chrome-exclusive, which is not something we want.
  - wizenheimer 2y ago
    > I don't understand the "Chrome" part at all It uses, relies and depends on Chrome exposed primitive such IndexedDB. Hence the term chrome native. Honestly, didn't give much thought to it. Just rolled with it :D
    - andybak 2y ago
      I think the reason you're getting so many comments on this aspect of it is because a lot of people on HN have been through several rounds of the browser wars and many care deeply about keeping the web vendor neutral (especially in terms of the near-monopoly that Google has). Whilst there's often good reasons to state that something will only work on Chrome - it's something that should probably be wrapped in a something approximating a mild apology: "Whilst this might work on any browser that supports IndexedDB I've currently only tested it on blah blah" - that kind of thing... https://caniuse.com/indexeddb https://caniuse.com/indexeddb
splitbrain 2y ago
I wish you'd written the introduction in the readme yourself instead of letting a LLM do it for you. "utilizes", "significant challenge facing", "disrupts workflows" and then several repeats of the same information... all tell tale signs.
- wasmitnetzen 2y ago
  But then again, multiple "it's" vs "its" spelling mistakes. Tell tale sign of a human, possibly a native speaker.
  - meandmycode 2y ago
    It actually reads distinctly not ai written, but calling things ai is vogue now
jwithington 2y ago
So one could retain the contents of every website every visited and then run local, in-browser searches against that content? Is that the idea? This li
jwithington 2y ago
So one could retain the contents of every website every visited and then run local, in-browser searches against that content? Is that the idea? This library would serve as the vector DB?
- wizenheimer 2y ago
  Imagine this. You stumbled across this insane search technique, which knows what you want. It doesn't care if you know the exact keywords or not, it just gets what you're after. We call it Vector Search / Semantic Search. Cool stuff right? You don't have to be a keyword wizard to get your stuff. But there's a problem. Unlike it's keyword search counterpart it expensive. Order of magnitude expensive. All because you need to maintain a vector search index and keep it available server side. Enter Tinkerbird. This little library takes the heavy lifting and pushes it client side. Stream your embeddings, front it via CDN or cache it on client. Now you get benefits of vector index without the costs of maintaining one.
solarkraft 2y ago
What the hell does “chrome native” mean? Is it an extension? Who uses it?
- mock-possum 2y ago
  I can’t figure that out either - especially when the readme mentions using typescript, which is very much not chrome-native - you have to compile to JavaScript before chrome will have anything to do when it - Maybe it’s referring to the use of IndexDB?
jamesmcintyre 2y ago
This looks really promising and thanks for sharing! I've actually been looking into this exact type of functionality lately, for anyone else I'm just going to drop some other projects in this area (also if anyone, including OP, want's to chime in on how they compare- i know there's all sorts of trade-offs ie. the similarity algo chosen, one of these relies on wasm, etc): https://github.com/tantaraio/voy https://github.com/tantaraio/voy https://github.com/nitaiaharoni1/vector-storage https://github.com/nitaiaharoni1/vector-storage https://github.com/danielivanovz/indexed-vector-store https://github.com/danielivanovz/indexed-vector-store https://github.com/yusufhilmi/client-vector-search https://github.com/yusufhilmi/client-vector-search Another area I've looked into briefly is also trying to generate vector embeddings fully in-browser. I know there will likely be tradeoffs with dimensionality / overall accuracy/performance but having a semi-decent way to do this in-browser would be awesome. This project only runs in node (but there's an issue that talks about how it might be modified to run in-browser): https://github.com/Anush008/fastembed-js https://github.com/Anush008/fastembed-js Looks like transformers.js supports embeddings in-browser: https://github.com/xenova/transformers.js/releases/tag/2.1.0 https://github.com/xenova/transformers.js/releases/tag/2.1.0 https://github.com/pinecone-io/semantic-search-example/blob/bc975f214a78ba2c479fc782b74cfa2510ae15f1/README.md?plain=1#L108-L117 https://github.com/pinecone-io/semantic-search-example/blob/... Also it appears the client-vector-search supports both embeddings & vector indexing https://github.com/yusufhilmi/client-vector-search https://github.com/yusufhilmi/client-vector-search Hope this helps others looking into this stuff!
KashyapArjun 2y ago
> TinkerBird uses IndexedDB as it's storage layer, which in turn builds upon Blobs and LevelDB storage systems. By using Indexeddb, it benefits from IndexedDB's adoption, stability and familiarity as a native choice for offline first workflows. Super impressive. Looking forward to use it!
masherm 2y ago
how would you use this within chrome?

107 points by wizenheimer 2y ago

Great project! Thanks for sharing, I'm looking for something like this. One thing I'm curious about tho is the evolution of the project. Github says the last update in the code was 7 months ago. Is it because it is ready/done?

oefrha 2y ago

It’s a vector db on top of IndexedDB. I fail to see what’s “Chrome” (or “native”) about it. Is there anything Chrome-specific that prevents it from working in any other modern browser?

keepamovin 2y ago
[flagged]
- oefrha 2y ago
  I’m not responding to the title. The project is introduced as “TinkerBird is a Chrome-native vector database designed for efficient storage and retrieval of high-dimensional vectors (embeddings).” The condescension against a valid question is also not appreciated. Edit: Can’t respond to reply since it’s been flagged dead: > needlessly ornery focus on something irrelevant For a web developer, browser support is one of the most important considerations. I can’t use the most amazing library in the world if that means 10%-50% of my users are locked out (not exaggerating, a mobile-centric site for a U.S. audience could see ~50% traffic from Safari). And to expand on the question, IndexedDB implementations can be quirky (at least historically), so without digging into the code it’s entirely conceivable to me that e.g. Safari support could be somehow troublesome.
  - keepamovin 2y ago
    [flagged]
wizenheimer 2y ago
That's right, it should. If it supports indexeddb, it supports tinkerbird. Ref: https://caniuse.com/indexeddb https://caniuse.com/indexeddb PS: Pretty early nonetheless, hacked it together during last Christmas. Gotta make it usable. PPS: I guess the README is dated. Built out a sample app for Tinkerbird. Feel free to take a look: https://github.com/wizenheimer/tinkerboard https://github.com/wizenheimer/tinkerboard
- oefrha 2y ago
  Cool, tried the demo on Safari. Might want to consider rewording to avoid misconception about browser support.
  - andybak 2y ago
    Or misconceptions about what it even is. I didn't realise it was a javascript library.
deleted 2y ago
[deleted]

keepamovin 2y ago

That's a pretty well written bug report: https://github.com/wizenheimer/tinkerbird/issues/5 https://github.com/wizenheimer/tinkerbird/issues/5 I hope you get the help you need for that!

wizenheimer 2y ago
Thanks, mate. Skill issue :D

samspenc 2y ago

I haven't played with it yet, but very impressive :) a full vector DB built on top of IndexedDB. That means it should run on all modern browsers, and should be able to handle large volumes of data (at least, proportional to disk space).

vmfunction 2y ago
How is the state of IndexedDB on Safari IO now days? Last heard that it is pretty bad with unexpected wip if it is not used for a while.
wizenheimer 2y ago
Thanks, mate. I guess the README is dated. Here's a sample app built around tinkerbird. Make it yours :D Playground: https://tinkerboard.vercel.app/ https://tinkerboard.vercel.app/ Repo: https://github.com/wizenheimer/tinkerboard https://github.com/wizenheimer/tinkerboard

nostrebored 2y ago

Avoiding the server call seems to only be time-efficient if calls to embed are time-efficient. The nice thing about server side inference is that you can use embedding models that can handle web-scale data independent of hardware

kovezd 2y ago
Time-efficiency is not the only metric to consider. Privacy, and offline mode are also important. Great tool for federated learning.

alexliu518 2y ago

Cool! Keep it up!

wizenheimer 2y ago
Thanks, mate. Incase, you feel like playing around. https://tinkerboard.vercel.app/ https://tinkerboard.vercel.app/

deleted 2y ago

[deleted]

hoten 2y ago

Is there something that makes this specific to Chrome or is this a web compatible library?

andybak 2y ago
Yes - I don't understand the "Chrome" part at all. I presumed it was a Chrome extension but it doesn't seem to be.
- hoten 2y ago
  They just meant it as a JavaScript library, apparently. OP- the term "chrome native" is pretty inaccurate and is giving people the wrong idea about what you're showing off.
  - wizenheimer 2y ago
    Appreciate the feedback, mate
    - Tagbert 2y ago
      yes, that makes it sound like it is Chrome-exclusive, which is not something we want.
- wizenheimer 2y ago
  > I don't understand the "Chrome" part at all It uses, relies and depends on Chrome exposed primitive such IndexedDB. Hence the term chrome native. Honestly, didn't give much thought to it. Just rolled with it :D
  - andybak 2y ago
    I think the reason you're getting so many comments on this aspect of it is because a lot of people on HN have been through several rounds of the browser wars and many care deeply about keeping the web vendor neutral (especially in terms of the near-monopoly that Google has). Whilst there's often good reasons to state that something will only work on Chrome - it's something that should probably be wrapped in a something approximating a mild apology: "Whilst this might work on any browser that supports IndexedDB I've currently only tested it on blah blah" - that kind of thing... https://caniuse.com/indexeddb https://caniuse.com/indexeddb

splitbrain 2y ago

I wish you'd written the introduction in the readme yourself instead of letting a LLM do it for you. "utilizes", "significant challenge facing", "disrupts workflows" and then several repeats of the same information... all tell tale signs.

wasmitnetzen 2y ago
But then again, multiple "it's" vs "its" spelling mistakes. Tell tale sign of a human, possibly a native speaker.
- meandmycode 2y ago
  It actually reads distinctly not ai written, but calling things ai is vogue now

jwithington 2y ago

So one could retain the contents of every website every visited and then run local, in-browser searches against that content? Is that the idea? This li

jwithington 2y ago

So one could retain the contents of every website every visited and then run local, in-browser searches against that content? Is that the idea? This library would serve as the vector DB?

wizenheimer 2y ago
Imagine this. You stumbled across this insane search technique, which knows what you want. It doesn't care if you know the exact keywords or not, it just gets what you're after. We call it Vector Search / Semantic Search. Cool stuff right? You don't have to be a keyword wizard to get your stuff. But there's a problem. Unlike it's keyword search counterpart it expensive. Order of magnitude expensive. All because you need to maintain a vector search index and keep it available server side. Enter Tinkerbird. This little library takes the heavy lifting and pushes it client side. Stream your embeddings, front it via CDN or cache it on client. Now you get benefits of vector index without the costs of maintaining one.

solarkraft 2y ago

What the hell does “chrome native” mean? Is it an extension? Who uses it?

mock-possum 2y ago
I can’t figure that out either - especially when the readme mentions using typescript, which is very much not chrome-native - you have to compile to JavaScript before chrome will have anything to do when it - Maybe it’s referring to the use of IndexDB?

jamesmcintyre 2y ago

This looks really promising and thanks for sharing! I've actually been looking into this exact type of functionality lately, for anyone else I'm just going to drop some other projects in this area (also if anyone, including OP, want's to chime in on how they compare- i know there's all sorts of trade-offs ie. the similarity algo chosen, one of these relies on wasm, etc): https://github.com/tantaraio/voy https://github.com/tantaraio/voy https://github.com/nitaiaharoni1/vector-storage https://github.com/nitaiaharoni1/vector-storage https://github.com/danielivanovz/indexed-vector-store https://github.com/danielivanovz/indexed-vector-store https://github.com/yusufhilmi/client-vector-search https://github.com/yusufhilmi/client-vector-search Another area I've looked into briefly is also trying to generate vector embeddings fully in-browser. I know there will likely be tradeoffs with dimensionality / overall accuracy/performance but having a semi-decent way to do this in-browser would be awesome. This project only runs in node (but there's an issue that talks about how it might be modified to run in-browser): https://github.com/Anush008/fastembed-js https://github.com/Anush008/fastembed-js Looks like transformers.js supports embeddings in-browser: https://github.com/xenova/transformers.js/releases/tag/2.1.0 https://github.com/xenova/transformers.js/releases/tag/2.1.0 https://github.com/pinecone-io/semantic-search-example/blob/bc975f214a78ba2c479fc782b74cfa2510ae15f1/README.md?plain=1#L108-L117 https://github.com/pinecone-io/semantic-search-example/blob/... Also it appears the client-vector-search supports both embeddings & vector indexing https://github.com/yusufhilmi/client-vector-search https://github.com/yusufhilmi/client-vector-search Hope this helps others looking into this stuff!

KashyapArjun 2y ago

> TinkerBird uses IndexedDB as it's storage layer, which in turn builds upon Blobs and LevelDB storage systems. By using Indexeddb, it benefits from IndexedDB's adoption, stability and familiarity as a native choice for offline first workflows. Super impressive. Looking forward to use it!

masherm 2y ago

how would you use this within chrome?