4 ms·
Show HN: I built haystack – Google for workplace knowledge
tl;dr: haystack is a workplace search engine for devs that runs in the browser.
Iv'e started working on haystack recently because I feel modern workplaces are in dire need of a good workplace search product that is free to use just like google.
Information is scattered between too many communication channels and platforms, we communicate with our peers through slack and email, share docs and specs on confluence, work with tickets on jira, commit code and have discussions on github, not to mention all the .docx, .ppt and .pdf that fly around the organization.
Existing search featuers plain suck, if you tried using confluence search you know what I mean, keyword search is terrible. Even when you find relevant looking results, they require you to commit to entering the page, and scroll through to get to the relevant paragraph.
What does haystack do?
- Enable you to search all your workplace applications from one place. (slack, confluence, notion, jira, github, outlook, gmail, etc...).
- Natural language queries ("How to do X”, "Do we support Y", "How do I connect to Z").
- No download, all the magic happens in the browser.
- Local browser storage option.
- Code references embedded in search results.
Example, "How to connect to integ2 machine" on haystack could give you:
"I've finished setting up the second integration server ... ssh connection details"
ssh -i private.pem ubuntu@ec2-integration2.eu-west-1.compute.amazonwes.com
aggregated from a slack communication you had a while ago.
Next Steps
Fine-tuning haystack for lower-end laptops with no dedicated GPU, public release date - Feb/March 23.
If you would like to get early access + you have dedicated graphics, there's a button in our landing page, and my email address.
I'll be here in the comment section!
- sandkoan 4y agoLooks pretty cool! I've been working on something similar, but was a bit discouraged by market saturation in the universal search space.... How does this compare to something like, say, Glean?
- _vxw6 4y agoWhat was the name of your product hmu hey@haystack.it, would love to learn from your experience. It's self-hostable, doesn't store your data, more focused on developers and bottom up motions, I want you to be up and running within 10 seconds.
- _vxw6 4y agoRegarding under the hood details, getting haystack up and running in the browser was quite a challenge, but here's what I ended up using: - Storage: IndexDB browser API for local browser storage, it stores read permission tokens to apps, the AI models and the document index. - Indexing: fine-tuned TinyBERT-based bi-encoder for indexing documents, messages and emails. - Searching: cosine similarity between query embedding and the built index, and then rerank using tuned TinyBERT cross-encoder. - Building useful search resuts: search result building involved fine-tuning a t5-small model for summarization and text regression. - Performance nodejs->browser js adaptations, wasm rewrites in rust for performance.
- mritchie712 4y agoNice! Do you only stored documents I visit in the browser (e.g. a specific slack thread / JIRA issue) or are you querying all the API's for each supported app?
- _vxw6 4y agoSo I'm specifically querying all the APIs of the applications you connected, and indexing all documents and threads in local browser storage.
- gardenfelder 4y agoComing soon: https://github.com/haystackoss/haystack https://github.com/haystackoss/haystack
- _vxw6 4y agoDude I was saving that for another post
- gardenfelder 4y agoLooking forward!!!
- _vxw6 4y agolol ok, come with your buddies ;)
- 6451937099 4y ago[dead]
- luckman212 4y agoUpvoting for the concept alone! This is a problem that 100% of the companies I work with suffer with. I wish every manager who showed up didn't force their own stack of tools downstream... When poor Paul wants to find the company's WFH policy, he needs to check Slack, then Google Drive, then Dropbox, then some file server somewhere, and finally ask Jennifer from HR who gives him a link to some TiddlyWiki page to find it. Good luck with the project and happy new year!
- _vxw6 4y agoThis is heart warming to hear! Happy new year!