3 ms·
Nice, I recently did a similar but much simpler thing and open-sourced it under MIT, maybe some bits and pieced will be useful https://github.com/eatmydata-org/
by dennis16384 4mo ago
Nice, I recently did a similar but much simpler thing and open-sourced it under MIT, maybe some bits and pieced will be useful https://github.com/eatmydata-org/eatmydata https://github.com/eatmydata-org/eatmydata
For example, MIT-licensed sqlite vector search extension.
Overall, I have a orchestrator - sql coder - js coder - dashboards, all without backend, running locally in the browser. It's mostly tested on small analysis and question answering with Gemini Flash Lite, and the overall target was speed from question to answer, including data sharing and waiting.
- arcb 4mo agoThere are a lot of cool and useful things in there. What are you most excited about?
- dennis16384 4mo agoFast response. I can upload Excel/csv and iterate under 10 seconds from question to result. Doing same thing in Claude with 10x less data takes 5 minutes.
- arcb 4mo agoI hear you on fast responses. One of the frustrations I've had using BI / data tools in the past was not being able to get local performance... which led to me exporting data to spreadsheets or local code. We're taking this to heart for BitBoard as well.
- dennis16384 4mo agoTotally. One thing that all major AI vendors are not doing currently is merging server AI with edge devices. For example, there is no way neither in Claude nor in ChatGPT to run your own WASM or JS or whatever AI produces directly in user's browser context as a tool/skill - there is no call site for that. The only option is remote server-side. My whole idea was that AI can perfectly write SQL and dashboard code knowing only the shape of your data and not it's contents. With direct upload to vendor now we're forced to share the contents.
- arcb 4mo agoI suspect stronger edge performance will come as a side-effect of local inference. Your point on edge tool calls is interesting and I'll think about that. Features like offline mode could be a great motivating reason. Re knowing the shape vs not the internals - I'm mixed here. It feels like there's always a sampling period where you have to look at contents in order to understand what you want. But edge AI (like antirez's work running DeepSeek on Mac) will let you have both. I'm excited for that future!
- dennis16384 4mo agoWhy would an LLM want to look into the contents, what for? We have low-cardinality data and yes this is safe to share and required to build an actual query. Then we have high-cardinality and possibly PII - there’s absolutely no reason to share that data, there’s nothing for LLM to analyse there. Also semantic index (vector search) will find relevant records much faster and more accurately that any chain-of-thoughts just with an LLM-authored search fn call. Further there are continuous numerical values and there’s not much LLM needs to see in there either. We can say, for example, if you look at data distributions when building your analysis, it can drive your analysis logic, but another point of view here is taht it creates unnecessary bias instead.
- arcb 4mo agoOn re-read I think I might have overreached in my reply. I think having local LLMs being able run tool loops to _transform_ data, rather than just summary or analysis, will become 1/ great for non-technical users, 2/ fast.