3 ms·
you'e 100% right to call this out, and I appreciate the deep dive. To be completely transparent: I’ve over-indexed on the vision and the architecture in this r
by rish2497 8mo ago
you'e 100% right to call this out, and I appreciate the deep dive.
To be completely transparent: I’ve over-indexed on the vision and the architecture in this repo rather than the functional implementation. The current state of the code is effectively a "spec-in-code" and a skeleton of the architecture I am building toward, rather than the production-ready engine my post implied.
The "LLM-generated-slop" comment hits home because I have been using AI tools heavily to scaffold the cross-platform boilerplate (the enums, the FFI bridges, and the project structures). In my excitement to show the "unified pipeline" vision, I pushed a version that is essentially a hollow shell of stubs.
Specifics on your points:
Empty projects: Correct. These are placeholders in the current monorepo structure.
Hardware Enums: You caught the stub. I am currently working on the actual Metal/Vulkan integration layers in a private branch, but I mistakenly pushed the "public skeleton" as if it were the finished core.
200ms TTFT: This is our internal target based on local benchmarks with raw llama.cpp implementations, but as you noted, it is currently "undefined" in the public Flutter wrapper because the bridge isn't actually moving tokens yet.
I genuinely appreciate the reality check. Building a "faster version of llama.cpp" is not my goal, my goal is the orchestration layer, but I clearly tried to "Claude Code" my way through the infrastructure too fast.
I’m going to take this feedback, go back to the shed, and focus on the actual C++ implementation before I post another update. Also, big respect to Telosnex/fllama your are the benchmark for a reason, and I clearly have a lot of work to do to reach that level of "real."
Thanks for keeping the community honest.
- advisedwang 8mo agoI can't tell if this is satire of a LLM response or an actual LLM response