kgeist

22 ms·

211.
by kgeist 10mo ago
From what I understand, the complexity stays there, it's just moved from the DB layer to the app layer (now I have to decide how to shard data, how to reshard, how to synchronize data across shards, how to run queries across shards wit
212.
by kgeist 10mo ago
I tried making something similar a while ago, and the main problem was that long-term memory makes it easy to move the AI into a bad state where it overfixates on something (context poisoning), or decides to refuse talking to me completely.
213.
by kgeist 10mo ago
What about constrained decoding (with JSON schemas)? I noticed my vLLM instance is using 1 CPU 100%.
214.
by kgeist 10mo ago
With guided decoding (structured output according to a schema), a model can sometimes return broken JSON - usually if it stops midway for some reason. In those rare cases, a better approach would be to simply retry, no? Trying to "fix&
215.
by kgeist 10mo ago
If they imply that Polish is best at instruction following in LLMs, then I doubt it. It's highly dependant on the prompts, schemas and the LLMs. I maintain bechmarks for our product across 33 languages. Slightly changing the prompts or
216.
by kgeist 10mo ago
I've used HTMX on a personal project of mine. Other than HTMX itself, it used Go templates + Tailwind for CSS. As a backend dev with almost no professional frontend experience, I was able to replicate a fairly large and feature rich Re
217.
by kgeist 10mo ago
I had this one too: I first denied all incoming requests and was about to allow SSH, but my SSH connection dropped :) Fortunately, I was able to restore the VM with the provider's VM console.
218.
by kgeist 10mo ago
I think I understand what went wrong. I was confused by the instructions and ChatGPT's UI. I asked the GitHub app to review my repository, and the app told me to click the GitHub icon and select the repository from the menu to grant it
219.
by kgeist 10mo ago
Tried the GitHub app, made sure everything was properly connected, and asked a question about one of my repositories. It repeatedly claimed (5 times) that it wasn't connected and couldn't do anything, telling me to check the check
220.
by kgeist 10mo ago
>the GPT-OSS models are also quite good I recently pitted gpt-oss 120b against Qwen3-Next 80b on a lot of internal benchmarks (for production use), and for me, gpt-oss was slightly slower (vLLM, both fit in VRAM), much worse at multiling
221.
by kgeist 10mo ago
How does Milvus compare to OpenSearch/ElasticSearch for hybrid search?
222.
by kgeist 10mo ago
Some additional cases we encounter quite often where UUIDs help: - A client used to run our app on-premises and now wants to migrate to the cloud. - Support engineers want to clone a client’s account into the dev environment to debug issues
223.
by kgeist 10mo ago
We had a similar problem in our monolith. Team #1 works on a feature, their code breaks tests. Team #2 works on another feature, their code is OK, but they can't move forward because of the failing tests from team #1. Plus often it tak
224.
by kgeist 10mo ago
I think what the article is doing wrong is treating all microservices the same. Microservices can be split into at least 3 different groups: - infrastructure (auth, messaging, storage etc.) - domain-specific business logic (user, orde
225.
by kgeist 10mo ago
I think if a project name is easily googleable, it's a good name. If I see "generic-tool-name failed" in the logs and can't immediately find what that thing even is, because I get lots of irrelevant search results, it&#x
226.
by kgeist 10mo ago
>Instead of asking the runtime for memory object-by-object, an Arena lets you allocate a large pool of memory upfront. You fill that pool with objects using a simple bump pointer (which is CPU cache-friendly), and when you are done, you
227.
by kgeist 10mo ago
If we take the most popular programming languages and look at what their reference (or most popular) implementations are written in, then we get: C++: JavaScript (V8), Java, C# C: Python, PHP, Lua, Ruby Self-hosted: Go, Rust Fa
228.
by kgeist 10mo ago
Yeah, that can work. Just yesterday I benchmarked load balancing of LLM workloads across 2 GPUs using a simple least_conn from nginx. The total token/sec scaled as expected (2 GPUs => 2x token/sec), and GPU utilization reached
229.
by kgeist 10mo ago
I mean, currently LLMs are stateless and you can get rid of all the poisoned data by just starting a new conversation (context). And OP introduces "long-term memory" where junk will accumulate with time
230.
by kgeist 10mo ago
>The model uses this internal error signal (the gradient) as a mathematical equivalent of saying, "This is unexpected and important!" This allows the Titans architecture to selectively update its long-term memory only with the
231.
by kgeist 11mo ago
>People still talk about fine tuning dedicated models being effective >it's still always better to use a larger generalist model than a smaller fine tuned one Smaller fine-tuned models are still a good fit if they need to run on-
232.
by kgeist 11mo ago
It depends on how you test it. I recently found that the way devs test it differs radically from how users actually use it. When we first built our RAG, it showed promising results (around 90% recall on large knowledge bases). However, when
233.
by kgeist 11mo ago
So that poop particles didn't spread beyond the bathroom.
234.
by kgeist 11mo ago
>A 2024 study showed evidence that even closing the lid may still lead to small viral particles escaping through gaps under the lid, resulting in viral cross contamination of the air and surfaces in a washroom
235.
by kgeist 11mo ago
https://en.wikipedia.org/wiki/Toilet_plume A door provides at least some kind of physical barrier.
236.
by kgeist 11mo ago
I already had experience with RAG before so I had a head start. You're right that it's not rocket science, but it's not just "press F to implement the feature" either P.S. No vibe coding was used. I only used LLM-as
237.
by kgeist 11mo ago
You're right, bad wording
238.
by kgeist 11mo ago
Well it took me 2 full-time weeks to properly implement a RAG-based system so that it found actually relevant data and did not hallucinate. Had to: - write an evaluation pipeline to automate quality testing - add a query rewriting step to e
239.
by kgeist 11mo ago
Reminds me of Shed Skin, but no mention of it. I wonder how they compare. https://en.wikipedia.org/wiki/Shed_Skin
240.
by kgeist 1y ago
I'm currently building RAG for our product (using Lucene). What I've found is that embeddings alone don't help much. With hybrid search (BM25+HNSW) they gave me only like +10% boost compared to BM25 alone (on average). In my

More ›