svcrunch

4 ms·

1.
by svcrunch 11d ago
If you'd like to experiment with Gemini 3.8 Live on a US telephone number, you can try Wokay [0]. It's an agentic memory demonstration that's built with LiveKit and Gemini. Tel: 408–897–4019 [0] https://wokay.goodm
2.
Pulley is shutting down and will cease all operations and services on Dec 8th
10 points by svcrunch 11d ago|6 comments
3.
by svcrunch 11d ago
I've been using the Vector APIs for years now, and I'm still waiting for them to GA! They are useful for neural information retrieval (RAG, memory), which relies heavily on content vectorization and similarity matching using their
4.
by svcrunch 12d ago
We recently red-teamed GoodMem with GLM 5.3. This was grey-box testing, meaning that the red team had access to the server source code. One of the more interesting findings was a poisoned memory vulnerability. Basically, this is when an att
5.
by svcrunch 12d ago
I'd like to mention the Little Dorrit Benchmark [1] which I have been running for a couple of years now. It has a few nice features: 1. It tests visual reasoning and structured output in a single task. 2. It seems to sort correctly on
6.
by svcrunch 12d ago
This is cool. I also ported it to GoodMem ( https://goodmem.ai ), which is the governed agentic memory layer we work on: https://github.com/PAIR-Systems-Inc/Agentic_RAG_GoodMem
7.
A $37 GLM 5.3 red team: the Alloy-modeled auth layer held, but two bugs outside(goodmem.ai)
1 points by svcrunch 17d ago|0 comments
8.
GPT-6 Astra soundly defeats Fable 5.1 on recognizing handwritten corrections(dorrit.pairsys.ai)
3 points by svcrunch 20d ago|1 comments
9.
by svcrunch 20d ago
The [Pelican Benchmark]( https://github.com/simonw/pelican-bicycle ) is in the LLM's training data and probably not a useful indicator of improving LLM capabilities any more. For the past two years, I've run th
10.
by svcrunch 2mo ago
For over a year, I've run the Little Dorrit benchmark. It's a vision task that converts handwritten editor marks on a printed page into structured JSON output. I hadn't updated the benchmark in several months, but there are s
11.
Handwritten-edit benchmark: Fable 5 is #1, Opus 4.8 regresses 55% on miscounting(dorrit.pairsys.ai)
4 points by svcrunch 2mo ago|1 comments
12.
by svcrunch 6mo ago
The grandparent is definitely wrong on (3). Yes, coding is a killer product, I agree with you. On (2), I agree with you for local models. BUT , there are also the open source Chinese models accessible via open-router. Your argument ("
13.
by svcrunch 7mo ago
I generally don't waste time with patents. I think most patents in deep learning can be overturned by prior art. My current approach to IP is trade secrets. If we publish, we are careful to avoid details that would make the techniques
14.
GPT-5.4 Scores 0.62 F1 on Understanding Handwritten Edits in Dickens(dorrit.pairsys.ai)
2 points by svcrunch 7mo ago|0 comments
15.
by svcrunch 8mo ago
Thanks for your interest. The rerankers are external, GoodMem is a unified API layer that calls out to various providers. There's no model running inside the database or the GoodMem server. We support both commercial APIs and self-host
16.
by svcrunch 8mo ago
Hi there, thanks for writing and sharing your experiences. I'm one of the builders of GoodMem ( https://goodmem.ai/ ), which is infra to simplify end-to-end RAG/agentic memory systems like the one you built. It&#x27
17.
by svcrunch 1y ago
Here's a problem that no frontier model does well on (f1 < 0.2), but which I think is relatively easy for most humans: https://dorrit.pairsys.ai/ > This benchmark evaluates the ability of multimodal language mode
18.
by svcrunch 1y ago
This is really cool.
19.
Can GPT-4o Accurately Read Handwritten Proofreading Marks?(dorrit.pairsys.ai)
1 points by svcrunch 1y ago|2 comments
20.
by svcrunch 1y ago
Various frontier LLMs were evaluated on their ability to interpret handwritten proofreading marks in printed literary text, using a small benchmark based on Charles Dickens's "Little Dorrit". Results are modest at best, and s
21.
by svcrunch 3y ago
No. But to your point, note that in 2020 neuroscientists introduced the Tolman-Eichenbaum Machine (TEM) [1], a mathematical model of the hippocampus that bears a striking resemblance to transformer architecture. Artem Kirsanov has a very ni
22.
by svcrunch 3y ago
While in Google Research, I worked with two of the authors of the "Attention is All you Need" paper, including the gentleman who chose that title. As others have pointed out, self-attention was already a known concept in the resea
23.
by svcrunch 3y ago
Take a look at the BEIR benchmark, which has served as one of the main drivers for development of neural IR systems since its introduction in 2020. BM25 presents a challenging cross-domain benchmark, and it wasn't till ~2022 that neura
24.
by svcrunch 3y ago
> but HNSW is the best 99% of the time for both performance and latency, and is implemented in almost every modern major vector store. In my experience, HNSW indexes are very expensive to build, relative to indexes like IVF. They also ha
25.
by svcrunch 3y ago
I think your comment is accurate, but regarding your last point: "However, fine-tuning on relevant, high quality, knowledge-rich question/answer pairs seems dominant, when such examples are available or can be generated." How
26.
by svcrunch 3y ago
While transformer-based AI is very powerful, and its potential uses in the business world nearly limitless, the issue of hallucination is holding back adoption. Here, Simon Hughes of Vectara introduces an open-source model, HEM, that can au
27.
by svcrunch 3y ago
The metrics presented in the blog post are those of our production model. When designing Boomerang, we tried to balance latency and search relevance in a manner that strikes the right balance for most use cases. On the other hand, GTR-XXL i
28.
by svcrunch 3y ago
I believe that retrieval-augmented generation is the right path to generative AI within organizations, at least for the next few years. Trying to directly fine-tune an LLM on your data also runs into issues with enforcing access permissions
29.
by svcrunch 3y ago
I'm curious, is HSNW the only option? Do you support IVF-style indexes? Also, FAISS is nice because it supports a pluggable storage layer. Is this something that's easily supported in USearch? Great work, and thank you for your co
30.
by svcrunch 3y ago
Thank you for this! I know that Cruikshank was the original illustrator of many of Dickens's novels, but I prefer the artwork of James Mahoney. As a point of comparison, the same scene by both artists: 1. "Oliver Rather Astonishes

More ›