3 ms·
If you only need classification, and you can provide some training data, you can ask Codex/Claude to build an embeddings + logistic classifier model for you Fo
by nico 14d ago
If you only need classification, and you can provide some training data, you can ask Codex/Claude to build an embeddings + logistic classifier model for you
For emails, I get 95% accuracy with this method, with only 50-100 examples for training
Training the model takes less than 5 minutes on a CPU
The resulting model is <1MB, and inference is sub 100ms
Some other cool things about this approach:
* the model doesn’t train on some “ideal” or general classification, instead it learns your preferences
* the model runs on pretty much any mobile device and can be retrained online on the device
* privacy, the whole training and inference is 100% local, no data goes anywhere (except whatever you feed codex/claude while building the model)
Note: to do a more general test, I made a classifier for the Banking77 dataset. The model is <10MB, trains in <30s on CPU and gets 94.5% accuracy, which puts it in the top 5?models by accuracy for that set (the best one is at 94.86%, but it’s 350MB in size and takes hours to train on a GPU).
- rgbrgb 14d agolove this idea. did you try comparing to jev?
- nico 14d agoYes, I ran some benchmarks. This architecture seems to match or beat Jev and Laya in all basic classification tasks (datasets tested: AG News, Emotion, MASSIVE Intent, Banking77) The type of task in which it does really well, especially against Laya, is classification with >50 classes But this architecture has no “reasoning”, so it performs rather poorly on tasks that require it, like the ones from the XLNI dataset (Jev/Laya do a lot better on this one) For the latter cases, you could probably enhance the architecture with a lightweight LLM, something like a Gemma model. Or even some basic MLP
- samuel 14d agoDo you realize people is using LLM's as classifiers, right? For lots of companies and developers reaching an API is feasible, while running a training pipeline, no matter how simple, is not. I know that they should still be gathering data for evaluation and they could use it to train a model instead. But they won't do it, for a variety of reasons. This is the same route but WAAAY faster and cheaper. And you can modify it like you do with code or prompts. It's really appealing, TBH.
- 0x457 13d agoI built a whole thing that collects data, trains classifiers, exports models and dataset just for that. Claude writes me a terraform file that contains shape of the classifier and dataset. For images it can create datasets based of another dataset (crop this region from images that have these labels). Originally it was so I can label data to fine-tune a VLM, but now a few tiny classifiers that run in milliseconds on cpu. Now its collecting data to make a domain specific BERT and do what Jev does.
- nico 13d agoVery cool. What kinda of classifications are you running? How big are the models/training sets? Also curious about if you plan on doing some sort of routing for the requests. Like detecting the type of task to decide which model to route the request to
- mjyoke1111 13d ago[flagged]