terhechte

7 ms·

61.
by terhechte 1y ago
No they're not. I prefer closed source LLMs because I'm not interested in having a small hardware park, but DeepSeek 0528 or Qwen 3 235 are really good models. Not as good as Claude 4, but absolutely good enough for a lot of devel
62.
by terhechte 1y ago
I sat down at a Piano yesterday and tried to play a beautiful song even though I never really used a piano before. Sounded horrible, must be the pianos fault. One reason to use LLMs is to understand when and how to use them properly. They&#
63.
by terhechte 1y ago
You could split it up in two separate entities. For vector search there's a myriad of good Rust projects. I've personally used: - https://crates.io/crates/lancedb - https://crates.io/crates&#x
64.
by terhechte 1y ago
You can run the 4bit quantized version of it on a M3 Ultra 512GB. That's quite expensive though. Another alternative is a fast CPU with 500GB of DDR5 RAM. That of course, is also not cheap and slower than the M3 Ultra. Or, you buy mult
65.
by terhechte 1y ago
Why did you bundle Redis and not use one of the many key value libraries available for Rust (or even sqlite)?
66.
by terhechte 1y ago
Would this also be possible with other LLM engines / GPUs? E.g. Llama / Apple Silicon or Radeon?
67.
by terhechte 1y ago
But for any coding task.
68.
by terhechte 1y ago
Just at note that > If the user doesn’t know the answer, the LLM can’t help. That’s not just a minor limitation—it’s foundational. doesn't mean the LLM is not useful. My favorite use case for LLMs is them doing something I know 100%
69.
by terhechte 1y ago
Can you give some examples where it didn't work for you? I'm curious because I derive a lot of value from it and my guess is that we're trying very different things with it.
70.
by terhechte 1y ago
I'm also a huge fan. Started watching him after a Japan trip. I like the format so much, I'd love to have something similar (day in the life) for various other countries.
71.
by terhechte 1y ago
It says `Coming Soon` for the `inference.py` for the quantized version. Does anyone happen to know how to modify the non-quantized version [0] to work? [0] https://github.com/Lightricks/LTX-Video/blob/main&#x2
72.
by terhechte 1y ago
No currently not. It would be easy to add though. I haven't updated the tool in a while (after using it to clean up my Gmail inbox). Thanks for pointing out the certificate!
73.
by terhechte 1y ago
I build something to visualize huge amounts of email (such as from Gmail) some years ago: https://github.com/terhechte/postsack
74.
by terhechte 1y ago
There's aider ( https://aider.chat ) or Claude Code ( https://docs.anthropic.com/en/docs/claude-code/overview ) or Codex ( https://github.com/openai/codex ) or plandex ( https
75.
by terhechte 1y ago
In the new agent panel not everything is editable. Maybe give it another try.
76.
by terhechte 1y ago
I'd also like to throw my own app into the mix http://hyperdeck.io
77.
by terhechte 1y ago
This is a very accessible way of playing around with the topic: https://transformerlab.ai
78.
by terhechte 1y ago
Local models, due to their size more than big cloud models, favor popular languages rather than more niche ones. They work fantastic for JavaScript, Python, Bash but much worse at less popular things like Clojure, Nim or Haskell. Powershell
79.
by terhechte 1y ago
Image input has been working with LM Studio for quite some time
80.
by terhechte 1y ago
Or GitHub. I’m always amused when people don’t want to send fractions of their code to a LLM but happily host it on GitHub. All big llm providers offer no-training-on-your-data business plans.
81.
by terhechte 2y ago
Just wanted to say that I was only someone who read OsNews daily back somewhere from 2002 to 2007 or 2008. So many interesting things going on. With the proliferation of web apps and containers, alternative operating systems are actually mo
82.
Give Cursor access to Rust's type system and tooling(github.com)
2 points by terhechte 2y ago|0 comments
83.
by terhechte 2y ago
I don't think the time grows linearly. The more context the slower (at least in my experience because the system has to throttle). I just tried 2k tokens in the same model that I used for the 120k test some weeks ago and processing too
84.
by terhechte 2y ago
I just loaded two models of different quants into LM Studio: qwen 2.5 coder 1.5b @ q4_k_m: 1.21 GB memory qwen 2.5 coder 1.5b @ q8: 1.83 GB memory I always assumed this to be the case (also because of the smaller download sizes) but never r
85.
by terhechte 2y ago
To add, they say about the 400B "Maverick" model: > while achieving comparable results to the new DeepSeek v3 on reasoning and coding If that's true, it will certainly be interesting for some to load up this model on a pri
86.
by terhechte 2y ago
Yes, that's what I tried to express. Large prompts will probably be slow. I tried a 120k prompt once and it took 10min to process. But you still get a ton of world knowledge and fast response times, and smaller prompts will process fas
87.
by terhechte 2y ago
Sure but the upside of Apple Silicon is that larger memory sizes are comparatively cheap (compared to buying the equivalent amount of 5090 or 4090). Also you can download quantizations.
88.
by terhechte 2y ago
The (smaller) Scout model is really attractive for Apple Silicon. It is 109B big but split up into 16 experts. This means that the actual processing happens in 17B. Which means responses will be as fast as current 17B models. I just asked
89.
by terhechte 2y ago
His purse is empty already, all’s golden words are spent. – William Shakespeare
90.
by terhechte 2y ago
How is the moral dilemma of employer-controlled brain implants or eugenics equal to AI? The reason the quote can be applied is because it is a genuinely useful technology to lots of people. That's not the case for eugenics robots. Did

More ›