cztomsik

9 ms·

91.
by cztomsik 3y ago
No. Rust forces you to spend endless hours doing mental gymnastics which shouldn't be needed in the first place (linked data-structures, owned arenas, or even just boxed slices are impossible in safe rust). And you just keep refactorin
92.
by cztomsik 3y ago
Small context window means you cannot embed the whole document, you are embedding just a part. So, if there is some information at the bottom which is dependent on something which is at the top, your embedding could be entirely wrong.
93.
by cztomsik 3y ago
No, usually you don't. Rust has closures, iterators, generics, different traits for operator overloading, smart pointers, etc. Zig doesn't have any of that. It's very interesting combination of low-level, predictable code, wi
94.
by cztomsik 3y ago
This. I have exactly the same experience, I can't believe how much I was able to ship with Zig and the code mostly feels like "done". You can always improve it, but there's no need to. With Rust, I was never happy, even
95.
Ava (run LLMs locally) for Windows(tomsik.cz)
2 points by cztomsik 3y ago|0 comments
96.
Efficient Sidebar Resizing with Preact Signals(tomsik.cz)
1 points by cztomsik 3y ago|0 comments
97.
by cztomsik 3y ago
So? 410M is reported, it does not mean it's actually used. I'm not sure if you are familiar with how virtual memory works but if you are, then I don't understand what's your point.
98.
by cztomsik 3y ago
I have similar feelings. Apple did a wonderful job with M1/M2, but on the software side, I have mixed feelings. Xcode is a mess, and Swift UI is not a complete replacement for Cocoa. And on the OS side, I feel like Apple is creating fe
99.
by cztomsik 3y ago
TLDR; do not trust Activity Monitor. > I gave up Sublime because of memory usage Isn't this just because of GPU-mapped memory/textures? Because you can't get much better with any other app, for example text-rendering is no
100.
by cztomsik 3y ago
I can't promise anything but there's apparently some problem with intel macs and I only have older machine here (with catalina) so I might give it a shot and if it works I can publish the build. EDIT: it's too old, sorry.
101.
by cztomsik 3y ago
This is macos application, you download it, double click it and it should work.
102.
by cztomsik 3y ago
Sorry, I thought I have answered already. No, everything runs as one native-like (mix of native and webview) application.
103.
by cztomsik 3y ago
Thank you, and I wish you luck with your project too, of course :)
104.
by cztomsik 3y ago
It depends, what version do you need?
105.
by cztomsik 3y ago
Thanks for the feedback, this is what I was hoping for. > Scrolling up should stop auto-scroll in the chat This is at the top-of the list for today, check the website/discord/twitter for updates. I want to do both - pin the but
106.
by cztomsik 3y ago
Exactly! I don't want to subscribe to anything, I want to pay once and have the license forever (for that specific version + bugfixes). But that makes OSS more complicated - people usually don't pay for something they can clone &a
107.
by cztomsik 3y ago
Thank you for your kind words, I couldn't say it better.
108.
by cztomsik 3y ago
Ok, let's try this, if this was (fully) open-source, would you pay $5/month for an access to a premium discord/formus with videos, examples, ability to propose new features or vote on the backlog?
109.
by cztomsik 3y ago
Not really, my closest competition is LM Studio, which is closed-source too. There's also Machato, which is also closed-source (and not even free).
110.
by cztomsik 3y ago
> IMHO, at some point, it's just about trust Yes, but you are also literally going to multiply several billions numbers together and pray for a random outcome. I don't know, maybe some middle ground would be feasible - like hav
111.
by cztomsik 3y ago
Thanks :) love your podcast BTW. See https://news.ycombinator.com/item?id=37562287 - I can't tell much more yet, but I feel like LLMs are way more useful than for a chat-bot or a vscode extension.
112.
by cztomsik 3y ago
No, you are right, it's a new term I made up. The idea is that I see this as a RAD tool (for non-programmers). So you would have this app open all the time and it would serve different tasks you have prepared before. But that part is n
113.
by cztomsik 3y ago
Fair point, but if this was open-source, would you really go and read all of the source code in several programming languages and also audit all of the dependencies? IMHO, at some point, it's just about trust. Now, to be 100% honest, I
114.
by cztomsik 3y ago
It has playground with prompt saving and (simple) templates, chat saving, builtin model downloader. It's also very small, most people don't care about that, but I do.
115.
by cztomsik 3y ago
It works with current version of GGUF models for llama.cpp - you can find them on huggingface, or you can convert them manually. Only a few download links are baked-in at the moment but whatever *.gguf file you put in your Downloads folder
116.
Show HN: macOS GUI for running LLMs locally(avapls.com)
58 points by cztomsik 3y ago|47 comments
117.
by cztomsik 3y ago
Agreed 100%, and some of the "simple" approaches like repetition penalty, can harm output, because for example markdown tables repeat a lot of characters, and if you just blindly apply repeat penalty, it will just stop doing what
118.
by cztomsik 3y ago
It is very common for small models, and sometimes even 7B model gets stuck. There was an article/paper showing that GPTs (whole family) quickly get confident in looping if there is anything loop-like in the window. So basically, as soo
119.
by cztomsik 3y ago
FYI llama.cpp can do that for a "while" https://github.com/ggerganov/llama.cpp/pull/1773 Somebody is also working on a whisper.cpp version, which is maybe even more interesting because if you have g
120.
by cztomsik 3y ago
Yes, it's true that you can do arenas in Rust, but they are harder to use in Rust, due to borrow-checking. For example you can't store arena in a struct without lifetime. So in a language which actively makes your life miserable i

More ›