krzyk

15 ms·

91.
by krzyk 4mo ago
AFAIR the general consensus is (was?): - llama.cpp for single user - vLLM for multi-user (e.g. enterprises) They are similar, but for different use cases.
92.
by krzyk 4mo ago
We need first to reach level of Sonnet 4.x, we aren't at that level yet.
93.
by krzyk 4mo ago
It was quite fun. But at my first work (begining of 2000s) there was one person that made a fun email, using From of head of company (or was it head of that particular division) to his coworker with congratulations for pay increase and prom
94.
by krzyk 4mo ago
I'm a bit new to rust or npm system. But I always thought NPM was what the author describes - just a random set of packages with git sources, which I thought was the main issue (leftpad etc.). Isn't that the case? What about one s
95.
by krzyk 4mo ago
Focusing outrage on politics is pointless. Technical people are rather good at learning new things, and ollama situation is a good learning experience. llama.cpp gets you more tokens/s even if you ignore ollama team bad behavior.
96.
by krzyk 4mo ago
> People say Electron is heavy and not great, but it's also the fastest way to support Mac, Windows, and Linux all at once. But why make an app when websites is enough? And I don't need to run n web browsers for that.
97.
by krzyk 4mo ago
Solar panel breakage doesn't depend on a graphics card.
98.
by krzyk 4mo ago
ollama is a wrapper on top of llama.cpp, and it makes llama.cpp slower, why use it? Also Ollama has other issues (like forgetting what it really is - a wrapper).
99.
by krzyk 4mo ago
Why would anyone use Ollama at all (aside from obvious reasons one can look up online) - llama.cpp used directly, without this wrapper is faster. Basically one has two real choices for local LLMs: llama.cpp (if single user) or vLLM (if mult
100.
by krzyk 4mo ago
Is autocomplete using LLMs really useful? Even with frontier models I found it to be about 50% right, I turned it of and prefer to use IntelliJ built-in, it is way more reliable. For me local models is all about quality, and how to achieve
101.
by krzyk 4mo ago
People are using 3090 (24GB) to run models, and it is the most cost effective way to run the. Yes, it is 2x faster, but memory wise you surely can spend 24gb on llm. Also there are smaller, still usefull models that can run on 8GB or less.
102.
by krzyk 4mo ago
Ollama is not a good choice - https://sleepingrobots.com/dreams/stop-using-ollama/ As for oprncode, doesn't the system prompt eat too much of the context? Local models are really constraint in regards contex,
103.
by krzyk 4mo ago
It is done in the open, but it adds complexity and it removes that made Unix/Linux great - composability, variety and replaces it with corporate introduced "stuff". And any distro is forced to support those additions because
104.
by krzyk 4mo ago
Yeah, this is quite concerning. And FedRamp has some issues with data being sent out. Our corp doesn't allow usage of local models because of concern about potential "agent sends out code to the net" issues.
105.
by krzyk 4mo ago
> As long as the company states the time period But they don't, they have the "30 days", but just after that they add "unless ....". So the time period is vague.
106.
by krzyk 4mo ago
> how long they keep it AFAIR it is not clear, because they write it is "30 days, but ...": > After 30 days, the data is deleted automatically, except in the rare cases where it's part of a safety investigation or we&#x
107.
by krzyk 4mo ago
> Note that Anthropic has committed not to train models on logged data, so I don’t understand some of the concerns here. What exactly is your threat model? That Anthropic would train models contrary to their terms of service? That you tr
108.
by krzyk 4mo ago
If it was more painful for the suing party, it was a good thing. Now all it needs is send some letter to github and they take down content, no questions asked, how is that better? It is for court to decide.
109.
by krzyk 4mo ago
old Siri was always limited in that regard. And Apple Intelligence supports just a fraction of languages: English, Danish, Dutch, French, German, Italian, Norwegian, Portuguese, Spanish, Swedish, Turkish, Vietnamese, Chinese (simplified), C
110.
by krzyk 4mo ago
Unfortunately there are more and more users on Linux and as a result windowsization/macosization of Linux is in progress (systemd, wayland, some scary stuff Poettering is doing with boot, snap/flatpack).
111.
by krzyk 4mo ago
Oh, I hoped it would be macos contained in those containers. Container for Linux are in millions, while I don't know if there are any with macos inside.
112.
by krzyk 4mo ago
Has anyone seen a recent LLM release that supports just one language?
113.
by krzyk 4mo ago
Not quite. It is up to Apple to design a system in which operators (even Apple) can't see your data. Apparently they designed it in a way that operator can see it (so it is cool if it is Apple, but not cool if it is someone else).
114.
by krzyk 4mo ago
Core not, but here it is. Apple designed the system in a way that the operator can invade your privacy. So if only Apple is the operator it is "OK", but if they allow other operators it is not.
115.
by krzyk 4mo ago
> As a small developer, the cost to support something like this would be so overwhelming I wouldn’t consider supporting the EU officially. Would you consider supporting US laws?
116.
by krzyk 4mo ago
Doesn't "DMCA", make, well, the DMCA notice a thing?
117.
by krzyk 4mo ago
If you install LSP. AFAIR their first versions used some kind of treee/structure for easy search, but they found out that grep was better/similar but with less complications (they now ship some kind of grep).
118.
by krzyk 4mo ago
But EU has a pretty uniform laws, so this comes to just one issue: time. Did Apple start working on this feature before EU implemented the law? This might be the case, but even if it was after, they could start working on implementing that
119.
by krzyk 4mo ago
Yeah, Siri was such a poor solution compared to Google (and Google's is also poor in EU) that no one would make a campaign. If Siri wants to be seen as anything it should first support every EU language and they can work from there.
120.
by krzyk 4mo ago
Some features don't land in Europe because US companies can't handle the amount of languages. For them it is English and maybe Spanish or Chinese because they don't care how heybmake money.

More ›