jononor

4 ms·

31.
by jononor 3mo ago
Dual 5060ti 16gb does over 100 tok/s on 35B A3B. Even with PCIE Gen 4 x4, which quite a lot of motherboards can do. Though Gen 4 x8 or Gen 5 x4 is slightly faster. Misc working notes on this hardware combo here, https://gith
32.
by jononor 4mo ago
Dual color filaments exist, and they do not mix at all... It gives the objects a nice transition when rotated. But indicates that color mixing in the nozzle is probably pretty difficult?
33.
by jononor 4mo ago
That OpenAI was in the wrong when they ignored everyone copyright, does not make it right to ignore their ToU. If a one wants IP and rule of law (incl contracts) to be respected, one should not violate others rights when it is convenient. O
34.
by jononor 4mo ago
A 10'000 hour entry fee does rule out a fair bunch of people though, in practice. While there are few artificial barriers to learning to code, there still are some natural ones, like time.
35.
by jononor 4mo ago
I do not know which is easier. I am not sure that is even well established in research for generative text tasks whether a translation-first or native-language-first is the most sample efficient? But for a national lab I think it is money
36.
by jononor 4mo ago
These models will never compete with frontier models and do not need to - it is about hitting a good-enough, not being the best. Behind the frontier, getting to a certain performance level, is getting easier over time - both sample and comp
37.
by jononor 4mo ago
It would require an investment, but those will pay dividends later, as it becomes easier to train LLMs on/for Norwegian. If we need to translate everything to English we might as well just drop using Norwegian altogether. Practically e
38.
by jononor 4mo ago
WebSerial in Firefox?! Finally! One of the very few things I use chrome for.
39.
by jononor 4mo ago
Yeqh that is a challenge. DDR5 and LPDDR5X are both manufacturable with DUV. So let's hope they still get access to that...
40.
by jononor 4mo ago
As a precondition I think we have to assume that the person in question 1) wants to learn and 2) is smart enough to absorb new info and apply it and 3) reflects enough to adjust their approach when hitting bottlenecks or making mistakes 4)
41.
by jononor 4mo ago
You are correct that bandwidth requirements depends a lot on the exact workload. And that in specific cases, it might be doable to have AM5 for multiple RTX6000Pro. The parent mentioned workloads that are general, and broader than inference
42.
by jononor 4mo ago
Foe multi GPU make sure you have enough PCIE lanes! That rules out consumer grade sockets like AM5, you would need Threadripper or EPYC.
43.
by jononor 4mo ago
There are likely _many_ paths to sustainable business models based on AI tech, that will come to fruition over the next decades. However whether they might not be as profitable as OpenAI and Anthropic are gambling on, is more uncertain.
44.
by jononor 4mo ago
Communication tech/tools enable more people to collaborate. It increases ability for labor that is far away from high value markets to contribute. Same goes for shipping tech wrt physical goods. On the global scale that is empowering t
45.
by jononor 4mo ago
The one true AGI metric!
46.
by jononor 4mo ago
This seems like a viable eval strategy. Presumably finding a bug requires some degree of understanding of the code, beyond just information retrieval. However it probably does not measure things like prompt adherence or ability to create co
47.
by jononor 5mo ago
Dynamic routing is the usual name for the piece that orchestrates which LLM will be used, based on query complexity. There in an open source implementation as part of the vLLM project (and probably others), it is a field of active research
48.
by jononor 5mo ago
Having local AI as a credible threat will keep them on their toes. Which will benefit consumers a lot.
49.
by jononor 5mo ago
Hardware sales would be an excellent business model for open weights. Nvidia is already on it with their Nemotron models. Any new LLM/NPU hardware companies would want to so the same, if noone else does it for them (Chinese labs curren
50.
by jononor 5mo ago
Super info, thanks!
51.
by jononor 5mo ago
Where/how did you buy your DDR4 from SZ? Interested in doing the same, but want reputable source/supplier.
52.
by jononor 5mo ago
That is not at all the intention of the ARC team. By ARC teams definition, passing any single ARC-AGI benchmark does not mean that AGI has been achieved. Instead, AGI would be considered achieved when we are no longer able to come up with n
53.
by jononor 5mo ago
Partitioning is not all that expensive. It is definitely worth testing for your specific workload. We use TimescaleDB, which relies heavily on postgres partitions, have a bit under 100 million rows in our active set (last 90 days), across 1
54.
by jononor 5mo ago
Network effect means it will be a huge and risky undertaking, and one needs to solve the bootstrap problem. But the costs of video delivery means that one would have to burn serious cash in the meantime. So it works in tandem. TikTok kinda
55.
by jononor 5mo ago
YouTubes biggest moat the last 10 years is probably more that all the viewers and creators are already there. Any competitor has a huge disadvantage - creators are not interested in a place without viewers, and viewers not in a place withou
56.
by jononor 5mo ago
With enough tokens, all bugs are shallow? :D
57.
by jononor 5mo ago
Yep that is annoying. There are USB-C magnetic charge adapters. It will prevent shit from getting into the slot, and easy to charge magsafe style. And of course you can easily take it out temporarily to use a standard USBC charging cable.
58.
by jononor 5mo ago
Time to migrate off Atlassian, and ban it for any use in the company. You cannot just help yourself to customer data like that. The data is not yours, never was, and never will be. Pay for a service that blatantly rips of our company IP? No
59.
by jononor 5mo ago
A decent amount of software developers and gamers do spend 3000 USD on a PC. That kind of hardware is going go get more and more capable over time wrt genAI models. Of course there will always be a gap to frontier closed hosted models. It i
60.
by jononor 6mo ago
You are the one that claimed the prices of those shuttle services were lower than that of WaferSpace. 7k USD for 1k chips of 20mm2 at 180nm. Is it not the case?

More ›