llm_trw

18 ms·

181.
by llm_trw 2y ago
I have a $30k computer at home. Are you going to give me one like that at work? If not I'm less productive there and only showing up to polish your ego. My corporate escorts rates are higher than my working rates accordingly.
182.
by llm_trw 2y ago
How much did the hardware to train gpt4 cost?
183.
by llm_trw 2y ago
Because ignoring a heroic effort from all the women in India the number of Indian developers does not double every 4 years. The number of flops a gpu can output on the other hand does.
184.
by llm_trw 2y ago
>The memory and compute requirements to develop and run these models make no sense There is the story that von Neumann flew off the handle the first time he saw an assembler. >>How dare you waste compute cycles on this frivolity? J
185.
by llm_trw 2y ago
49 20 72 61 74 68 65 72 20 74 68 69 6e 6b 20 74 68 61 74 20 74 68 65 20 6e 61 6d 65 73 20 6f 66 20 6b 65 79 77 6f 72 64 73 20 6d 61 74 74 65 72 20 61 20 6c 6f 74 2e
186.
by llm_trw 2y ago
As we have seen here it won't be a Western company that saves us from the dominant monopoly. Xi Jinping, you're our only hope.
187.
by llm_trw 2y ago
The word you're looking for is copyright enfrignment. That's the secret sause that every good model uses.
188.
by llm_trw 2y ago
https://xkcd.com/538/
189.
by llm_trw 2y ago
For the tech discussions I'm interested in burning cpu/GPU cycles for proof of work is a good way to make replies expensive enough that only people who care will post then. Another option is a web of trust. It's finally the y
190.
by llm_trw 2y ago
To misquote Douglas Adams: >In other words—and this is the rock-solid principle that underpins their entire facade—their most profound character flaws are entirely masked by the sheer distraction of their glaringly obvious, superficial c
191.
by llm_trw 2y ago
To a first approximation all breakthroughs are ignored for years until someone else steals them. Transformers weren't scaled at google because it would eat their ad business. We could have had current level AI in 2020 if they'd be
192.
by llm_trw 2y ago
In 2015 Melbourne there were 5 different groups meeting about Linux weekly and one group about every programming language in the top 20 of stack overflow. The endless lockdowns destroyed pretty much all of civic society there and the place
193.
by llm_trw 2y ago
Two years of lock downs killed most of civic society. It used to be that there were several interesting meetups every day. Now you'd be lucky to find one once a month.
194.
by llm_trw 2y ago
>Far from being a US-specific trend, this is sweeping the western world. The share of young people on either side of the Atlantic who regularly meet up socially with friends, family or colleagues has dropped sharply. In Europe, the share
195.
by llm_trw 2y ago
Is this alone or 'alone'? I remember in 2000-2006 spending a ton of my time with friends at a lanshop, because our computers were terrible. It would have counted as in person time, but once we got better computers and internet we
196.
by llm_trw 2y ago
Yeah, I can replicate work out of a paper quite easily if half or more of the contributors have Chinese, Indian or Russian email addresses. A lot of small to medium sized companies fighting for mind share also release one script deploy pipe
197.
by llm_trw 2y ago
The progress isn't happening at Western universities.
198.
by llm_trw 2y ago
In ml basically every paper from a Western university is either useless to industry or not replicable. It is very much a crisis and one that academia has no interest to solve. In a sane world we'd start with figuring out what the myste
199.
by llm_trw 2y ago
It addresses b too since decompositions are always smaller than the original tensor. It's usually the case that memory access is also slower than matrix multiplications so this will be faster. Burning flops to save memory movement.
200.
by llm_trw 2y ago
The attention economy and its consequences.
201.
by llm_trw 2y ago
Only if the paper is handwritten.
202.
by llm_trw 2y ago
> But when they came to writing, Theuth said: “O King, here is something that, once learned, will make the Egyptians wiser and will improve their memory; I have discovered a potion for memory and for wisdom.” Thamus, however, replied: “O
203.
by llm_trw 2y ago
You mean the charity* foundation** Open***AI?
204.
by llm_trw 2y ago
Up to 4 layers is pretty standard these days. It's more than 8 where you start running into issues.
205.
by llm_trw 2y ago
ML papers by Western universities barely touch on the problems that practitioners face. The only papers I see that are routinely useful have half the authors having a .in or .cn email at the end with the rest having Indian and Chinese names
206.
by llm_trw 2y ago
Huh, that's surprisingly not actively misleading.
207.
by llm_trw 2y ago
>NVidia would have to seriously fumble. Worked for AMD in the CPU market. That said I'm deeply worried about anyone whose based their company on amd gpus. The only reason why they do well in hpc is because there's an army of dr
208.
by llm_trw 2y ago
We used to have shops in schools back when funding for education went to education instead of administrators. That's not been the case for a generation so no one under 70 even knows what we've lost.
209.
by llm_trw 2y ago
We go from: Support is coming in three months! To This card is ancient and will be no longer developed for. Buy our brand new card released in three months! Every damned time.
210.
by llm_trw 2y ago
Imagine nvidia supported only the 4090, 4080 and 4070 for cuda at the consumer level. With the 3090 not being supported since the 40xx series came out. This is what amd is defending here.

More ›