moffkalast

96 ms·

1861.
by moffkalast 2y ago
Yeah on one hand you have people afraid of wifi and simultaneously sending their kids to play all day outside in UV index 11 without a hat or sunscreen. Things that are "natural" being inherently safe and anything technological li
1862.
by moffkalast 2y ago
Weren't microwave ovens invented because someone's chocolate melted in their pocket while operating a radio transmitter? Maybe it's just a lady godiva story since it seems weird that the person wouldn't feel overly hot a
1863.
by moffkalast 2y ago
Hmm right, I read that wrong. Still, interesting data I think.
1864.
by moffkalast 2y ago
Pretty much, yep. There was definitely a more significant jump there in the middle where 7B models went from being a complete waste of time to actually useful. Then going from being able to craft a sensible response to 80% of questions to 9
1865.
by moffkalast 2y ago
Llama-1, 1T tokens, dumb as a box of rocks Llama-2, 2T tokens, smarter than a box of rocks Mistral-7B, 8T tokens, way smarter than llama-2 Llama-3, 15T tokens, smarter than anything a few times its size Gemma-2, 13T synthetic tokens, slight
1866.
by moffkalast 2y ago
"Beware of shark"
1867.
by moffkalast 2y ago
I mean maybe I'm missing something but it seems like it behaves exactly the same way as subtraction? At least for integers it's definitely the same, for floats I imagine it might handle equals better?
1868.
by moffkalast 2y ago
> C++ spaceship operator > (A <=> B) < 0 is true if A < B > (A <=> B) > 0 is true if A > B > (A <=> B) == 0 is true if A and B are equal/equivalent. TIL of the spaceship operator. Was this adde
1869.
by moffkalast 2y ago
> The idea isn't to replace the compiler with an LLM, the tech is not there yet What do you mean the tech isn't there yet , why would it ever even go into that direction? I mean we do those kinds of things for shits and giggle
1870.
by moffkalast 2y ago
Is it really possible to even disrupt Youtube? It's been a constant in our lives for the past 20 years and is basically a historical record by now. By a rough estimate, they have to keep buying over 1% of the total world production of
1871.
by moffkalast 2y ago
And I get reminded of amogus.
1872.
by moffkalast 2y ago
Turns out LLM alignment is super easy, barely an inconvenience.
1873.
by moffkalast 2y ago
It's multilingual. Genuinely. Compared my results with some people on reddit and the consensus is that the 27B is near perfect in a few obscure languages and likely perfect in most common ones. The 9B is not as good but it's still
1874.
by moffkalast 2y ago
Phi is notorious for benchmark overfitting. It's good, but not as good as it looks on the charts. On the Lmsys leaderboard it places a whole 23 spots behind Llama-3-8B which it also claims to soundly beat on the above. So YMMV.
1875.
by moffkalast 2y ago
I suppose it would act as a concrete separator when instruct tuning, but lots of prompt templates don't use it, especially older ones like Alpaca. Maybe it leads to more overall coherence?
1876.
by moffkalast 2y ago
The 4k sliding window context seems like a controversial choice after Mistral 7B mostly failed at showing any benefits from it. What was the rationale behind that instead of just going for full 8k or 16k?
1877.
by moffkalast 2y ago
> Table 4 | Relevant formatting control tokens used for Gemma models > User turn: user > Model turn: model > Start of conversation turn: <start_of_turn> > End of conversation turn: <end_of_turn> > Beginning of
1878.
by moffkalast 2y ago
"I mean, how hard can it be?"
1879.
by moffkalast 2y ago
If Figma takes a little inspiration from Adobe (they were almost acquired at one point after all) they'll realize they can change their TOS at any time for any reason with impunity. Such statements are at best dubious and at worst comp
1880.
by moffkalast 2y ago
Math is not reality. Physics isn't even reality, they're just theoretical frameworks that are easy to work with and somewhat align with what we've observed. "All models are wrong, but some are useful" as they say. I
1881.
by moffkalast 2y ago
Vista was bloated for its time, but man those widgets it had were so fun. Clock, dual car-like speedometers for CPU and RAM, calendar, weather, stocks, sticky notes. Luckily Android copied the idea soon after.
1882.
by moffkalast 2y ago
Peace was never an option.
1883.
by moffkalast 2y ago
Lightbulbs are probably not the best thing to compare against, they're famously high in energy consumption and terribly inefficient. Even LEDs are absolutely awful and barely crack 30% total efficiency, most of what they make is heat.
1884.
by moffkalast 2y ago
I'm sure the company would've paid him a small bonus, and kept the 99.9999% of the profits from it for themselves.
1885.
by moffkalast 2y ago
Yeah, 5 years in a 2x3m cell with total social isolation? People almost went mad locking down 2 months during covid an they had internet. I doubt he'll ever be the same, or even a functioning person again.
1886.
by moffkalast 2y ago
Maybe they ought to fine Apple 50 cents per installed app from their App Store, since apparently they consider that perfectly fine and reasonable.
1887.
by moffkalast 2y ago
You know what's also great for quick edits and ssh boxes? Nano. No learning curve and it's perfectly cromulent.
1888.
by moffkalast 2y ago
Sure, it happens. How often it happens really depends on so many factors though. For example, I have this setup where a model has some actions defined in its system prompt that it can output when appropriate to trigger actions, and the inte
1889.
by moffkalast 2y ago
I mean I can definitely remember lots of cases for myself, in school especially, when I made the same mistake again repeatedly despite being corrected every time. I'm sure today's language models pale in comparison to your flawles
1890.
by moffkalast 2y ago
Replace "gpt and gemini and all the others" with "people" and funny enough your statement is still perfectly accurate. You have a rough mathematical approximation of what's already a famously unreliable system. Expe

More ›