Zacharias030

11 ms·

91.
by Zacharias030 1y ago
Those must have been about other things than LLMs though. Meta has huge inference loads for other types of models.
92.
by Zacharias030 1y ago
The crypto mind cannot comprehend
93.
by Zacharias030 1y ago
This was a hot topic in 2019-2020 and I just thought I‘ve read an article like this back then from NYU and another from the National University of Singapore, because I and many others were at the time independently working on graph problems
94.
by Zacharias030 1y ago
This is changing rapidly. Google announced they are serving 500T tokens per month. State of the art models are currently trained with less than 30T tokens. Even if training tokens are more costly to run (eg, a factor of 3x for forward, back
95.
by Zacharias030 1y ago
I like how the smallest Eurocentrism is greeted with the wagging finger to be inclusive on hackernews, while the expectation is that 50 state acronyms are well understood by any reader from Lazio, Lorraine, or Thuringia ;)
96.
by Zacharias030 2y ago
HN at its best! What do you do for a living, Sir,
97.
by Zacharias030 2y ago
I‘ve only seen it done by hedge funds so far. What were you referring to?
98.
by Zacharias030 2y ago
I believe that if the reader is familiar with PPO, they will immediately understand GRPO as well. I‘ve heard people say that GRPO giving a zero gradient in cases where neither the current sample nor the group scores give any reward is advan
99.
by Zacharias030 2y ago
Actually no! If we take their paper at face value, the crucial innovation to get a strong model with efficiency is their much reduced KV cache and their MoE approach: - where a standard model needs to store two large vectors for each token
100.
by Zacharias030 2y ago
do you?
101.
by Zacharias030 2y ago
If you don’t like the title, wait till you see this acronym: „… we introduce the Tensor ProducT ATTenTion Transformer (T6), a new model architecture…“
102.
by Zacharias030 2y ago
I‘m not sure I believe that when just yesterday, I ran a bunch of data analysis, simulation, and visualization based on a single csv and produced 5-10 decent matplotlib plots in a 90 minute back and forth between OpenAI canvas, vscode, pyth
103.
by Zacharias030 2y ago
My iOS Safari has it. I turned it on for the NYT, because I wanted a dark theme and then turned it off again because I realized that I like what they do with their pages (still have an ad blocker turned on though, because subscribers still
104.
by Zacharias030 2y ago
What was the service called? Never heard of it.
105.
by Zacharias030 2y ago
https://archive.is/7qeD5
106.
by Zacharias030 2y ago
Love this answer.
107.
by Zacharias030 2y ago
I know from personal experience that it is tiring for the knees. What makes you think that it is „bad“?
108.
by Zacharias030 2y ago
I find this a very interesting observation! (Don’t have a dog) Note though, that the paper describes the different speeds that are possible when walking with equal efficiency to be quite narrow in comparison to running. There is this relate
109.
by Zacharias030 2y ago
What kind of „AI“ / machine learning was used?
110.
by Zacharias030 2y ago
This problem was treated with a bit more sophistication by Harvards Lieberman in „A story of the human body“ and the corresponding nature article [0] establishing persistence hunting as an ancestoral hunting technique of homo sapien. See fi
111.
by Zacharias030 2y ago
Could you point me to some references to learn about his?
112.
by Zacharias030 2y ago
Note that this is a „small o“, so o(1) captures terms that „divided by 1“ go to zero as m to infinity. https://de.m.wikipedia.org/wiki/Landau-Symbole
113.
by Zacharias030 2y ago
This sounds like a detailed account of burnout. Take some time away from it all and find your thunder and curiosity again?
114.
by Zacharias030 2y ago
Consider that climate change doesn’t care for percentages, but total CO2equivalents. Coal is still the third-fastest GROWING energy source in TWh (from the article), so the question is really 90% of what?
115.
by Zacharias030 2y ago
this.
116.
by Zacharias030 2y ago
John Carmack, https://en.m.wikipedia.org/wiki/John_Carmack
117.
by Zacharias030 2y ago
Interesting! Can you elaborate?
118.
by Zacharias030 3y ago
definitely not :)
119.
by Zacharias030 3y ago
I have used ghstack for exactly the problem you describe, but I like how graphite managed trees of stacks, which is better than ghstack‘s linear stacks
120.
by Zacharias030 3y ago
Wouldn’t it be beneficial to get the 9 refactors merged even though the 10th commit needs iteration? It would make it much easier for colleagues that can then branch off of the tested and merged refactors without running into merge conflict

More ›