sanxiyn

10 ms·

121.
by sanxiyn 2y ago
Anthropic publishes system prompts and at least for the case of Claude 3.5 Sonnet 2024-11-22 asking questions is explicit. "Claude engages in authentic conversation by responding to the information provided, asking specific and relevan
122.
by sanxiyn 2y ago
They aren't.
123.
by sanxiyn 2y ago
In context, retail pricing is very meaningful. The next sentence is "lower prices lead to much more use". That is, price elasticity of demand is large, and here price is retail price.
124.
by sanxiyn 2y ago
We don't know OpenAI's actual cost, but presumably Sam does know.
125.
by sanxiyn 2y ago
LLMs actually do have such hunch, they just don't utilize it. You can literally ask them "Would you do better if you started over?" and start over if answer is yes. This works. https://arxiv.org/abs/2410.
126.
by sanxiyn 2y ago
> OpenAI charges $2.5 for 1 million input tokens, or units of data processed by the AI model, while DeepSeek is currently charging $0.014 for the same number of tokens. This is somewhat misleading, because OpenAI price is for uncached an
127.
by sanxiyn 2y ago
It also mentions Dropbox.
128.
by sanxiyn 2y ago
Immaterial. It's not like people live in Delaware, corps do.
129.
by sanxiyn 2y ago
This is golden. Thank you so much.
130.
by sanxiyn 2y ago
Is anyone else claiming to have made copy and paste error in, say, executive orders?
131.
by sanxiyn 2y ago
Sadly, the situation is not yet in the past tense and is in progress. Yoon was impeached and impeachment is being litigated in the Constitutional Court. Yoon is arguing some ridiculous excuses which would have been funny if this was not ser
132.
by sanxiyn 2y ago
Note that $5.5M training cost is for DeepSeek V3. DeepSeek R1 training cost is unknown.
133.
by sanxiyn 2y ago
IQ is a poor predictor of, say, income, in the absolute sense. Correlation is something like 0.4. But compared to what? Compared to personal psychological metrics (of which IQ is one), IQ performs extremely well as a predictor. Things like
134.
by sanxiyn 2y ago
In my experience, even usual software development is mostly writing exceptions. Because there are more ways to be wrong than ways to be right.
135.
by sanxiyn 2y ago
I think the paper was probably done honestly, but also very poorly. They claimed synthesis of 36 new materials. When reviewed, for 24/36 "the predicted structure has ordered cations but there is no evidence for order, and a known,
136.
by sanxiyn 2y ago
In compiler context, "devirtualization" I encountered usually meant compiling a virtual call to a direct call. See for example "Devirtualization in LLVM and Clang" on LLVM blog: https://blog.llvm.org/2017
137.
by sanxiyn 2y ago
No, it isn't. It emphasizes importance of Glorot initialization and ReLU.
138.
by sanxiyn 2y ago
Geoffrey Hinton (now a Nobel Prize winner!) himself did a summary. I think it is the single best summary on this topic. Our labeled datasets were thousands of times too small. Our computers were millions of times too slow. We initia
139.
by sanxiyn 2y ago
Jamba 1.5 Large is 398B params (94B active) and weights are available. https://arxiv.org/abs/2408.12570
140.
Plasmidsaurus: The unreasonable effectiveness of plasmid sequencing as a service(owlposting.com)
4 points by sanxiyn 2y ago|1 comments
141.
by sanxiyn 2y ago
Results get reproduced without code or weights all the time. I note that in this case training data and evaluation benchmarks are public.
142.
by sanxiyn 2y ago
Sounds like DISCO to me, although details differ.
143.
by sanxiyn 2y ago
Limits on automation do not seem reasonable to me.
144.
by sanxiyn 2y ago
GPGPU is instructive. It is not easy, but possible to write software for it. That's why it succeeded.
145.
by sanxiyn 2y ago
Yes, but Linux (hence Android) doesn't have that problem, because its interface is system call not libc.
146.
by sanxiyn 2y ago
The Deep Learning Compiler: A Comprehensive Survey (2020) is in fact a comprehensive-as-of-2020 but now somewhat dated survey of the field. Unfortunately, I am not aware of better overall survey more up to date. https://arxiv.org
147.
by sanxiyn 2y ago
You can look at the code of a substantial project in text, as well as in action, here. https://omar.website/posts/notes-from-dynamicland-geokit/
148.
by sanxiyn 2y ago
Cerebras is known to use TSMC, so your speculation about boutique fab is incorrect. https://cerebras.ai/press-release/cerebras-systems-smashes-t...
149.
by sanxiyn 2y ago
Not even necessarily a loss let alone big, since their comparison includes margin.
150.
by sanxiyn 2y ago
As OpenAI themselves admits, structured output feature in question was developed in open source world first with zero funding.

More ›