sanxiyn

28 ms·

481.
by sanxiyn 3y ago
"5 nm" does not correspond to any physical dimension, so it is nonsensical to compare to size of atom.
482.
by sanxiyn 3y ago
Transistor is scaling, for now. SRAM is not! TSMC N3E SRAM density is equal to N5 SRAM density. This is something of an inflection point. https://www.tomshardware.com/news/no-sram-scaling-implies-on...
483.
by sanxiyn 3y ago
I mean, evolution is also "highly contested". Controversy surrounding "the idea of IQ" is as interesting as those around evolution, in other words, not at all. Scientifically, it is a closed case. Being highly contested
484.
by sanxiyn 3y ago
Yes, a ton. I recommend https://www.amazon.com/Intelligence-That-Matters-Stuart-Ritc...
485.
by sanxiyn 3y ago
I agree. It is clearly a lie and it is unfortunate Bard is spreading misinformation.
486.
by sanxiyn 3y ago
I agree it is bad for researchers, but I think you should consider "comments like this" are coming from users. AI was a highly unusual field in terms of sharing latest research. Car companies don't share their latest engine r
487.
by sanxiyn 3y ago
Since Chatbot Arena Leaderboard https://lmsys.org/blog/2023-05-10-leaderboard/ agrees with you, it's not just you.
488.
by sanxiyn 3y ago
Well, MPT-7B is also commercially usable and openly licensed: https://www.mosaicml.com/blog/mpt-7b
489.
by sanxiyn 3y ago
Bard's context length is equal to that of GPT-3.5.
490.
by sanxiyn 3y ago
From personal experience, Bard's quality seems to be between GPT-3.5 and GPT-4, closer to GPT-4 if you have to bet. Except when fresh or live data matters, where Bard is clearly superior. (Bard's training data is up to Feb 2023 co
491.
by sanxiyn 3y ago
MIR optimization doesn't just move work from LLVM to Rust, it saves work, because it runs before monomorphization (template specialization in C++).
492.
by sanxiyn 3y ago
Of course it's possible, but the problem is training data. Web crawl doesn't come with color.
493.
by sanxiyn 3y ago
Without pointer provenance, all addresses escape. Just cast integer to pointer. I agree strict aliasing is a relic of the past that should be abolished ASAP, but that's quite different from pointer provenance.
494.
by sanxiyn 3y ago
This is for Google Cloud users. My understanding is that Google mostly uses TPU internally.
495.
by sanxiyn 3y ago
ControlProblem subreddit implements this and I was quite impressed.
496.
by sanxiyn 3y ago
There is no trickery going on for MPT. The model is open source. MPT-7B-StoryWriter-65k+ was trained on books with 65k context length so there is nothing not "real" about it. The point of ALiBi is that you can reuse training you d
497.
by sanxiyn 3y ago
On chat.openai.com below text box there is a link to ChatGPT release notes. Current link text is "ChatGPT May 3 Version". The link leads to https://help.openai.com/en/articles/6825453-chatgpt-release-...
498.
by sanxiyn 3y ago
The decision needs to be made before starting training. Maybe there is a clever way to add it after the fact in the style of LoRA? First, that would be a different method in its own right (just as LoRA is), second, I can't see how to d
499.
by sanxiyn 3y ago
No. As far as I know, they haven't said anything about this. Neither did OpenAI about gpt-4-32k. MosaicML did say something about MPT-7B-StoryWriter-65k+: https://www.mosaicml.com/blog/mpt-7b . They are using ALiBi
500.
by sanxiyn 3y ago
Google models are now available as API on Google Cloud.
501.
by sanxiyn 3y ago
Agreed. This is an important feature. Not all people speak English.
502.
by sanxiyn 3y ago
> Keeping ownership over bits shipped to a customer is solved with the law rather than technical means. Yes in theory... In practice, what happened with LLaMA showed people will copy and distribute weights while ignoring the license.
503.
by sanxiyn 3y ago
We don't know (both for previous model LaMDA and new model PaLM 2), but it is less important for Bard because Bard has access to live data from Google search.
504.
by sanxiyn 3y ago
No, I am not saying that. Since PaLM 2 is a transformer model (they didn't disclose almost anything about the model architecture, but they did disclose that), it has a context length limit. What I am saying is that you can't infer
505.
by sanxiyn 3y ago
I agree that if training data is what matters, it is likely that no one can compete with Google with Google Books, which scanned 25 million volumes (source: http://www.nytimes.com/2015/10/29/arts/internat
506.
by sanxiyn 3y ago
No. Autoregressive models don't have model specific limit to output tokens, it's just when to stop looping.
507.
by sanxiyn 3y ago
It is plausible that they were seeing the same thing.
508.
by sanxiyn 3y ago
It probably is that, just not officially. You know, officially, BFR stands for Big Falcon Rocket.
509.
by sanxiyn 3y ago
Well, that's why I said public. Personally, I don't think release notes https://help.openai.com/en/articles/6825453-chatgpt-release-... hinted at any such thing, and I think quantization is more likely t
510.
by sanxiyn 3y ago
Yes, DistilBERT https://arxiv.org/abs/1910.01108 is in fact the closest case I know of. But it is too small (distilling from 110M to 66M) and both BERT and DistilBERT is intended to be used (and benchmarked) with separ

More ›