ipsum2

43 ms·

721.
by ipsum2 3y ago
Which headphones do you have? It's not hard to replace the battery in most popular brands.
722.
by ipsum2 3y ago
So it's not a super-computer, its a single server with 8 GPUs. Hilarious branding.
723.
by ipsum2 3y ago
commenting about Messenger on Android when the article is about code-based iOS optimizations is not really relevant.
724.
by ipsum2 3y ago
The article is about the iOS version of Messenger.
725.
by ipsum2 3y ago
Yes, it's the same one.
726.
by ipsum2 3y ago
Seems like a waste of time machine, considering this article was written before the blog post.
727.
by ipsum2 3y ago
Article is from 2020.
728.
CodeGen2 Models for Program Synthesis(github.com)
4 points by ipsum2 3y ago|0 comments
729.
by ipsum2 3y ago
Didn't MosaicML do the training for them?
730.
by ipsum2 3y ago
Wow, how did you find this?
731.
by ipsum2 3y ago
Yeah, that's right. This class of problems (what are adequate priors, how to learn them automatically) is under the field of representation learning in ML.
732.
by ipsum2 3y ago
Wrong, Mojo is not a runtime for Python.
733.
by ipsum2 3y ago
No it won't. The language is not a super-set of Python, its another language that somewhat resembles Python. You can't drop in some Python code and run it with Mojo and expect it to work.
734.
by ipsum2 3y ago
First Republic had a market cap of $22 billion 2 months ago. It's not mentioned in the press release, do the investors get nothing?
735.
by ipsum2 3y ago
Very cool, how does the stitching work?
736.
by ipsum2 4y ago
State of the art automatic score transcription is quite bad currently.
737.
by ipsum2 4y ago
Kickstarters like the one you linked don't suffer from the tragedy of the commons because people are essentially just pre-paying for a product. With funding an open source ML model, there's little incentive to not be a free-ride
738.
by ipsum2 4y ago
It's not efficient to do 175B. Training a smaller model (65B) on more data gives better performance for the same compute.
739.
by ipsum2 4y ago
The tragedy of the commons is at play here. We could get amazing ML models rivaling the best if people interested could pool together money for a $100k or $1 million training run. But there doesn't seem to be the willingness, so we hum
740.
by ipsum2 4y ago
To put it in context, in almost all areas of research (physics, biology, chemistry, electronics, etc), running experiments is expensive. ML is in the category that there can still be advances done by amateurs at home. I don't think it&
741.
by ipsum2 4y ago
The approach is different, it can use pretrained models, i.e. stable diffusion, which is a pretty exciting research development. This means that it only requires 'fine-tuning' existing models to get this result.
742.
by ipsum2 4y ago
This is the same approach as Mini-GPT4 of training a linear layer using frozen image encoder and text decoder. the difference is that they use CLIP instead of BLIP, and LLaMA instead of Vicuna (which was trained on LLaMA. Interesting that t
743.
by ipsum2 4y ago
Huh? There's plenty of open source LLMs. Pythia, GPT-NeoX, GPT-J, GPT-2, BLOOM-176, are ones I can think of off the top of my head. Pythia is the best performing one IIRC.
744.
by ipsum2 4y ago
Strange that the author claims Jax's vmap is what's doing the heavy lifting, but doesn't use PyTorch vmap to make the benchmark comparable.
745.
by ipsum2 4y ago
Unfortunately moving to Gitlab or Sourcehut doesn't really help, because the underlying model (GPT-x) is trained on the entire internet, so that includes all scrape-able websites. The only way for your data not to be used in GPT (and t
746.
by ipsum2 4y ago
> though in practice, the model might have a hard time generalizing to much longer context lengths than it saw during training What's the benefit of RWKV then? The results are practically identical to transformers.
747.
by ipsum2 4y ago
Seems like a funnily bad argument, since horse population was reduced by 90% after cars were invented.
748.
by ipsum2 4y ago
> We call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4. How do you quantify this? The model size, architecture, or dataset has not been revealed by OpenAI.
749.
by ipsum2 4y ago
FYI, there's something fishy going on in this thread. Multiple people from the LightningAI team theaniketmaurya (developer advocate for Lightning AI) and rasbt (developer at Lightning AI) are shilling for this post without disclosing t
750.
by ipsum2 4y ago
Everyone except Google uses Nvidia for training. Cerebras, Gaudi, and other custom AI accelerators have unable to surpass Nvidia in performance/$ and performance/watt yet.

More ›