amilios

11 ms·

91.
by amilios 3y ago
I mean, it's a well known fact that LLMs do badly in the "long tail" of the distribution: anything they haven't seen many many times (e.g. relatively obscure specialist knowledge), they are definitely very likely to get
92.
by amilios 3y ago
To clarify for people familiar with the names but not what corresponds to what: BERT - Encoder-only - embeddings for downstream tasks GPT/OPT/etc - Decoder-only - language generation T5/T0 - Encoder-decoder. Kind of does both
93.
by amilios 3y ago
Very interesting. I'll admit I really enjoyed the contrasting reviewer judgements, having the counter-perspective right beside the report text. Is this standard practice for US Gov reports? If not it absolutely should be!
94.
Idefics: An Open Reproduction of State-of-the-Art Visual Language Model(huggingface.co)
3 points by amilios 3y ago|0 comments
95.
Neo 360: Neural Fields for Sparse View Synthesis of Outdoor Scenes(zubair-irshad.github.io)
1 points by amilios 3y ago|0 comments
96.
Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models(arxiv.org)
4 points by amilios 3y ago|1 comments
97.
by amilios 3y ago
I see no issue with this to be honest. Would we consider someone using VSCode in a company where everyone is still using nano as "cheating", because they decided to use a superior tool to augment their productivity? Is using Githu
98.
by amilios 3y ago
One way to improve this issue is to focus on retrieval-augmented models, i.e. models that include an explicit retrieval step from a corpus of documents and try to "cite" their sources as they produce text. However then oftentimes
99.
by amilios 3y ago
Anyone able to give a quick rundown how this works without modifying the firmware?
100.
by amilios 3y ago
In terms of the origin of a lot of the slang being from marginalized communities (LGBT) or communities of color, it's really interesting to think how much this has been driven by the internet, and the increased range that people produc
101.
by amilios 3y ago
It's incredible how far we've come with quantization and parameter-efficient fine-tuning a la QLora. It's legitimately possible to fine-tune multi-billion parameter models on Google Colab or a consumer GPU. I can see how some
102.
by amilios 3y ago
Would be wonderful if Congress or the US Gov did something like the EU: https://social.network.europa.eu/ A common ActivityPub server for all EU institutions.
103.
by amilios 3y ago
I feel like a big part of this is just affordability -- at a time when more and more people are struggling financially, busses are a more attractive option in terms of cost than flights in lots of cases. Simple as that
104.
by amilios 3y ago
For? Web? ML? It's like asking what vehicle people use, depends on the purpose!!
105.
by amilios 3y ago
Is there any law or anything else binding that requires Twitter to compensate users for confiscating their handles? If not, is this really surprising?
106.
by amilios 3y ago
The thing is, even if the risk of myocarditis specifically is higher in young men with the vacccine vs. with infection, the vaccine protects against much more than just myocarditis. So even if the myocarditis risk goes up (relatively slight
107.
by amilios 3y ago
Hmm this is a good point, diffusion requires several (many?) inference passes as you refine the noise into an image, right? Makes sense that this is more expensive to scale up. Thanks for the explanation!
108.
by amilios 3y ago
I always wondered why the vision models don't seem to be following the whole "scale up as much as possible" mantra that has defined the language models of the past few years (to the same extent). Even 3.5 billion parameters i
109.
by amilios 3y ago
Yeah unfortunately I think this is the result of the stochasticity of sampling from the LLMs with non-zero temperature, it'll give a different answer every time and some answers might trigger the guardrails and others might not. I am c
110.
by amilios 3y ago
Interesting. It worked for me! Response: Early stage venture capital firms typically have a few non-investor roles that are critical to their operations. Here are some of the most common non-investor roles at early stage venture capital fir
111.
by amilios 3y ago
Personally I think it's a combination of a) much better resources being available than in the past, which explain concepts clearly and in relatively simple terms, and b) much better tooling existing now than ever before, so many things
112.
by amilios 3y ago
At this point a lot of people use "ChatGPT" to refer to the ChatGPT model with the GPT-3.5 backend
113.
by amilios 3y ago
Is this with the 4-bit quantization? The only issue with it is that inference is incredibly slow with it on right now, but that should be fixed up in the next few weeks I think.
114.
by amilios 3y ago
How much GPU memory do you have access to? If you can run it, Guanaco-65B is probably as close as you can get in terms of something publicly available. https://github.com/artidoro/qlora . But as other comments mention,
115.
by amilios 3y ago
It probably does have some sort of simplified internal model of the world, who's to say it doesn't?
116.
by amilios 3y ago
Does it? I loaded Guanaco-65B on a 48GB graphics card using the new flag and it loaded just fine. I don't think the unquantized model fits on a 48GB card. So I'm not quite sure if this is true.
117.
by amilios 3y ago
The phrase in the original homeric greek is "οίνωψ πόντος". "οίνωψ" is a combination of "οίνος" = wine, and ωψ = "view/look" (related to the modern όψη). So it's not wine-"colored"
118.
by amilios 3y ago
You want a plausible scenario for destruction via AI? Here's one. Some idiot governments put a neural network behind missile defense or nukes. Because ANNs are uninterpretable black boxes, at some point the system does a counterdefense
119.
by amilios 4y ago
https://www.reddit.com/r/OpenAI/comments/12696oq/im_dating_a... This one is wild. I expected that Black Mirror episode to become real eventually, but definitely not so soon...
120.
by amilios 4y ago
Yes, the model has something like an EOF token which it emits for the output to end. It is part of the probability distribution that the model predicts.

More ›