golol

27 ms·

361.
by golol 3y ago
I didn't notice that is a Pareto distribution! Well, that is quite nice indeed.
362.
by golol 3y ago
Let f: R -> R be the probability density function of X. It fulfills int f(x) dx = 1 and we furthermore assume that f is differentiable. We can write the identity as int_a^oo x f(x) dx / int_a^oo f(x) dx = 2 a. We rewrite th
363.
by golol 3y ago
It seems like the Lindy effect for a positive real number valued ramdom variable X, is the following identity E[X | X >= a] = 2 * a
364.
by golol 3y ago
I use the same strategy. When I see an information grabbing headline I note that "X may be true" but unless this affects my life or it is a subject I'm interested in I'm perfectly fine with keeping it uncertain.
365.
by golol 3y ago
Imo LLMs are absolutely the CYC dream come true. Common sense rules are learned from the data instead of hand written.
366.
by golol 3y ago
this is the same concept as the bitter lesson, am I correct? I don't see a substantial difference yet.
367.
by golol 3y ago
Nice to see number of levels of reasoning depth mentioned. I personally believe the size of a (well-trained) LLM determines how many steps of reasoning in sequence it can approximate. Newer models get deeper and deeper, giving them deeper r
368.
by golol 3y ago
How is this original work? I bet you can find 100 implementations of some kind of recursive summarization memory for LLMs on Github, one of them on mine.
369.
by golol 3y ago
I'm just thinking of google's GATO, it seemed to have absolutely no trouble integrating a large number of modes.
370.
by golol 3y ago
I don't really see it. O(N^2) for context window length is not an issue as you don't need particularly much longer context windows than for text. You don't have to run this at 30 FPS, 12 would already be enough to understand
371.
by golol 3y ago
I have never managed to understand why you need univalent foundations as opposed to just using some dependent type theory like Lean. As far as I see Lean could do all the mathematics I could imagine. I can only imagine that univalent founda
372.
by golol 3y ago
I don't understand why Yann is so focused on his "Animals are smarter than AI" analogy. If compute wasn't so limited, couldn't we just train a transformer on video, audio and text data? I don't see why it would
373.
by golol 3y ago
ChatGPT (3.5) is not state of the art
374.
by golol 3y ago
>LLMs won't get intelligent. That's a fact based on their MO. They are sequence completion engines. They can be fine tuned to specific tasks, but at their core, they remain stochastic parrots. This is absolutely wrong. There is
375.
by golol 3y ago
I don't think there is a magic alrogithm at play. I think it's all about studying the data - the geometry of the data manifold. In the case of a transformer more precisely learning the geometry of the stochastic process of tokens.
376.
by golol 3y ago
This is the real breakthrough of LLMs imo. Karpathy had a series of tweest about this a year ago. It makes sense: Prettained LLMs learn to continue sequences of tokens. These are language tokens but the language is so complex and rich that
377.
by golol 3y ago
0.1% chance My reasons are: - I don't recall seeing any evidence that OpenAI has included new data in pretraining beyond the previous limit (Sept. 2021?) for GPT-3.5 or GPT-4 - Maybe they did finetuning or RLHF on new data but this is
378.
by golol 3y ago
> Can you share your knowledge with millions at once? Yes, I might run a course or something. You are still not entitled to pay.
379.
by golol 3y ago
But a model learning from data and reproducing it in some fashion is absolutely not perfect information transfer.
380.
by golol 3y ago
It should be treated as learing. If it truly stores and reproduces a photo (to some high accuracy), then there are already laws in place that handle this. Your client using the output may infringe on the photographer's rights, which ma
381.
by golol 3y ago
I find it very strange to think that you are entitled to anything in return when something views and processes content you have publicly shared.
382.
by golol 3y ago
Also the Martian iirc.
383.
by golol 3y ago
Yea I'm being a bit cheeky. I just want to push back a bit against hard sci-fi. It can be really interesting to come up with new science as opposed to just coming up with new engineering given known science. Greg Egan books made me rea
384.
by golol 3y ago
IMO if there is no fictional science in your story, then it is engineering fiction, not science fiction.
385.
by golol 3y ago
People are just having fun entertaining the very real possible and "believing". What's wrong with that? It is more interesting to just go ahead and say that they are 100% sure LK-99 is real whent here are no stakes. Having di
386.
by golol 3y ago
It is simple as in doesn't require expensive materials or tools or a particularly long process. It can nevertheless be tricky to execute correctly and depend on luck
387.
by golol 3y ago
Reads like a T-online article lol
388.
by golol 3y ago
thank you :)
389.
by golol 3y ago
The common understanding seems to be that producing true LK99 is very difficult as you need to get the crystal structure just right. Many samples probably have mixes of the correct and other structures. That's why some float and some d
390.
by golol 3y ago
Eh. To me it looks like the authors extends "lines-as-edges" explanation of why we can understand line drawings to the claim that the visual cortex works almost only on edge detection, which is obviously bunk and so easily refuted

More ›