numba888

7 ms·

61.
by numba888 2y ago
That's possible only if they use the same tokens. Which likely requires they share the same tokenizer. Not sure that's the case here, R1 was built on OpenAI closed model's output.
62.
by numba888 2y ago
yep, they didn't do even this a few years back
63.
by numba888 2y ago
> Two studies published in the latest issue of Science have revealed that birds, reptiles, and mammals have developed complex brain circuits independently, despite sharing a common ancestor. They didn't show those 'complex brai
64.
by numba888 2y ago
Small models are just tools. Even many of them will make only a toolset. They don't evolve in AGI by themselves. But putting them together in a structure (brain) may result in something close. Like big smart calculator. It takes more t
65.
by numba888 2y ago
o1 is more than just math solver. And you cannot possibly train that much in a small model. However smaller specialized models looks to be the right way to handle world's complexity. Sort of mixture of experts on one level above. Orche
66.
by numba888 2y ago
Without other groups it's racism. That reminds me my last job. On my question why I'm getting spam from some group I wasn't interested in they responded like "because we are sending it to everyone, ha-ha". Typical w
67.
by numba888 2y ago
The polls show majority doesn't like Trump's idea. Almost half of US understands that. From this side Canada looks like exemplar democracy. No crime, no border, no zillions of migrants/year. Probably we don't know all th
68.
by numba888 2y ago
Hmm.. agent cannot do self-supervised learning without actually doing it. The trick is to keep it in a sandbox.
69.
by numba888 2y ago
> doing AI research means generating new knowledge about AI itself, and I don't see any evidence of that. Wouldn't colorization count as research? In vision domain there are a lot of papers like this. Just arranging and rearran
70.
by numba888 2y ago
> wonder if r1 or 03 can do it, with the right instructions of course. for o3-mini-high it's too difficult to write even simple ebt. not many implementations on the web to learn from(?) reasoning isn't strong enough for this. b
71.
by numba888 2y ago
one xor, yes, but on most systems smaller memory footprint is better. there are many more nodes than iterators. so savings can be significant. another possible optimization is when 'key' fits in pointer or two. which is usually th
72.
by numba888 2y ago
Looks like author never heard about single pointer double linked list. That could save him 2 pointers per each node. The idea: for simplicity taking list. single pointer saved in the node is xor of next and previous nodes. When moving forwa
73.
by numba888 2y ago
Sounds like we need more typists and horses. Will hordes of traffic controllers help, or there is a better way?
74.
by numba888 2y ago
> gave up on Langchain during my first attempt 6 months ago Why? If it's not a secret. I'm just looking for something, not sure actually what... :-\
75.
by numba888 2y ago
> isn't this dangerous? isn't the efficiency given at the expense of safety and interpretability? Final text is only a small part of model's thinking. It's produced from embeddings which probably have much more in the
76.
by numba888 2y ago
> RL spontaneously should create new additions to language to help with Yes, but it may take millions of years. One of the main reasons of LLMs success is their amazing trainability. For every input token it produces predictable output.
77.
by numba888 2y ago
> hings like feeling pain and pleasure can machine feel? without that there is no AGI according to definition above. and the second question: are animals "GI"? they don't have language and don't solve math problems, n
78.
by numba888 2y ago
it's more complicated than this. I mean what you get is defined by what you put in. At first is was random or selected internet garbage + books + docs. I.e. not designed for training. Than was tuning. Now we can use trained model to ge
79.
by numba888 2y ago
> it's always true that training on the output of another model will result in a worse model. Not convincing. You can imagine model doing some primitive thinking and coming to conclusion. Then you can train another model on summarie
80.
by numba888 2y ago
github has a bunch of them for years, the most known from Andrej Karpathy: https://github.com/karpathy/nanoGPT some other have MoE implemented.
81.
by numba888 2y ago
> The judges should apply the law, and the same case should ideally lead to the same verdict regardless of the judge. hmm.. :) I like this. But the reality is very different and some factors which shouldn't matter can change the out
82.
by numba888 2y ago
AIs are new search engines today. So, you need to decide if you want visibility or not. If yes then blocking is like hitting yourself in the balls. While legal it can be painful if you 'succeed'.
83.
by numba888 2y ago
and in the next lifetimes too. humans are good at making problems. only lobotomy and AGI Gods can sort of 'solve' them.
84.
by numba888 2y ago
By the time book comes out it's outdated. DeepSeek has its own cut-off date. And here is the problem: AI needs to be trained on something. Use of AI reduces the use of online forums, some of them are actively blocking access, like redd
85.
by numba888 2y ago
> current LLMs would likely give better verdicts He probably meant _brainwashed_ LLMs. They can consistently produce desired results if you wash them the right way. It's more about personal opinion than computation. Actually it woul
86.
by numba888 2y ago
Today it's within the reach, I think. While effective context window is small for big files it should be possible to use RAGs to get relevant pieces in smaller window. That will be mostly data structures and functions' declaration
87.
by numba888 2y ago
Looks outdated by now. Try R1. Seriously, I've been using LLMs for coding for a while and can say early experience was disappointing, but they get better and better fast. The latest o1 looks a lot better than 4o. It's reasonable t
88.
by numba888 2y ago
anyway it's not dramatic. vs 50 for Amazon. $147 was close to historical max for NVidia. Not fair either. last month in was less than $140 average, just estimate.
89.
by numba888 2y ago
> there's no evidence yet that the deepseek architecture would even yield a substantially more performant model with more compute. It's supposed to. There was an info that the longer length of 'thinking' makes o3 mode
90.
by numba888 2y ago
Looks like panic sell off, but main question is: do models with DeepSeek architecture still scale up? If yes, then bigger clusters will outperform in near future. NVidia wins as tide rises all boats, and them first. If not, then it's s

More ›