in-silico

6 ms·

31.
by in-silico 5mo ago
While there is a limit to the amount of information you can fit in a fixed-size state, the theoretical ceiling is pretty high. A Hebbian associative matrix (one of the simplest and weakest memory constructions) can store about 0.7 bits of i
32.
by in-silico 5mo ago
They basically just added DeltaNet hypernetworks to existing LLMs. Nothing super novel or groundbreaking, but a moderately interesting read.
33.
by in-silico 5mo ago
> Keep in mind that all LLMs are trained first on text and then fine-tuned on code. No, they are trained on a mixture of text and code from the start.
34.
by in-silico 5mo ago
What about things like AlphaZero and Atari gameplay, where the model has zero prior knowledge and learns superhuman ability purely using RL? With sufficient RL sampling/training, there's no reason an LLM couldn't similarly de
35.
by in-silico 5mo ago
What does RL do then if not discover strategies and solutions that weren't in its training data?
36.
by in-silico 5mo ago
This is true after pretraining, but reinforcement learning allows the model to discover strategies and ideas that weren't in its training corpus.
37.
by in-silico 5mo ago
I wonder how different their method actually is from other sub-quadratic sparse attention methods like Reformer [1] and Routing Transformer [2]. [1]: https://arxiv.org/abs/2001.04451 [2]: https://arxiv.org&#
38.
by in-silico 5mo ago
Using the logarithmic mean of your range of about 3 kg of CO2 per day, and the fact that the average car emits about 0.2 kg of CO2 per km, this means that a typical day of Gemini coding produces about the same amount of CO2 as a 15 km (~9 m
39.
by in-silico 5mo ago
The ARC-AGI benchmark is basically this already
40.
by in-silico 6mo ago
People are trying to solve it with software too, even if you don't hear about it. The most high-profile example is the latest set of Qwen models, which replace most of the attention mechanisms with Gated DeltaNet (which uses constant m
41.
by in-silico 6mo ago
Modern kv caches can contain up to 1 million tokens (~3000 pages of text). It's not that short, it's like 48 straight hours of reading.
42.
by in-silico 6mo ago
Either someone hard-coded it in a system prompt to the reward model (similar to how they hard-coded it out), or the reward model mixed up some kind of correlation/causation in the human preference data (goblins are often found in good
43.
by in-silico 6mo ago
Earlier in the article: > In 2014, when Magee attached electrodes to rodents to record their neural activity,
44.
by in-silico 6mo ago
For one, everything humans' "intelligence" knows about solving the problem is contained within the finite brain size for the particular person and life. Unless the memory contents of the brain are being saved to storage and r
45.
by in-silico 6mo ago
This is great to see. I did trained some research models using the existing PyTorch/XLA on TPUs, and it was a mess of undocumented behavior and bugs (silently hanging after 8 hours of training!). If anyone is trying to use PyTorch on T
46.
Recursive Superintelligence Raises $500M Funding Round at $4B Valuation(ft.com)
1 points by in-silico 6mo ago|0 comments
47.
by in-silico 6mo ago
What would have to change about future systems to make you think they're conscious in a way that modern systems aren't? That is to say, what evidence would you need from a system in order to think that it's conscious?
48.
by in-silico 6mo ago
What about robots? Not necessarily humanoid robots, but the classic RL demonstrations that can scurry around and achieve simple goals? In the computational functionalist argument, the thing that we share with cats, pigeons, and robots (and
49.
by in-silico 6mo ago
So it's more of a semantic argument than an actual rejection of the idea that you experience qualia/sentience/something?
50.
by in-silico 6mo ago
I'm curious about your doubting your own consciousness statement, given that "we humans are conscious" is pretty axiomic to its definition and one of the few pieces that most agree with.
51.
by in-silico 6mo ago
From my observations, there are generally four camps in the machine consciousness discussion: 1. People who haven't really thought about it, and assume they're conscious because they talk like a human. 2. People who haven't r
52.
by in-silico 6mo ago
As measured by #_no_answer/(#_incorrect + #_no_answer) the top current models can do it 60-70% of the time (Grok 4.20 is the best with 83%): https://artificialanalysis.ai/evaluations/omniscience
53.
by in-silico 6mo ago
The purpose of modern prediction markets is to transfer money from gamblers to the market broker. Communicating insider knowledge is how the brokers legally/morally justify that business model.
54.
by in-silico 6mo ago
This assumes that the model's behavior and memories are faithful to their english/human language representation, and don't stray into (even subtle) "neuralese".
55.
by in-silico 7mo ago
Model-free RL doesn't mean you aren't training a model. It means that you aren't explicitly building a model of the environment's f(s,a)=(s',r) transition function, which methods like Dreamer do. Q-learning only app
56.
by in-silico 7mo ago
In theory you do lose information compared to parameters with more bits. In practice, neural networks aren't able to store much more than 2-4 bits of useful information per parameter (regardless of the precision), so models like this
57.
by in-silico 7mo ago
> The bellman equations (exactly as written above) are not found in ML libraries. This is because they work assuming you know a model of the data. Most real world RL is model-free RL. Q-learning (the usual application of the Bellman equa
58.
by in-silico 7mo ago
> generative-AI-is-strictly-limited-to-training-data Anyone who understands reinforcement learning already knows that's not the case.
59.
by in-silico 7mo ago
I'm not accusing you in particular, but I feel like there's a lot of circular reasoning around this point. Something like: AI can't discover "new math" -> AI discovers something -> since it was discovered by A
60.
Epoch confirms GPT5.4 Pro solved a frontier math open problem(epoch.ai)
480 points by in-silico 7mo ago|706 comments

More ›