andy12_

4 ms·

1.
by andy12_ 5d ago
I hope so. And I would really like to try an actual Jev open source model. But it will make it more difficult to market it when someone releases something like that because of so many of these "open source Jev-like model".
2.
by andy12_ 5d ago
You can achieve open-vocabulary classification by making the final weights in the softmax come from a category encoder instead of being fixed learned weights. So instead of softmax(encode(input)*learned_weights) You have softmax(encode(inpu
3.
by andy12_ 5d ago
All the people that are just writing an Jev-like API on top of a normal LLM are missing the point. What makes Jev special is the training data; it's how it's trained. The architecture is probably nothing special. Just a text encod
4.
by andy12_ 8d ago
I think the demo is this way simply because looking at the LLM find wacky ways of implementing features without breaking the law is fun and drives the point across. In practice I imagine you would write something like "If you don'
5.
by andy12_ 8d ago
If the LLM changes the laws to bypass them that's on you. The whole point of this is that you don't have to manually review most code written; only the laws. If the LLM changes the laws and you ignore it that's a you problem.
6.
by andy12_ 9d ago
Very cool! You should include in the readme the game Astra implemented from scratch in Bend 2. https://x.com/VictorTaelin/status/2098007807261892927
7.
by andy12_ 16d ago
They need to be loaded into shared memory. The weights might fit in global memory if the VRAM is big enough, but they still need to be moved to shared memory for computation.
8.
by andy12_ 16d ago
Meanwhile, my job commute is a 30 minute walk to the train station or... a 30 minute bus trip to the station (yeah, taking a bus literally saves no time at all). Plus a 50 minute train ride plus another 30 minute walk. Honestly, I would 100
9.
by andy12_ 22d ago
That's a terrible metric, because people going on a vacation probably aren't going there purposefully to commit crimes. What you want to do is look for increases in crime in a given place during holidays https://coolidg
10.
by andy12_ 23d ago
> It sounds more like the models did close to what they were told to do Absolutely not. If I tell a kid to "Get good grades on the next math test" I don't expect the kid to try to kidnap their teacher to extract the next q
11.
by andy12_ 23d ago
I don't want someone to blame. I want agents to be aligned by default. Their good behavior shouldn't depend on all users at all times using them correctly, because everyone will not just[1] use them correctly at all times. > If
12.
by andy12_ 24d ago
> This framing makes it seem like the agents all did this on their own, and the poor hapless engineers at OpenAI couldn't possibly contend with properly sandboxing them. Great, so we can basically ignore AI alignment altogether and
13.
by andy12_ 1mo ago
> I think the fundamental principle is that this approach messes with the distribution in ways that deviate from the trained model. But it doesn't! The distribution doesn't change at all. The only thing that changes is that sa
14.
The Obsessed Encoder(enigma.inc)
1 points by andy12_ 2mo ago|0 comments
15.
Nvidia Vera Rubin NVL72 measured to have 10x TPS per megawatt than Blackwell(twitter.com)
4 points by andy12_ 2mo ago|0 comments
16.
by andy12_ 2mo ago
The automated AI pipeline also had an automatic grading model to try to reduce false positives. But anyway, my point was that in that case the prompt involved was indeed pretty much "hey, ChatGPT, solve an unsolved problem, thanks.&quo
17.
by andy12_ 2mo ago
> but it is worth noting that this wasn't a matter of "ChatGPT, solve this unsolved problem. Make no mistakes." It wasn't the case for this, but when OpenAI disproved the Unit Distance Conjecture, it was really done a
18.
by andy12_ 3mo ago
When Google Maps routes me using a smaller secondary road instead of the main road that I would otherwise have used , I've always wondered whether that significantly changes the amount of traffic that smaller road sees. It's funny
19.
by andy12_ 3mo ago
> Even interns can understand ambiguous asks with a bit of help This is not a case of an ambiguous task. This is literally trying to judge a model based on information it cannot possibly know, like trying to judge someone based on whethe
20.
by andy12_ 3mo ago
It's pretty much confirmed by OpenAI here [1]. > We generally treat GPT-5.5’s safety results as strong proxies for GPT-5.5 Pro, which is the same underlying model using a setting that makes use of parallel test time compute. And Gem
21.
by andy12_ 3mo ago
No, GPTCyber is specifically trained for cybersecurity, and GPT-5.5-pro is just an ensemble of many subagents, not an actual model. Mythos is simply a much bigger model in terms of parameters and I don't think OpenAI will have anything
22.
by andy12_ 3mo ago
I think what's unexpected is that it seems that some cases of model errors are truly caused by the model being misaligned? In the "Catching a model fabricating data" example I would have thought that it was just the model bei
23.
by andy12_ 3mo ago
I think it makes more sense to make it so that major versions are different pretraining runs, and minor versions are simply the same pretraining run that was finetuned to different degrees. But it seems that that isn't cool anymore.
24.
by andy12_ 3mo ago
I mean it as in, train a model across different clusters instead of a centralized cluster. It's been shown that it's possible to train 10B models this way. If more research effort was put into this, that would be great I don'
25.
by andy12_ 3mo ago
To be fair. There is a security concern angle: even open-source models could be trained as sleeper agents that act adversarially (for example, adding backdoors) when used in specific national companies in specific settings. This is very dif
26.
by andy12_ 3mo ago
I'm from Spain and I also hate these projects with passion. Creating models that speak multiple languages is a solved problem. Having each European Nation train its own useless "sovereign model" in its own language is a total
27.
by andy12_ 4mo ago
This is making me extremely depressed. If this was coming from Anthrohpic I would just need to wait for OpenAI to drop a similar model. But if this comes from the US government, they will do the same to OpenAI when the moment comes. Similar
28.
by andy12_ 4mo ago
I don't know if you are aware, but some people reported in Twitter that Fable 5 may flag the message regardless of content if it knows (from either pretraining knowledge or memories) that you work in either of those fields. I don'
29.
by andy12_ 4mo ago
> Performance on benchmarks has practically leveled off Ehm, no? DeepSWE[1] for example shows that new models like gpt-5.5 continue to show big improvements compared to older models. > Also prices are going up. Prices for frontier int
30.
by andy12_ 4mo ago
Claude can indeed decide to terminate conversations on its own using a special tool[1] if it feels "uncomfortable" with how the conversation is going. Also, very famously, in the middle of recording Computer Use demos, Claude stop

More ›