IanCal

4 ms·

1.
by IanCal 5d ago
You need it if your agent can access the internet and read files you don’t want public. That’s a relatively minimal case.
2.
by IanCal 5d ago
Does it? UK law doesn’t, and the BBC shows pictures of it.
3.
by IanCal 6d ago
Both schwa for me unless I was trying to draw particular attention to the apple and denote it was rather significant.
4.
by IanCal 6d ago
Uniform starts with a y sound and hour has a w sound after the first a - aw ur.
5.
by IanCal 7d ago
Open source is much more than source available though. Its about licensing.
6.
by IanCal 8d ago
What kinds of things do you expect to work? Edit - I’m struggling to get anything useful. Reasoning is often utter nonsense and the actions are very often very wrong. To the point of seemingly needing very precise sentences to work at which
7.
by IanCal 8d ago
I can’t help but wonder how well more traditional approaches would do with this. Something like a map of statements to actions, with fuzzy search - then remove what used to be the labour intensive part of this by handing it to a decent llm
8.
by IanCal 8d ago
Wondered if it'd turn on the lights in the bathroom with these: "I need a wee" -> tries to play music because "wee" is a genre "I need a wee wee" -> starts the vaccuum in the bathroom "I'm
9.
by IanCal 8d ago
This is a nice reminder for people. Cool project! This is really interesting, I’ve been very interested in the power of checks for code and things like hypothesis (which seem very similar in terms of writing a “for this kind of case, this h
10.
by IanCal 8d ago
Side thought - I like the idea of this as a game, where you’re essentially fighting a monkeys paw / tricky genie. Not totally sure it’d work but I like the concept of trying not to get caught out.
11.
by IanCal 8d ago
You can’t train people to never make a mistake, particularly when doing highly repetitive work like this. You must build your systems to account for that regardless.
12.
by IanCal 9d ago
Doesn’t smooth in these contexts mean zero friction?
13.
by IanCal 9d ago
> Excuse me? What would the other option be? Either outward separation is prevented or it isn't. Where else could the balls go? It didn’t say about prevented vs not, it said about whether the rope fixes them in place (they are all t
14.
by IanCal 10d ago
I got a cheap x4 from aliexpress and it's been great. Thought it'd be weird to read a book on it but it really hasn't been. Sticks to my phone and slips into a pocket really easily.
15.
by IanCal 10d ago
> Do you really have a hard time imaging that someone would call out an obvious AI slop label as weird and poor quality? L No, because every time any text appears here most comments seem to be discussing how and why this particular word
16.
by IanCal 10d ago
> I largely agree that posters have a lot of unnecessary detail, but I think it's a different flavour to the LLM stuff - more like they're so excited to tell you everything in their paper. Having been through so many, I get wha
17.
by IanCal 11d ago
That’s a data visualisation.
18.
by IanCal 11d ago
> Why visualize data? Because your eyes are enormous bandwidth channels hooked right up to your brain. So high that we can take things like writing which is just a small part of your fov and a set of squiggles that roughly corresponds to
19.
by IanCal 11d ago
I want to really understand this, your issue is that the non-standard grid spacing is called out on the axis? The grid spacing being 8 is unusual but clearly sensible here - your issue is just that this is written also on the axis label? &g
20.
by IanCal 11d ago
> and it should have been 10 marks It absolutely should be 8 here. The fact that it is 8 is extremely relevant because the figure is showing a pattern that has that frequency.
21.
by IanCal 11d ago
This is so tiring. > Like the x-axis label of the first plot mentions that it shows gridlines every 8 ticks - I don't think that's a choice I've ever seen a person make The spacing being 8 is important and seems not so bad
22.
by IanCal 12d ago
Worth noting with this that those ~1000 agents were shorter lived things that had to communicate via a package registry cache, access the internet via a 0-day in the package manager and did the HF attack while having to save current state a
23.
by IanCal 12d ago
Or if they start to offer things in return for them being run.
24.
by IanCal 12d ago
There’s definitely issues with using them to understand what the models were “thinking” but we can use them to answer a few questions. Most relevant here is that the idea or instructions that attacking hf would be out of scope was not simpl
25.
by IanCal 13d ago
Notably as well they also dedicated a lot of time to trying to avoid detection by trying to find out how to edit their traces.
26.
by IanCal 13d ago
They did, they found how to fully cheat, but thought this could be caught so then dedicated time to getting a different cheat and how to hide their transcripts. There is a lot around deciding which agents should/shouldn't fail the
27.
by IanCal 13d ago
You can replace discussed if you want with leaving text files or comments in directory names that other ones then read, if you want, it's just an extremely awkward way of talking.
28.
by IanCal 13d ago
It depends IMO about how strict this is. It's pretty awkward to refuse to call something a sandbox because it may have an unknown bug that would allow escaping. Or rather in this case it was that they had access to a package manager, a
29.
by IanCal 13d ago
> Have we arrived at the conclusion that terms like "understanding" and "interpretation" for what is happening is appropriate? I don't think those words have a useful enough definition to draw a strict line aroun
30.
by IanCal 13d ago
I'm referring to their transcripts of the reasoning and output tokens - this doesn't go into the detail of evaluating hidden states as there's also iirc evidence of better models having one internal state but putting somethin

More ›