Y
HN Search
Hacker News Search
new
|
comments
|
top
|
jobs
tMcGrath
searching PlanetScale…
1.
▲
2.
▲
3.
▲
4.
▲
5.
▲
6.
▲
5 ms
·
1.
▲
by
tMcGrath
2y ago
Yes - we'd never normally turn features up this much as it breaks the model quite badly, but we put this in the post to show what that looked like in practice.
2.
▲
by
tMcGrath
2y ago
Thank you! I think some of the features we have like conditional steering make SAEs a lot more convenient to use. It also makes using models a lot more like conventional programming. For example, when the model is 'thinking' x, or
3.
▲
by
tMcGrath
2y ago
We'll be open-sourcing these SAEs so you're not required to do this if you'd rather self-host.
4.
▲
by
tMcGrath
2y ago
We're planning to release the weights once we do a moderation pass. Our SAE was trained on LMSys (you can see this in our accompanying post: https://www.goodfire.ai/papers/mapping-latent-spaces-llama/ ). No im
5.
▲
by
tMcGrath
2y ago
I'm one of the authors of this paper - happy to answer any questions you might have.
6.
▲
Look Inside Your AI
(preview.goodfire.ai)
1 points
by
tMcGrath
2y ago
|
0 comments
7.
▲
An empirical approach to AI safety
(banburismus.substack.com)
2 points
by
tMcGrath
3y ago
|
0 comments
8.
▲
Acquisition of Chess Knowledge in AlphaZero
(arxiv.org)
2 points
by
tMcGrath
5y ago
|
0 comments
9.
▲
by
tMcGrath
5y ago
The paper is measuring I/O behaviour, rather than the complexity of the mechanisms generating that behaviour. Transistors might have quite complex physics, but are designed to have relatively simple I/O behaviour.
10.
▲
by
tMcGrath
6y ago
I'm not sure that's all there is to it - at least some scientists are in it for the explanation: they want to know more about something. In this case the prediction is just a useful check that their proposed explanation is compati
11.
▲
by
tMcGrath
8y ago
Modern Classical Physics is a great book - I've recently started working through it (just about to move on to chapter 2). I'd be interested in chatting about it with others/cross-checking solutions. Anyone who's interest
12.
▲
by
tMcGrath
8y ago
As far as I know, a good differential-geometric understanding of nonequilibrium thermodynamics still hasn't been achieved. The central issue is understanding how changes in control parameters (for instance concentrations of catalysts i
13.
▲
by
tMcGrath
8y ago
As I understand it her point it that what we take to be fundamental particles have certain properties - charge, spin, etc - and no others. If they had other properties, this would lead to measurably different outcomes in collision experimen
14.
▲
by
tMcGrath
8y ago
I think the standard reference is probably Spivak's 'Calculus on Manifolds' but this never really did it for me. If you have a background in physics then some combination of Nakahara's 'Geometry, Topology and Physic
15.
▲
by
tMcGrath
9y ago
In this case that's not true though - the article is about someone who used critical thinking/reasoning skills to make a huge amount of money, and credits this in large part to his training in philosophy. It's possible, of co
16.
▲
by
tMcGrath
9y ago
I'd like to flag up this paper on HMC and minibatch inference: https://arxiv.org/abs/1502.01510v1 . It seems that SGHMC might not always behave as well as you might like. I should say this isn't to pour cold w
17.
▲
by
tMcGrath
9y ago
I agree - it's a surprising and cool paper. There has been some work done on fooling network ensembles by constructing constructing a Bayesian posterior over weights using dropout [0]. This is an ensemble of weights for the same networ
18.
▲
by
tMcGrath
9y ago
You are absolutely right that a lot of work to date (especially in applications and metrics-driven research) has focused on 'functions for vision problems'. But this is far from the only idea - that's the point I'm tryin
19.
▲
by
tMcGrath
9y ago
As commenters elsewhere have pointed out, this really only caps salary at around £60k - this is actually pretty low for an experienced ML engineer. I've heard Prof Pantic advocate for salary caps for PhD students leaving academia befor
20.
▲
by
tMcGrath
9y ago
I suppose it's limited by the size of 'the set of things you can do with functions'. This seems like a large enough set that I wouldn't be worried about hitting its limits too soon...
21.
▲
by
tMcGrath
9y ago
There's a reason the axes are left unlabeled in this graph. If you look at a paper that gives values over time for these quantities you'll find that this 'effect' mounts up to about 3 IQ points over the range discussed i
22.
▲
by
tMcGrath
10y ago
There's also an interesting set of applications in statistical physics: many systems can be modelled by Markov chains whose rate parameters can change through time. Changing these parameters non-quasistatically (i.e. so that the system
23.
▲
by
tMcGrath
13y ago
For statistics I can really recommend 'All of Statistics' by Larry Wasserman ( http://www.stat.cmu.edu/~larry/all-of-statistics/ ) It runs through a lot of important topics, particularly in inference, with
24.
▲
by
tMcGrath
13y ago
If you want to approach physics from a mathematical background I strongly recommend Peter Szekeres' book 'A Course in Modern Mathematical Physics' http://www.amazon.co.uk/gp/aw/d/0521829607/ref=redir_mdp_mob...
25.
▲
by
tMcGrath
13y ago
I've been reading this and it's fantastic - approaching the subject from the tools needed to understand the way the material works mathematically (Lagrangians etc) rather than as a series of separate physical problems is an excellent idea.
26.
▲
by
tMcGrath
14y ago
I'm afraid I have to disagree here. As someone who runs a small business which, for not-immediately-relevant reasons isn't in a position to try for startup-style explosive growth, I'd be really interested in hearing about how people in simi