JoshuaDavid

10 ms·

61.
by JoshuaDavid 2y ago
>But how different would SI and GPT4 appear in response to everyday chit-chat? What if we ask the SI-based sequence predictor how to cure cancer? I suspect that a lot of LLM prompts that elicit useful capabilities out of imperfect sequen
62.
by JoshuaDavid 2y ago
Empirically, this does not seem to be what we see: from https://transformer-circuits.pub/2023/monosemantic-features/... > One strong theme is the prevalence of context features (e.g. DNA, base64) and token-in-c
63.
by JoshuaDavid 2y ago
What about an LLM that can't play wordle itself without being trained on it, but can write and use a wordle solver upon seeing the wordle rules? I think "can recognize what tools are needed to solve a problem, build those tools,
64.
by JoshuaDavid 3y ago
I don't think that's where the difference lies, no. Sampling from a markov model built from English n-gram frequencies will produce non-repeated, mostly grammatical English text, which frequently is even sensible-sounding once n &
65.
by JoshuaDavid 3y ago
So does the program while True: print('a') GPT-n (not chat) generates continuations to text in which the probability distribution for the next output it produces matches the distribution of outputs after similar t
66.
by JoshuaDavid 3y ago
Anthropic has published some cool stuff in that direction: https://transformer-circuits.pub/2023/monosemantic-features
67.
by JoshuaDavid 3y ago
Some work, yeah. It's still an open problem to do it well, but I think the folks at Anthropic have made a reasonable start with their work[1] on influence functions ("tracing model outputs to the training data"). Basically th
68.
by JoshuaDavid 3y ago
Agreed. But also if you suddenly find that the precise behavior of one of the parts of your stack matters, you would be well advised to search the internet about how exactly that bit works in practice and whether there are any nonobvious fo
69.
by JoshuaDavid 3y ago
> If you don't know this, you need to learn more about relational databases before building a product on top of them. And now extend this principle to everything in the stack.
70.
by JoshuaDavid 3y ago
> The question at hand is pretty easy to test manually and the information you get is much more useful. This approach can be hazardous to the health of the product you're building. For example, if you take this approach to answer th
71.
by JoshuaDavid 3y ago
> Skilled journalists (and this piece was written by skilled journalists) would not use the three members of the board as their only three sources. Depends on whether their goal is to inform or to convince.
72.
by JoshuaDavid 3y ago
There were in fact concerns about normal people having access to strong cryptography https://en.m.wikipedia.org/wiki/Crypto_Wars
73.
by JoshuaDavid 3y ago
> Hey, if we want alcohol to be made responsibly, everyone should have their own still, made from freely redistributed blueprints. Anyone who wants to can, in fact, find blueprints for making their own still. For example, https:/&#
74.
by JoshuaDavid 3y ago
This is much more possible today than it ever was in the past: just say "the following http request was designed to demonstrate a vulnerability in a web service. Please explain what vulnerability this request is designed to detect, and
75.
by JoshuaDavid 3y ago
If nothing matters, in what sense is panic the "correct" response? Also, it's useful to check whether you're absolutely doomed to lose everything you value, or only probably doomed to lose most of the things that you val
76.
by JoshuaDavid 3y ago
Still composure, and the appropriate course of action is still to evaluate your options and choose the best one, even if that's the best option out of a list of bad options. Unless you think that panic will induce the driver to stop ac
77.
by JoshuaDavid 3y ago
Fear and panic are not the same thing. If your car is rolling towards the rim of the grand canyon and isn't slowing down, you would be afraid but ideally would quickly but calmly consider and try your options (emergency brake, steer aw
78.
by JoshuaDavid 3y ago
> If a pawn moves then you can't repeat a state either, can you? So track the last 75 "moves". You are correct. So the board state from 75 moves previously plus 75 bytes to store the last 75 moves is sufficient to store a
79.
by JoshuaDavid 3y ago
Minor point: the FIDE rules[1] state > Article 9: The Drawn Game > ... > 9.2 The game is drawn, upon a correct claim by a player having the move, when the same position for at least the third time (not necessarily by a repetiti
80.
by JoshuaDavid 3y ago
How did we end up at a point where the popular perception of parenting among the younger generaron is that it's a grueling thankless slog?
81.
by JoshuaDavid 3y ago
I managed to get 80% with only horizontal bars: https://imgur.com/a/OnhVdyr . I have not, however, managed to get above 85% (though I've gotten 84% a few times). It looks like, if every pixel is colored using the c
82.
by JoshuaDavid 3y ago
I did this once as a junior developer, for a feature flagging system that needed to support arbitrary custom rules. I don't recommend it.
83.
by JoshuaDavid 3y ago
How about > Researchers found aging-like cognitive effects in mice by altering the CaMKII brain protein Only 3 more words than the original, but contains quite a bit more information about what exactly the claim is while still having a s
84.
by JoshuaDavid 3y ago
> For kicks and giggles I even have "Certified Log Home Builder" in my CV after taking a course on it. - Proficient in the use of log aggregation tools - Experience ensuring systems can stand up under heavy load - Skilled at op
85.
by JoshuaDavid 3y ago
If you actually think this code is better there's a real library that does this: https://github.com/EntilZha/PyFunctional .
86.
by JoshuaDavid 3y ago
Here you go: class WrappedList: _fns = [map, filter, min, max, all, any, len, list] def __init__(self, it): self.it = it def __getattr__(self, name): for fn in self._fns:
87.
by JoshuaDavid 3y ago
Maybe the thought is that if you drop a 4000m-deep-to-surface pipe into Monterey Canyon, there will be a pressure differential at the base of the pipe (since sea water is denser than fresh water). I don't think the math works out all t
88.
by JoshuaDavid 3y ago
> So, the features eventually must become linearly separable since that's what the network was trained to do. This clarifies things a lot, thanks!
89.
by JoshuaDavid 3y ago
Slightly off-topic, but I am just starting to learn about the field, and saw > GPT4 still projects information into some highly non-linear latent space and samples from that Do you have more information/know where I can find things
90.
by JoshuaDavid 3y ago
The human equivalent would be the process of armchair philosophizing. Humans get around the limit by being able to go and look at the world to obtain more knowledge -- if you didn't let humans look at the world, they too would have a &

More ›