DougBTX

8 ms·

61.
by DougBTX 2y ago
> The actual process of token generation works precisely the same I’d be wary of generalising it like that, it is like saying that all programs run on the same set of CPU instructions. NNs are function approximators, where the code is ex
62.
by DougBTX 2y ago
How do generics change this? Can a type bound on a generic work like a static interface implementation specification, and recover the cost of wrapping the struct?
63.
by DougBTX 2y ago
To me it says that they _do_ train global models with customer data, but they are trying to ensure no data leakage (which will be hard, but maybe not impossible, if they are training with it). The caveats are for “local” models, where you w
64.
by DougBTX 2y ago
No idea of this is even vaguely in the right direction, but this comes to mind: Unsupervised speech-to-speech translation from monolingual data https://research.google/blog/unsupervised-speech-to-speech-t...
65.
by DougBTX 2y ago
There is literally a whole bunch of people very focused on narrow definitions. It seems like cognitive dissonance trying to square the idea that a dog which can fetch a ball is intelligent but a computer which can write poetry isn’t.
66.
by DougBTX 2y ago
Such a strict interpretation would also exclude spaces.
67.
by DougBTX 2y ago
Not really compilers, but the underlying GPU libraries. Here’s a good jumping off point: https://pytorch.org/docs/stable/generated/torch.use_determin...
68.
by DougBTX 2y ago
Same, location identified by the architecture of the buildings in the background and the car's numberplate... of a car with no numberplate driving through a wood.
69.
by DougBTX 2y ago
Yes, probably. At temperature zero the model will be completely deterministic, so a particular prompt will always produce the same result (ignoring for a second that some fairly common optimisations introduce data races in the GPU). On the
70.
by DougBTX 2y ago
No, an algorithm may be written in many ways, each of which would encode to a different Gödel number. > To encode an entire formula, which is a sequence of symbols, Gödel used the following system. Given a sequence (x_1, x_2, x_3, ...,
71.
by DougBTX 2y ago
There are also FAISS binary indexes[0], so it'd be great to compare binary index vs binary index. Otherwise it seems a little misleading to say it is a FAISS vs not FAISS comparison, since really it would be a binary index vs not binar
72.
by DougBTX 2y ago
Absolutely, have been for decades, see expert systems: https://en.m.wikipedia.org/wiki/Expert_system (I take the strong position that code is data, so an if statement in code is equivalent to an if-then rule stored in
73.
by DougBTX 2y ago
Agreed, we shouldn’t use “invariant” as a shorthand for “not expected to change”. We can still talk about “implicit assumptions” and “explicit assumptions”, so the article still works after that replacement. The headline isn’t as punchy tho
74.
by DougBTX 2y ago
The projection logic has an orientation parameter (which you'd probably always want, so that you can pick initial rotation), so the rotation animation itself is just updating the param on a timer. Basically line 182 plus the calls to r
75.
by DougBTX 2y ago
Probably just because it is harder to implement, the CSS would be an optimisation of a specific case on top of the general code. Classic cost (in time and overall complexity) vs performance tradeoff.
76.
by DougBTX 2y ago
Would this be a reasonable explanation? > MLPs are universal function approximators, but these models are big enough that it is better to train many small functions rather than a single unified function. MoE is a mechanism to force diffe
77.
by DougBTX 2y ago
> largest integer in the original list So if the original list was [10, 1] and negative numbers were allowed, then [10, 2, -1] would be allowed.
78.
by DougBTX 3y ago
This might interest you: https://transformer-circuits.pub/2023/monosemantic-features An embedding is essentially a set of neuron activations, the number of combinations of activations is much bigger than the number of
79.
by DougBTX 3y ago
That’s an easy one: the company exists to provide The Services, revenue from commercialising the content supports the company.
80.
by DougBTX 3y ago
Biggest fine from last year was apparently $1.2 billion.
81.
by DougBTX 3y ago
The mechanism is described in the FAQ here: https://gofetch.fail > The GoFetch attack is based on a CPU feature called data memory-dependent prefetcher (DMP), which is present in the latest Apple processors. We reverse-engin
82.
by DougBTX 3y ago
I’ll admit to not having looked at the details at all, but a possible explanation is that almost all the time is spent on inter process communication overhead, so if that also happens before the timer starts (eg, the data has been transferr
83.
by DougBTX 3y ago
Another way to look at it is that concatenating files should sum the line count. Concatenating two empty files produces an empty file, so 0 + 0 = 0. If “incomplete lines” are not counted as lines, then the maths still works out. If they cou
84.
by DougBTX 3y ago
It depends on where the timing code is. If the timer starts after all the data has already been loaded, the time recorded will be lower (even if the total time for the whole process is higher).
85.
by DougBTX 3y ago
One man’s hallucination is another’s creativity.
86.
by DougBTX 3y ago
Interesting! And "Science" is all the way down in Class 500. https://en.wikipedia.org/wiki/List_of_Dewey_Decimal_classes
87.
by DougBTX 3y ago
I’ve just tried with ChatGPT on iOS, you can press stop then immediately respond while the AI is still generating the response.
88.
by DougBTX 3y ago
No, either this is fair use, so no license is required, or the current licences must be followed (including attribution which seems like the first thing that gets forgotten).
89.
by DougBTX 3y ago
> To prove that GitHub Copilot trains on non permissive licenses, we just disable any post-generation filters and see what GPL code we can generate with minimal context This seems like a key line. I can get my browser to output GPL code
90.
by DougBTX 3y ago
The broader idea of “pass a value between functions without naming it in the caller” crops up in a few other places outside FP. In Rust there is the “builder pattern”[0] where the builder isn’t mentioned directly: ByValueBuilder::new(

More ›