DougBTX

15 ms·

151.
by DougBTX 3y ago
On the other hand, ViT uses non-overlapping patches anyway, so the impact may be minor. Example code: https://nn.labml.ai/transformers/vit/index.html
152.
by DougBTX 3y ago
There are quite a few comments here questioning why a list like this might exist, the focus should be on this in the intro: > bypass local security restrictions in misconfigured systems. Someone may think that they can give users restric
153.
by DougBTX 3y ago
A blog with ads every few paragraphs isn’t where we’ll find writers with standards.
154.
by DougBTX 3y ago
Something like this may grow out of the GGML project, which is gaining traction. They already have a weights format which can be loaded with mmap, though AFAIK the model architecture still needs to be defined in C++.
155.
by DougBTX 3y ago
Is unguided random sampling more likely to find a solution than in-order enumeration?
156.
by DougBTX 3y ago
For more on this approach, see docs on single table design principles: https://aws.amazon.com/blogs/compute/creating-a-single-table...
157.
by DougBTX 3y ago
Nice find. The docs say... -n compare according to string numerical value -u output only the first of an equal run ...so the implementation seems clear enough. With -n the comparison is numerical, with -u lines which comp
158.
by DougBTX 3y ago
Yes, though is suspect it is cultural. A team that writes slow flakey tests probably writes slow flakey production code too, for the same reasons.
159.
by DougBTX 3y ago
> A model is a ... view on data. I think this is part of the reason the MVC names persist, they are so generic that you can describe the Model as a type of View!
160.
by DougBTX 3y ago
Maybe they’re looking for contrast in a particular scene, eg bright beside dark. If a sequence as a whole is very loud, while a later sequence is quiet, they might have to drop the volume down because of the loud part, effectively compressi
161.
by DougBTX 3y ago
> The upshot for me is that this feature adds RAII to JavaScript That doesn’t appear to be the case, a resource can be returned, but this is block scoped. It appears to be closer to a using statement in C#.
162.
by DougBTX 3y ago
In this case, it looks like the private company which did the work is also getting away with it... https://www.caracal.agency/en/projects/discover-eu
163.
by DougBTX 3y ago
No, rotate by 90 degrees. The plane is normal to the line, so the line doesn’t lie in the plane.
164.
by DougBTX 3y ago
This is the best implementation I've seen, but only for Hugging Face models: https://github.com/1rgs/jsonformer
165.
by DougBTX 3y ago
At least the argument for road safety has moved on from “we can’t make roads safe” to “we don’t want to spend our money making roads safe”.
166.
by DougBTX 3y ago
An astonishing number of debates are between one group of people saying “Foo needs A” and another saying “Foo needs B” when really... Foo needs a good helping of A and B.
167.
by DougBTX 3y ago
For Rust, pre-GAT, there was no way to “output” a type which could be “called” with further arguments (very fuzzy terminology, sorry, best I can do!) or maybe in other words, you could write functions which retuned values, but not new funct
168.
by DougBTX 3y ago
That’s a more realistic option than you’d think. What is the point of a software Q&A site anyway, why not just read the docs? In my experience, Q&A is useful because it summarises and consolidates disparate information into a concis
169.
by DougBTX 3y ago
Do you strictly need that level of tokenisation precision to meet your high-level goals?
170.
by DougBTX 3y ago
Can’t get much more tech CEO than Sam Bankman-Fried
171.
by DougBTX 3y ago
In the insight case, the model has to classify the terms in a short paragraph into broad generic classes, then report which terms have different classes. In the recall case the model has to rank all utterances in its training data based on
172.
by DougBTX 3y ago
In a prompt, can you just tell the model which letters make up each token? Eg a list of ag = a g etc. I imagine a dictionary of that for all tokens in the training data would help.
173.
by DougBTX 3y ago
Yes, but it is normal for sayings like that to have contradictory versions, so that one is available in any situation. For example: "Too many cooks spoil the broth" vs. "Many hands make light work" "Look before you
174.
The Merge (2017)(blog.samaltman.com)
1 points by DougBTX 3y ago|0 comments
175.
by DougBTX 3y ago
The difficulty with PARITY for Transformers seems to be that it relies heavily on the position encoding. The longer the input sequence, the more likely it is to read the wrong bit, then all the subsequent output is wrong. Instruction tuned
176.
by DougBTX 3y ago
> I used this prompt to get it to compute semi-reliably for 16+ digit sequences That’s what I saw too, errors coming in around item 16 to 30, depending on small changes in the requested output. It looked like it was reading the wrong ite
177.
by DougBTX 3y ago
Interesting, this is a harder problem than I'd guess. Prompts like this fail around line 16 with ChatGPT using GPT-3.5: Show the full output of this Python script in a single code block, with no other commentary: ``` lst
178.
by DougBTX 3y ago
At least for short sequences it works, but I guess the trick is to make it work for long sequences! Prompt and output from GPT-4: Write a Python function to compute PARITY incrementally on a list of 0 and 1. For example, the output for [0,
179.
by DougBTX 3y ago
Take a look at the linked paper, they train using batches with about 5% of ImageNet in them (batch size = 2^16), which is pretty large from my perspective. Overall they show diminishing returns for larger batch sizes after a certain point.
180.
by DougBTX 4y ago
Batch size can be used for regularisation, but using it for that will limit training performance. From the Google Research Tuning Playbook: > The batch size governs the training speed and shouldn't be used to directly tune the valid

More ›