anonymoushn

18 ms·

91.
by anonymoushn 1y ago
How does SentencePiece choose the initial vocabulary, which is trimmed down to determine the final vocabulary which has these desirable properties?
92.
by anonymoushn 1y ago
If labs provided the corpus and source code for training their tokenizers, it would be a lot easier to produce results about tokenizers. As it is, they provide neither, so it is impossible to compare different algorithms running on the same
93.
by anonymoushn 1y ago
providing a large list of bitrotted URLs and titles of books which the user should OCR themselves before attempting to reproduce the model doesn't seem very useful.
94.
by anonymoushn 1y ago
I evaluated Windsurf at a friend's recommendation around half a year ago and found that it could not produce any useful behaviors on files above a thousand lines or so. I understand this is mostly a property of the model, but certainly
95.
by anonymoushn 1y ago
You can't have monomorphization or the old async feature if you need to be able to convert individual files to C.
96.
by anonymoushn 2y ago
The behavior of "let" with for loops where the variable is declared more times than it is initialized, despite the source code having one declaration that is also the only initialization, is not very explicit.
97.
by anonymoushn 2y ago
generic simd abstractions are of quite limited use. I'm not sure what's objectionable about the thing Rust has shipped (in nightly) for this, which is more or less the same as the stuff Zig has shipped for this (in a pre-1.0 compi
98.
by anonymoushn 2y ago
these are facts about the C and C++ stdlib sort functions which nobody should really use.
99.
by anonymoushn 2y ago
there are no loads in the above unsafe block, in practice loadu is just as fast as load, and even if you manually use the aligned load or store, you get a crash. it's silly to say that crashes are unsafe.
100.
by anonymoushn 2y ago
This sort of measure is a decent match for BPB though. BPB=-log(document_probability)/document_length_bytes and perplexity=e^(BPB*document_length_bytes/document_length_tokens). We already train models by minimizing perplexity, and
101.
by anonymoushn 2y ago
fractions is a great answer!
102.
by anonymoushn 2y ago
Yes.
103.
by anonymoushn 2y ago
I always ask in these threads, what sort of data do you have that cannot be divided into several integers, such that sorting by those integers according to some priority implements your comparator correctly, and also your comparator is not
104.
by anonymoushn 2y ago
It sounds like the C++ is not well-optimized then?
105.
by anonymoushn 2y ago
I've been using Zig for nearly 4 years now. A lot of changes in that period were not great, but I haven't really wanted to use anything else.
106.
by anonymoushn 2y ago
When people go to Nextdoor to drum up opposition to a development on the basis that they have enjoyed driving by that disused refinery for a decade, maybe we shouldn't care whether they think "anything other than a disused refiner
107.
by anonymoushn 2y ago
Why do we want people to buy houses? It incurs transaction costs and makes it difficult for people to relocate to be near their jobs and such. It seems like we should optimize solely for the cost of meeting people's needs - either by p
108.
by anonymoushn 2y ago
I have tried to explain the sense of "good investment" that is meant by policymakers and voters and the cliche above. You have tried to explain that there is another sensible definition available. I don't disagree, but I thin
109.
by anonymoushn 2y ago
"Housing is a good investment" means the price of the land and house increases over time compared to SPX or treasuries or whatever.
110.
by anonymoushn 2y ago
Even better, if you click through to the linked source he doesn't say "low-level" at all, or make any claim that is at all like the claim he is cited as making!
111.
by anonymoushn 2y ago
You can use https://github.com/simd-everywhere/simde if you like. In general portable SIMD libraries are of limited utility because having different primitives available on different architectures often means that you
112.
by anonymoushn 2y ago
I suspect that it is not worth using AVX2 vector gathers on any CPU. But certainly you could end up with the best implementation varying between microarchitectures for other reasons.
113.
by anonymoushn 2y ago
I'm primarily writing "general-purpose" code (especially parsers and formatters) rather than code that does the same math operation on a big array, so it's usually not reasonable to even use the same approach to the prob
114.
by anonymoushn 2y ago
I have no idea how to get the compiler to generate wider-than-16 pshufb in the general case, for example, and for the 16-wide case, writing the actual definition of pshufb prevents you from getting pshufb while writing a version with UB get
115.
by anonymoushn 2y ago
Zig ships LLVM's internal generic SIMD stuff, which is fairly common for newish systems languages. If you want dynamic shuffles or even moderately exotic things like maddubs or aesenc then you need to use LLVM intrinsics for specific i
116.
by anonymoushn 2y ago
I don't find it too surprising that people who are smarter make more money.
117.
by anonymoushn 2y ago
the old commits are still in your reflog, and if they belong to a branch that isn't only on your machine, you can nag a coworker if you don't know how to use that
118.
by anonymoushn 2y ago
It ends up being a worthwhile thing to do if someone has checked in a large file that shouldn't be in git or whatever
119.
by anonymoushn 2y ago
You never add files, create commits, or rebase? Often people arrive on a team that has committed a grave sin in the past and all future team members are forced to use workflows involving submodules as penance.
120.
by anonymoushn 2y ago
No, I don't think it is a big deal for compilation speed.

More ›