whoateallthepy

5 ms·

1.
by whoateallthepy 2y ago
I learned Git from an O'Reilly book and I loved that it started with the internals first. The git CLI has some rough edges, but once you have concepts of work tree, index, commits and diffs down, it is extremely powerful. magit in Emac
2.
by whoateallthepy 3y ago
One point: if you are re-reviewing, other platforms (e.g. Phabricator, Gerrit) have much more developed ways to compare changes relative to one another.
3.
by whoateallthepy 3y ago
One thing to bear in mind is that these embedding vectors are high dimensional, so that it is entirely possible that the token embedding and position embedding are near-orthogonal to one another. As a result, information isn't necessar
4.
by whoateallthepy 3y ago
The tokenization is done by the tokenizer which can be thought of as just a function that maps strings to integers before the neural network. Tokenizers can be hand-specified or learned, but in either case this is typically done separatel
5.
by whoateallthepy 3y ago
This is a great set of comments/questions! To try and answer this a bit briefly: The input string is tokenized into a sequence of token indices (integers) as the first step of processing the input. For example, "Hello World&qu
6.
by whoateallthepy 4y ago
At the end of last year I put together a repository to try and show what is achieved by self-attention on a toy example: detect whether a sequence of characters contains both "a" and "b". The toy problem is useful becaus
7.
by whoateallthepy 4y ago
I put together a repository at the end of last year to walk through a basic use of a single layer Transformer: detect whether "a" and "b" are in a sequence of characters. Everything is reproducible, so hopefully helpful