tylerneylon

10 ms·

31.
by tylerneylon 3y ago
Ok, a snarky way to phrase things, but it's a good point - in general the vectors could add to zero and the angles may not align. So perhaps a better expression may be a metric for tension = [ norm(sum(piece_vectors)) + abs(angle_displ
32.
by tylerneylon 3y ago
You're speaking as if floating point representations of numbers have zero utility. One approach here could be numerical. An algorithm could work with vectors represented as floating points. Another approach, which depends on the relati
33.
by tylerneylon 3y ago
To give the answer credit, that answer does suggest adding the vectors (the same). It is also much more thorough than what I said, and I like the images. I like the answer and I was attempting to iterate. I think these two things could be i
34.
by tylerneylon 3y ago
I suspect the mathematical analysis could be even simpler. Here's one idea: View each track piece as a 2d vector. Add up the vectors. In a zero-tension setup, the sum is (0, 0). As a metric for tension, assume any mismatch in position
35.
by tylerneylon 3y ago
SEEKING WORK | SF Bay Area | Remote/Hybrid/In-office Machine learning engineer, PhD, 15 years experience I'd love to work together on a contract basis and give you the AI/LLM-based solution you want, with the understandi
36.
by tylerneylon 3y ago
The article is pointing out that, if we think of Newtonian mechanics as a set of math equations, then there exist simple systems in which the future of the system has multiple equally-valid solutions to those equations. Mathematically, it’s
37.
by tylerneylon 3y ago
I like your description because it's relatively succinct and intuitively suggests why the modified softmax can help the model handle edge cases. It's nice to ask: How could the model realistically learn to correctly handle situati
38.
by tylerneylon 3y ago
Oh, you're right about the timing. It's probably not ChatGPT.
39.
by tylerneylon 3y ago
This post, to me, is about the rise of ChatGPT — but I do think over-moderation is a huge problem. I had a hard moment on the gamedev stackexchange where I was stuck trying to learn how to do something in OpenGL. A moderator immediately clo
40.
by tylerneylon 3y ago
Both of your points are basically true, but I think a better way to model the problem is as a set of similar-length vectors being linearly combined by a probability vector. Mathematically, we can write v_out = V * w, where v_out is the vect
41.
by tylerneylon 3y ago
1. Summary The author is suggesting that we add 1 to the denominator of the softmax that is used within attention mechanisms (not the final output softmax). The softmax inside an attention unit allows it to see key/query matches as pro
42.
Sequence to Sequence (Seq2seq) and Attention(lena-voita.github.io)
3 points by tylerneylon 3y ago|0 comments
43.
by tylerneylon 3y ago
https://tylerneylon.com/ The most popular article is "Learn Lua in 15 Minutes." I write about code, math, and philosophy; eg, neural networks, permutations, or consciousness. I also run a weekly machine learning &
44.
by tylerneylon 3y ago
Yes; one way to see this is with a little calculus. It all boils down to the fact that the indefinite integral of x^(n-1) is x^n / n. (That's where 1/n comes in.) Suppose we have an n-dimensional pyramid. It has an (n-1)-dim&
45.
by tylerneylon 3y ago
A decent approximation to n! is sqrt(2pi n) * (n/e)^n. A worse approximation is that n! is about (n/e)^n. So, if b(n) is the base in b^n=n!, then b(n) is about (n/e). [I see this as the key observation of the article.] Keepin
46.
by tylerneylon 3y ago
A way to grok the "+1" in "not, then +1" is, writing in binary: 10000 - x = 1111 - x + 1 = not(x) + 1 In other words, not(x)+1 in binary as the same as subtracting x from the next higher power of two. Lukas_Skywalker&#x2
47.
by tylerneylon 4y ago
Thanks!
48.
by tylerneylon 4y ago
Would you mind providing a pointer to the the part of the OpenAI cookbook which explains how to build the matrix you multiply the embeddings by? Sorry if it was there and I missed it. Thanks!
49.
Post Mortem of a Sneaky Bug Whilst Using Python Multiprocessing(tylerneylon.com)
5 points by tylerneylon 4y ago|0 comments
50.
by tylerneylon 4y ago
I'm building software to provide better human-in-the-loop recommendations for media, content, and retail companies. I'm also keeping up with developments in machine learning, large language models, and image generation.
51.
by tylerneylon 4y ago
This is great work. I'm impressed. Just in case you happen to know the answer: How does taking tree samples (as in your third link) not harm the tree? It seems inevitable that it would do so, at least intuitively.
52.
by tylerneylon 4y ago
The use case I see the most in my career is to use LSH to help solve the "ANN" problem = approximate nearest neighbors (typically with ranked results). I've seen ANN used many times for near-duplicate detection and in recomme
53.
by tylerneylon 4y ago
R is the set of real numbers. Z is the set of integers {.., -2, -1, 0, 1, 2, ...}. Z is called "Z" due to influential German mathematicians who chose the letter to stand for "zahlen" ("number" in German). As a
54.
by tylerneylon 4y ago
Hi, I'm the author of this article. I'm grateful for the positive comments! I plan to explore FALCONN (pointed to by gajjanag) and the beautiful article of Bartosz Ciechanowski pointed out by lgvld. My original motivation for writ
55.
by tylerneylon 4y ago
Klara and the Sun by Kazuo Ishiguro is amazing. It's about one possible future of our relationship with AI. It paints multiple perspectives on this relationship while being implicitly judgmental. I'm being vague because the story
56.
by tylerneylon 4y ago
Thanks! I hadn't previously thought about the internal boost that large well-sponsored (aka corporate) teams get this way. It seems worth being aware of. I'm certainly in favor of encouraging & including researchers who work i
57.
by tylerneylon 4y ago
What would be a better method for machine learning folks to take? As a sincere curiosity / desire to learn, not meant as a rhetorical implication that I disagree. I interpret your criticism to mean that ML folks tend to re-use a test s
58.
by tylerneylon 4y ago
This readme lands to me like this: "People say deep learning killed stats, but that's not true; in fact, DL can be a huge mistake." Ok, I fully agree with their foundational premise: Start simple. But: They've overstated
59.
by tylerneylon 4y ago
No, really, it can. I'll give some examples. These are all via GPT-3, which I can access from the OpenAI api or online playground. 1. Summarizing papers: Here are some examples of paper summaries: https://twitter.com/un
60.
by tylerneylon 4y ago
I agree that the model does make mistakes. Your examples sound realistic, and I hope we make more progress to improve on models that can propagate stereotypes and similarly negative aspects that can arise in training data. I had meant to cr

More ›