benjismith

8 ms·

31.
by benjismith 3y ago
I find myself wondering if Apple applied some kind of back-channel pressure to oust Riccitiello. With Unity at such a privileged position in the developer ecosystem of the upcoming Apple Vision Pro, I can imagine that Apple execs were pisse
32.
by benjismith 3y ago
I'm a software engineer, and I think the fully remote-work culture can often be less personally fulfilling. I enjoy going to an office, having a change of scenery, interacting with people (both close friends and casual acquaintances),
33.
by benjismith 3y ago
I'm still trying to get a handle on that part myself... But my ever-evolving understanding goes something like this: The "Query" matrix is like a mask that is capable of selecting certain kinds of features from the context, w
34.
by benjismith 3y ago
Okay, here's my attempt! First, we take a sequence of words and represent it as a grid of numbers: each column of the grid is a separate word, and each row of the grid is a measurement of some property of that word. Words with similar
35.
by benjismith 4y ago
No, I don't pay for it myself. My employer setup the WeWork and pays the monthly bill, on the premise that there would be a handful of people in our city (Portland, OR) who also want to work from an office. There have a been a handful
36.
by benjismith 4y ago
I'm 45.
37.
by benjismith 4y ago
I must be one of the very very few software engineers that would prefer to RTO. (I'm not at Amazon, btw). I do enjoy the flexibility of being able to occasionally work from home. But I miss the bustling office culture, going out to lun
38.
by benjismith 4y ago
Okay, somebody posted a thread on Twitter explaining how this works... The language model is capable of generating python scripts to solve certain text-processing tasks, and then it re-prompts itself by reading the python outputs back into
39.
by benjismith 4y ago
Lol, good point! I just meant "this isn't related to Thinking Fast and Slow. It's just the tokenizer". But yeah, the inner workings of the language model are so complicated as to be almost completely incomprehensible, ev
40.
by benjismith 4y ago
Good question! I don't know :)
41.
by benjismith 4y ago
It's really not so complicated. This is just an issue with text tokenization, and the fact that the learning model never actually sees the raw input bytes. All modern LLMs use a tokenizer to convert a sequence of bytes into a sequence
42.
by benjismith 4y ago
This is awesome! I just finished watching the Unit 10 video ("Neural Networks I") and it filled in quite a few gaps in my understanding. I really love that you build a complete working example, all the way down to the matrix multi
43.
by benjismith 4y ago
The central thesis of his argument, that large language models are capable only of large-scale mimicry, is flat-out wrong. > I think that people are led to believe that this system actually understands human language, which it certainly
44.
by benjismith 4y ago
I liked the article, and I'm sympathetic to the overall point... > The kind of hoops I have to jump through to get types "just right" in a web app versus a library is dramatically different. It's rare in a web app for
45.
by benjismith 4y ago
Sounds great! Very interested when the SaaS offering opens up. Definitely not keen on running a Kubernetes cluster for the sake of simplifying ML operations.
46.
by benjismith 5y ago
The "prompting" ability you're referring to is called "instruction following", and here are some descriptions of it. https://openai.com/blog/instruction-following/ I think the differences
47.
by benjismith 5y ago
I'm super excited about this! I'm on the cusp of releasing a model into production that was fine-tuned upon your 6B model, and the results are quite excellent. I'd be very curious to try out the 20B model the next time we ret
48.
by benjismith 5y ago
As the original author of this post, I'm always amused to see it coming back again and again on Hacker News. A lot has changed since 2005, but a lot of things have stayed the same. What a weird craft we practice... Anyhow, thanks for a
49.
by benjismith 5y ago
I've been following your work for years (and I'm actually neck-deep in a ShareDB project right now) so I just want to say thank you for all of your contributions! I especially enjoyed this post.
50.
by benjismith 5y ago
Very nice!
51.
by benjismith 6y ago
Hi Alex, I just want you to know, I really admire what you're doing, and I hope you're wildly successful at it!
52.
by benjismith 6y ago
Right now the "similar books" thing is based on a "topic-model"... So books are more likely to be similar if they're roughly in the same genre and discuss similar kinds of topics (dragons, computers, romance, spies,
53.
by benjismith 6y ago
I respect your skepticism :) It's easy to imagine exceptions to the idea of a simple numerical word-scoring algorithm... Of course, a word like "bad" might be used ironically, or in some other slang-sense, with a different li
54.
by benjismith 6y ago
Oops, I almost forgot... The one viz component I'm using is the excellent WordCloud2 library by Timothy Guan-tin Chien... https://timdream.org/portfolio/wordcloud/
55.
by benjismith 6y ago
Awww thank you! I really appreciate it! I'm not using any visualization libraries. It's all just hand-coded javascript... I've been meaning to learn D3 for a long time, but I haven't gotten around to it yet.
56.
by benjismith 6y ago
YES! These are pretty much exactly the methods I used when I developed my project http://prosecraft.io You can see the emotional story arc -- the shapes of the stories -- for more than 16,000 books. I train a Word2Vec model on t
57.
by benjismith 6y ago
Very interesting. It's pretty good with song structure, formatting, themes, etc... The one major thing it seems to be missing is rhythm and rhyme. I can imagine decorating the data with a pronunciation dictionary, so that you can coerc
58.
by benjismith 6y ago
This is delightfully clever. I'm the developer behind http://prosecraft.io so I always enjoy learning about new linguistic toys :) Thank you!
59.
by benjismith 7y ago
Looks very interesting. Though I would love to see a few sample simulations to get a better idea of what's possible...
60.
by benjismith 8y ago
This makes me so happy :)

More ›