axg11

31 ms·

301.
by axg11 5y ago
NN == kNN is a really interesting thought, but I think it’s the wrong framing. All learning is bound by kNN, including, I would argue, human intelligence. When we compare and contrast things, be it words, objects, or images, the very best w
302.
by axg11 5y ago
What do you mean by: large language models cannot generalize beyond the training set? That’s one of the main reasons they’re so impressive, they _are_ able to do this, to a limited extent.
303.
by axg11 5y ago
The pushback is simple: Twitter stock price was higher than Musk’s offer for most of 2021. We are in a downturn affecting the entire tech industry, and it’s likely that prices will return to previous levels at some point. Elon’s offer is a
304.
by axg11 5y ago
Elon’s views are extreme in the sense that no current and popular social media is being run with a light touch approach to moderation. I’m not sure how he plans to combat scams and bots, two issues which go against an absolutist free speech
305.
by axg11 5y ago
Current LLMs are very compute inefficient. I also think retrieval transformers can bring a few orders of magnitude in efficiency improvements. Combined with architectural improvements I think we can get there.
306.
by axg11 5y ago
We are still extremely early in this space. This is the mainframe era of generative machine learning. I'm expecting three big leaps in the next 10 years (in order): 1) Generative algorithms reach a level where the content is indistingu
307.
by axg11 5y ago
> YT and Google video search are too focused on who creates the content Correct - YouTube's feature roadmap is driven by monetization, which is in turn driven by monetizing attention through advertising. How often do you search dire
308.
by axg11 5y ago
In the new remote tech world, it’s easier than ever to interview while on the job. Obviously it depends on your meeting load and how regularly unplanned work/meetings arise. I have a suspicion this is another factor contributing to low
309.
by axg11 5y ago
This is amazing - thank you for sharing!
310.
by axg11 5y ago
This is fascinating and hints at a future possible scientific study: using phones across the globe to map cosmic ray events. I'm not a physicist so I can't speak for the value of such data. If cosmic ray events do not occur unifor
311.
by axg11 5y ago
Could be sufficient for (limited) market making. There are dozens of explanations for why Coinbase would want to hold these coins. They're not just an exchange anymore, they offer many other services.
312.
by axg11 5y ago
Very nicely explained. Also hints at the next big frontier for protein folding: improving the prediction of those disruptive effects.
313.
by axg11 5y ago
Could not agree more. We keep moving the goalposts for what constitutes intelligence, perhaps that is a net good, but I can't help feeling that we are taking incredible progress for granted. Both DALL-E and large language models (GPT-3
314.
by axg11 5y ago
You're discounting the possibility that this is _all_ that intelligence is: understanding and composition. Arguably, DALL-E has already surpassed the average person's ability to create images. Sure, there are many creatives that c
315.
by axg11 5y ago
I'm really confused about why you think this would even be a valid criticism. If I asked you to draw clothing, do you calculate cloth dynamics? No. You would immediately look for either a real life or image example, then you would draw
316.
by axg11 5y ago
Why is everyone discounting the possibility that this is a Coinbase controlled account? They would surely need to purchase the tokens before they list them.
317.
by axg11 5y ago
Thanks for your perspective. We’re still in disagreement but I wouldn’t bet on either side of the AGI debate with any significant conviction. Embeddings are very good at a few things: combining concepts (addition), untangling commonalities
318.
by axg11 5y ago
Already plugged this book elsewhere in the thread, you might be interested in "The Mind is Flat". One chapter of the book explores the concept you're describing. Our brain creates the illusion of a "full picture" wh
319.
by axg11 5y ago
Speculating here: if the brain really uses embeddings similar (in concept) to neural network embeddings, the mechanism could explain a lot of the peculiarities of the brain. Embeddings are naturally entangled, so are memories. For example,
320.
by axg11 5y ago
Compression is a component of general intelligence. A few years ago I was very sceptical of machine learning ever leading to general intelligence. I've since changed my mind. There are a lot of parallels to this work and the concept of
321.
by axg11 5y ago
Read "The Mind is Flat" - your idea is a theme of the book. We already have a few examples of adversarial information for humans: optical illusions being the most widely discussed.
322.
by axg11 5y ago
I wonder if Cruise or other AVs have a programmed threshold to "give up"? A human driver in the middle lane might decide to keep driving on and take a different route rather than hold those people up.
323.
by axg11 5y ago
Some estimates: - 500M tweets per day - 30 words/tokens per tweet - 40% of all tweets thrown away due to being duplicate/spam/bots = 9B tokens generated per day
324.
by axg11 5y ago
Everything you said is true but there is an alternative perspective: Elon founded two of the most successful companies in existence. Both companies heavily innovated on hard technological problems in order to achieve commercial success. Hen
325.
by axg11 5y ago
Elon's companies have a unique and very different culture in comparison to Twitter. Of course, Elon was not set to be CEO, but to your point, the prospect of "Elon-ification" of Twitter was going to upset a lot of employees.
326.
by axg11 5y ago
Is this type of manipulation really a result of social media? It was entirely possible before: rich investor announces purchasing of a large stake in public company, event is reported in traditional media (newspapers), rich investor is offe
327.
by axg11 5y ago
A very odd set of events and no explanation regarding Elon's change of mind. A pessimist would think that Elon has decided to go for a "hostile" takeover since he is no longer bound by the percentage holding cap. An optimisti
328.
by axg11 5y ago
At this point I'm comfortable in putting transformers as one of the top three developments in machine learning history. The way things are headed, they may turn out to be one of the most important "discoveries" ever made by h
329.
by axg11 5y ago
Great perspective that interviewing is "re-leveling" and seeking recognition elsewhere. Ultimately, good work that improves technical debt and maintenance should be recognized if you have a good manager. I think a lot of issues ar
330.
by axg11 5y ago
Do promotions not occur at your company?

More ›