andreyk

8 ms·

31.
by andreyk 3y ago
People dunk on Meta for overhyping the metaverse and spending absurd amounts of money on it, but I don't think they get enough credit for the amount of useful R&D that absurd amount of money has enabled. Sure, from a business persp
32.
by andreyk 3y ago
This covers three things: Llama.cpp (Mac/Windows/Linux), Ollama (Mac), MLC LLM (iOS/Android) Which is not really comprehensive... If you have a linux machine with GPUs, i'd just use hugging face inference ( h
33.
by andreyk 3y ago
afaik the term MoE is generally used in the context of neural networks, with the term mixture models being more general. But you are right it's not exclusive to NNs.
34.
by andreyk 3y ago
Good point - it's a loose analogy training wise, and not applicable to inference time.
35.
by andreyk 3y ago
MoE is a type of neural network architecture, not an approach to lifelong learning. As they say " We propose a new Shared Knowledge Lifelong Learning (SKILL) challenge, which deploys a decentralized population of LL agents that each se
36.
by andreyk 3y ago
I work at a company using LLMs, and so far Falcon has been the only commercially licensed LLM that was remotely close to ChatGPT in terms of performing well on our use case (MPT did not work well when we tried it). And a ton of the open sou
37.
by andreyk 3y ago
"But all of these things [reasons why people don't care about your blog] are not a problem, because you shouldn’t care even a little bit about what other think." I mean... Maybe you should care sometimes? Maybe it is good to
38.
What Do LLMs Know About Linguistics? It Depends on How You Ask(thegradient.pub)
2 points by andreyk 3y ago|0 comments
39.
by andreyk 3y ago
weird title, note that the tweet said "so yes, GPT4 is *technically* 10x the size of GPT3, and all the small circle big circle memes from January were actually... in the ballpark?" It's really 8 models that are 220B, which is
40.
Crypto Firms Start Looking Abroad as U.S. Cracks Down(nytimes.com)
2 points by andreyk 3y ago|1 comments
41.
by andreyk 3y ago
yeah, by controller I was thinking VR controllers like those of oculus, which afaik are not generally compatible with other products. The battery doesn't seem too bad, but having used VR with wires I think it still wont feel the same a
42.
by andreyk 3y ago
"For weight purposes, an external battery will sit in your pocket and connect to the Vision Pro through a cable." Interesting choice... can't say I am a fan "The headset's visionOS operating system can be controlled
43.
by andreyk 3y ago
"But I feel like something that gets lost in the discussion of AI-generated artwork is the turds." You know, we already have a word for this... "flaws". Does the notion that AI-generated artwork has flaws really get lost
44.
by andreyk 3y ago
Lots of lead up to it, but the punchline is: "My current mental model of ChatGPT is that it’s akin to a “Maximum Likelihood Estimator for the Entirety of Human Knowledge”. There are two very different ways to interpret that: (1) Meh, i
45.
by andreyk 3y ago
The actual title "Why the Original Transformer Figure Is Wrong, and Some Other Interesting Historical Tidbits About LLMs" is way more representative of what this post is about... As to the figure being wrong, it's kind of a n
46.
by andreyk 3y ago
Just a quick clarification, attention of the same sort transformers used was already being employed in RNNs for a while. Thus the name "attention is all you need", it turned out you can just remove the recurrent part which makes i
47.
by andreyk 3y ago
This article is great insofar as it describes how to write an article that appeals to HN. But having been writing and submitting to HN for like 8 years with maybe a dozen front page entries, I think the hack is simpler: pick a topic people
48.
by andreyk 3y ago
FYI, this is MusicLM, which was just recently published and appears to be way ahead of anything out there so far. You can check out examples here https://google-research.github.io/seanet/musiclm/examples/ So
49.
by andreyk 3y ago
Curious why you think this? PaLM2 looks great, and Google has been productizing cutting edge AI pretty fast for years.
50.
Rights and Regulation: The Future of Generative AI Under the First Amendment(skynettoday.com)
1 points by andreyk 3y ago|0 comments
51.
In-Context Learning, in Context(thegradient.pub)
2 points by andreyk 3y ago|0 comments
52.
Why I'm no longer writing stories with AI(storiesby.ai)
74 points by andreyk 3y ago|104 comments
53.
Ask HN: Have you fine-tuned LLMs to know the contents of a specific code base?
7 points by andreyk 3y ago|1 comments
54.
by andreyk 4y ago
Most if not all the research papers from EDR was co-authored with people from Google robotics, so it's not all that weird. I think at least some of the people from EDR wound up at Google robotics anyway.
55.
by andreyk 4y ago
From what I hear Levine is a beast, super hard worker with incredible memory and thinking speed. So I would say he most likely is aware of all the stuff his name is on, at least at a high level. Especially this sort of big project.
56.
by andreyk 4y ago
I've definitely found this to be the case with writing. ChatGPT gets you a "valid" output, but usually a bland and lame one. I think it's better for editing and brainstorming, or for simple tweaks. Specialized tools like
57.
by andreyk 4y ago
Almost nothing is quite an exaggeration - we know a whole lot about GPT3 (their paper was quite detailed), and even if OpenAI made some tweaks beyond RLHF the underlying model and training objective are most likely the same.
58.
by andreyk 4y ago
TLDR: we don't know the data some of these models (particularly GPT4) is trained in, and labor practices related to their moderation seem shady. Pretty misleading title, this is more of an overview of bad things to be aware of wrt larg
59.
by andreyk 4y ago
World events and economy aside, there are also clear trends with teenagers (and people as a whole) having fewer close friends than ever - loneliness is more common than ever. Our culture and society as a whole is declining in its ability to
60.
Don't “Plan for AGI” Yet(lastweekin.ai)
2 points by andreyk 4y ago|0 comments

More ›