7 ms·
Q-Transformer
- amelius 3y ago33% success rate for opening a drawer?
- varjag 3y agoThat's a great start. Opening a drawer is pretty hard problem.
- methodical 3y agoCorrect- as a reference for OC this is known as Moravec's Paradox (https://en.wikipedia.org/wiki/Moravec%27s_paradox https://en.wikipedia.org/wiki/Moravec%27s_paradox)
- varjag 3y agoIndeed: we have machines doing novel work in organic synthesis but none yet to empty a dishwasher.
- amelius 3y agoThis already exists: https://www.youtube.com/watch?v=UAG_FBZJVJ8 https://www.youtube.com/watch?v=UAG_FBZJVJ8
- pixl97 3y agoAt least to me the key point of the article >Encoded in the large, highly evolved sensory and motor portions of the human brain is a billion years of experience about the nature of the world and how to survive in it. The deliberate process we call reasoning is, I believe, the thinnest veneer of human thought, effective only because it is supported by this much older and much more powerful, though usually unconscious, sensorimotor knowledge. We are all prodigious olympians in perceptual and motor areas, so good that we make the difficult look easy. Abstract thought, though, is a new trick, perhaps less than 100 thousand years old. We have not yet mastered it. It is not all that intrinsically difficult; it just seems so when we do it.
- amelius 3y agoI'm not convinced. There's a lot of creative thinking going on in evolutionary anthropology, but the proofs are meager.
- polygamous_bat 3y agoThe other robot project posted here yesterday opened drawers at around 80% though https://news.ycombinator.com/item?id=38453047 https://news.ycombinator.com/item?id=38453047 And they did it in many different homes.
- deleted 3y ago[deleted]
- sfink 3y agoMy robot can open drawers at near 100%, but it takes the form of a stick of dynamite. Still working on closing the drawers afterwards, though...
- coolspot 3y agoPerhaps you could automate rebuilding the drawers with some IKEA/TaskRabbit APIs.
- amelius 3y agoThey should probably optimize for minimal friction force. If you're pulling the handle not entirely in the direction of movement, you're not doing it right.
- luke-stanley 3y agoThis came out a while ago, but I can see why it's been reposted. It's another approach to searching the solution space with rewards and transformer language models. Like XoT and others as I pointed out in my last comment: https://news.ycombinator.com/item?id=38471388 https://news.ycombinator.com/item?id=38471388
- dartos 3y agoIt’s probably being reposted bc people think it has to do with Q*
- thehucklecat 3y agoI really wish I could understand this. I can half way get it loaded into my brain and then...
- Philpax 3y agoWorth noting that this came from DeepMind in Sep-Oct of this year, so it's not related to OpenAI's "unfortunate" leak [0] of Q*. [0] https://www.theverge.com/2023/11/29/23982046/sam-altman-interview-openai-ceo-rehired https://www.theverge.com/2023/11/29/23982046/sam-altman-inte...
- jansan 3y agoAlso, it is not Amazon Q, which was presented yesterday: https://aws.amazon.com/de/q/ https://aws.amazon.com/de/q/
- 3cats-in-a-coat 3y agoAlso it's not Q from Star Trek, or Q stars which are an exotic form of matter immediately preceding the singularity of a black hole.
- GTP 3y agoAlso, it's not Android Q.
- kristjansson 3y agoOr James Bond’s Q.
- behnamoh 3y agoor Q.
- deleted 3y ago[deleted]
- px43 3y agoI wouldn't go that far. Immediately after Q* leaked, Jimmy Apples pointed to this thread on Twitter : https://twitter.com/polynoamial/status/1676971508911198209 https://twitter.com/polynoamial/status/1676971508911198209 It's Noam Brown's (former Deep Mind) "I'm joining OpenAI" thread from Back in June, and he talks about how he wants to bring some of the work from AlphaGoZero etc into OpenAI's AGI architecture. It looks like this Q-transformer work shares a lot of the same lineage of what Q* is supposed to be. This is Google's attempt at mushing Q-learning into transformer land, which is probably what Q* is as well. It's not the same thing, for sure, but they are at least sibling bodies of work.
- spacebacon 3y agoOperation reclaim the letter Q within a reasonable objective reality.
- VikingCoder 3y agoWhat's the letter? What's the letter? What's the letter? The letter of the day is... Q!
- deleted 3y ago[deleted]
- wiz21c 3y agoJames Bond secret weapon ?
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- maurits 3y agoBeing implemented as we speak, by the always impressive LucidRains [1] [1]: https://github.com/lucidrains/q-transformer https://github.com/lucidrains/q-transformer
- GaggiX 3y agoIt's kinda funny how Lucidrains came back to make numerous commits to this repo following Q*.
- itissid 3y agoApparently this guy like a bunch of others like https://github.com/ggerganov/ggml https://github.com/ggerganov/ggml are implementing transformers from papers for people that want them. Pretty cool.
- milanove 3y agoI’m glad to see people in the open source community taking the initiative to implement these tools for the general public. It’s one thing to understand the high-level concepts in a paper, but another entirely to implement them efficiently. Some people just really have a knack for reading academic papers and implementing those ideas in software.
- pama 3y agoThe additional link is indeed very useful, however, with one exception, it is limited to inference, ie code for evaluating a model using somebody else’s trained model weights. The original link has code for training (which can also be used for inference, of course.) Inference is super important, of course, so it makes a lot of sense to put effort to make it work well on all kinds of hardware; the papers typically focus on training and only then evaluating/validating a new model.
- rtev 3y agoLucidrains also created the much-missed EpicMafia, which still doesn’t have a good replacement after shutting down. Exceptionally skilled person!
- brcmthrowaway 3y agoIt's clear we're searching for the god algorithm of AI, just like physicists are searching for theory of everything. Are transformers the answer though?
- mirekrusin 3y agoIf they were we'd stop searching already, no?
- goldenkey 3y agoThere's really no God algorithm needed, just something good enough to assist with research of the next tier of hardware, energy, and code for AI.
- Workaccount2 3y agoIt seems more likely that there will be multiple avenues to AGI, all with their strengths and weaknesses. But perhaps the "God AI" will be a multifaceted model composed of many different models acting in unison.
- falcor84 3y agoExactly, just as the true god has seven aspects [0]. [0] https://en.wikipedia.org/wiki/Themes_in_A_Song_of_Ice_and_Fire#Faith_of_the_Seven https://en.wikipedia.org/wiki/Themes_in_A_Song_of_Ice_and_Fi...
- bee_rider 3y agoOn the other hand, most of the original gods were parts of polytheistic pantheons. Maybe a bunch of models that represent identities and biases could be more useful, they could argue amongst themselves, presenting a more full point of view, users could become familiar with the particular perspectives.
- kstrauser 3y agoI could see something like the "modularity of mind" model of human consciousness, where multiple approaches are working on "subconscious" solutions to a given problem in parallel, with a top layer deciding which is appropriate at the moment. My human brain doesn't use the same algorithm for learning to play a song on a piano as learning to play a new board game. I'm not an AI person, but it seems reasonable to imagine we'd have different "modules" to apply as needed. AlphaGo probably sucks at conversation. ChatGPT can't play Go. The part of my brain writing this couldn't throw a baseball. The physics engine that lets me throw a baseball couldn't write this. Is there a reason we'd want or need one specific AI approach to be universally applicable?
- nihaan 3y ago[flagged]
- deleted 3y ago[deleted]
- itissid 3y agoCould one of the roomba companies now make those things recognize the objects and avoid them, instead of dead reckoning or RF based obstacle avoidance?
- itissid 3y agoJust to be clear the "scalable" part here is assuming the dataset of human demonstrations is available it can in a closed loop learn better than before? In other words, am I right in understanding that the mapping from Abstract actions to concrete steps in the State space is provided at a low "enough" cost?
- itissid 3y agoLike in Workflow Guided explorations like: https://arxiv.org/pdf/1802.08802.pdf https://arxiv.org/pdf/1802.08802.pdf, for example, if you said "Forward the email" the Bot needs to sample from a set of actions which in this case would be DOM elements and the JS API to call to click the right button to forward the email. This would be a harder/interesting to test out. Can it learn to forward an email on firefox desktop browser and then do it on an iphone? It feels like opening a drawer once learn't, the bot has a (world?) model of what all drawers might look like, so it can open different drawers in any world. But this might not generalize to web interfaces and more specifically, how to do those actions on those interfaces? Not to take away from what this paper's scope and achievements.
- 33a 3y agoHow does this stack up to diffusion policy learning? https://diffusion-policy.cs.columbia.edu/ https://diffusion-policy.cs.columbia.edu/ Looking at the figures and videos it seems... worse? Sort of surprised they didn't compare it but I guess they're trying to limit the discussion to purely reinforcement learning methods. EDIT: Ah, I see this is an older result and was published concurrently to the diffusion policy paper, so it's likely the authors didn't know about it in time to add the extra comparisons.
- nabakin 3y agoTo save you guys some time, this is a 2 month old paper and not made by OpenAI. It could be related to Q* (I don't know enough to determine that) but it's not OpenAI's Q*.
- deleted 3y ago[deleted]
- osti 3y agoIs Q the new hottest letter right now?
- smokel 3y agoTo make this strand of discussion slightly more interesting, I hereby present you a page [1] with the history of Q-tips. Apparently, the "Q" stands for Quality :) [1] https://www.qtips.com/about/ https://www.qtips.com/about/
- nothrowaways 3y agoQ is nothing without the *
- GTP 3y agoYou're the Q to my *!
- nothrowaways 3y agoI love q so much.
- Filligree 3y agoThis does seem pretty similar to Q-learning. So yes, things-based-on-Qs are hot right now.
- deleted 3y ago[deleted]
- rvz 3y agoHow flies get onboard with the latest trend these days. First it was the LK-99 hype, now it is Q. Chasing hype like flies to a light bulb.