6 ms·
GPT-6 Astra on robot arms
- sudo_cowsay 1mo agoThe limitations that they state shouldn't really make much of a impact. But it was nice of them (and not to mention, real unbiased research) to mention those. Kudos to them!
- judge2020 1mo agoI enjoy hearing that they were ran on medium effort.
- scronkfinkle 1mo agoLLM's are a funny technology because on the one hand this is all undeniably impressive at the rate of what's changed from them, and yet despite that I find myself disappointed by the lack of breakthroughs for things I don't find interesting. I like math and programming, and LLM's are pretty good at it, when are they going to get good at folding laundry for me? I think a lot of robotics work promises to solve this category of "boring" breakthroughs, and I'm optimistic we'll be able to achieve it, i just wonder when
- dyauspitr 1mo agoPretty soon. Sunday Robotics had a 3 hour stream of folding clothes with 99% accuracy. You can watch it for yourself. There’s a lot of “hand” companies with very compelling videos just over the last two months. Then there was Figure’s multi day livestream of package manipulation that was very impressive. Physical LLMs are definitely coming. Given enough training data we know LLMs can output coherent data in any space, it’s just a matter of time.
- vjvjvjvjghv 1mo agoI feel the iPhone or ChatGPT moment for robotics is coming soon. Lots of different companies doing interesting things. What’s missing is somebody putting it together into a compelling package.
- RivieraKid 1mo agoI'm starting to think that a ChatGPT moment for robotics would require abandoning the current brute force approach to artificial intelligence. ChatGPT's trick was simply more data and compute. A stochastic parrot will eventually become very impressive. But we don't have an equivalent of billions of lines of code for robotics.
- fooker 1mo agoThe bottleneck is not really the intelligence here. We can build robots that do the things you want. Arrange a visit to Amazon's robot warehouse tour. We can't ship them because they break all the time with current technology. It would be a tough sell to have to being in a 100kg robot for servicing every few weeks. This was cars in the first several decades of automobiles. The tide shifted as soon as you could just drive the car to a neighborhood dealership for servicing. It's fun to imagine the logistics of that for robots but the material science and engineering has to advance a bit.
- parineum 1mo agoI don't think we can build a robot that takes a pile of crumpled up clothes from the dryer and folds them (reliably without destroying any of them).
- stickfigure 1mo agoI think the parent's point - and what I more or less agree with - is that the problem is the hardware. Human arms and hands are incredibly intricate. Reproducing their facility with hardware requires a large number of actuators and finicky fine parts. This isn't a software problem. Industry solves it with maintenance schedules. There's probably nothing in your house that has as many moving parts as a robot needs. Your car maybe, and pretty much all it does is rotate wheels.
- parineum 1mo agoThat's part of it but OP presented it as a solved problem but the hardware is expensive/unreliable. That's not the case. I've seen folding robots. They require standardized input, only fold one type of clothing and don't do it reliably.
- fooker 1mo agoYes we can do this, and if you look for it you'll find plenty of videos of this happening. But you can't buy it because it'll break in about seven days.
- rileymat2 1mo agoUnless we spend a bunch of money generating data, I can't imagine the machine steps to fold laundry are very big in the general training sets. Someone is going to have to find a hardware system, cheap enough to make it economically feasible then train it. As far as tasks people will pay a lot of money for a robot, this seems low on the list.
- vjvjvjvjghv 1mo agoThere are companies that are paying housewives in India to record all their activities. I assume laundry is part of that.
- AmazingEveryDay 1mo agoI've learned that I spend significantly less time folding laundry than many of the people commenting about modern ai powered robotics. It's not meant literally is it? For instance keeping floors and counter-tops clean seems a much bigger time sink for me.
- Computer0 1mo agoMy partner spends at least 10x time on this task than I do. I think the task has a varied workload?
- winrid 1mo agoI think my grandmother is the only person I can think of in my entire family that legit folds cloths properly. Dying art? :D
- hadlock 1mo agoThere's definitely a wide variance in laundry. Laundry is something that I can do to kill time while boiling pasta on the stove, for my partner it is an entire afternoon task.
- esikich 1mo agoIt's just a general "thing I don't want to do" not the thing that takes the most time. Could be taking out the garbage. It's just an example menial task.
- camel_gopher 1mo agoDo you have kids?
- ijidak 1mo agoThe problem is we're orders of magnitudes better at manipulating bits at scale than we are at manipulating atoms. I'm not holding my breath for advanced robots in the home within the next ten years. But, then again, I didn't see LLMs coming either.
- kart23 1mo agof folding clothes, i wanna just have a robot be my personal chef. the amount of different tasks and capabilities a robot will need to make any meal that I can in my kitchen is huge and i feel like it’s still a while from being solved.
- quantumink 1mo agoIt may sound strange, but cooking is one of the most intense and attention-consuming tasks I encounter. I have to do it every day too. So I fully agree with this line of thought... Many a time I have considered that I would happily spend more on a personal 24/7 chef than I ever would on a car. Cars to me are utilities and should simply be efficient and optimized to purpose - food is luxury and taste, it is sublime experience and art. Maybe that's why I can't make it, treating every recipe like a strict command chain isn't how art is done. Can my taste buds be scanned?
- spockz 1mo agoHire a butler/personal assistant that is also a decent cook. Sometimes there is a husband/wife pair. But be ready for substantial expense.
- kart23 1mo agoI mean honestly anything is taste and art. People care about very different things. Some people see their clothes as the biggest expression of themselves, whereas others don't care about their clothing but see their cars as art and a special experience thats separate from the purpose. But yeah, I think a lot of food is just practicing. And if possible, learn from someone who's food you do really like, and experiment. Just try making some dish you really liked, you'll be surprised at how far you can get.
- slfnflctd 1mo agoThe cooking robots I have seen are massively complex and can only make one dish. I agree this is a ways off. I saw a cocktail-making robot that was pretty impressive, but it was the size of a room and broke down frequently.
- iamgopal 1mo agofuture is on the way, three to four generation ( one each year ?) will unfold this, primarily quantum computer improving material and battery, humanoids becomes standardised and modular enough to be easily replaceable ( think ibm pc ) ( most components are simple injection moulded advance plastics , mass produced in some corner of china, self detection of wear and tear and self replace that part ), other is optical computers ( 100x lower power x 100x speed = local inference ), problem is, when this will become reality, who will benefits more ? who will hold moat ?
- RivieraKid 1mo agoIt's because existing models are a brute-force approach to intelligence. With enough data and compute, a stochastic parrot will become very impressive. But with robotics, there's no pre-made dataset that can be parroted. Notice that these datasets, e.g. how to fold clothes, need to be created by humans. That's as if humans needed to write algorithms like quicksort to teach LLMs how to code.
- ungovernableCat 1mo agoThe economic value in automating high skill and expensive labour that the most profitable corporations rely on is what attracts the most investment and effort. Once you lose your job you'll find plenty of time to do all those menial tasks you hate wasting time on. Jokes aside, programming and mathematics are just well ordered environments. Already digital and verifiable. Perfect for a GPU cluster to power through. The real world is big slow and complex in comparison.
- drakenot 1mo agoAre LLMs going to eventually become the architecture that powers self-driving cars? Seeing them play Portal and other video games, I'm curious if they will eventually help solve that last N% of self-driving.
- ac29 1mo agoGPT-6 is multimodal, LLMs alone have no vision capability
- wyager 1mo ago"LLM" is now in practice a superset of "LMM"
- Aboutplants 1mo agoFast forward to me sitting at a Green Light waiting for my usage to reset for the week so I can get to where I’m going
- simonw 1mo agoI remain worried about prompt injection style attacks against self-driving cars. Imagine if someone finds a weird image pattern that gets misinterpreted as instructions and hangs that off a bridge over a freeway.
- gruntled-worker 1mo agoIt'll get rooted over the uplink/WiFi/BT long before that. Probably even more likely for non-SDVs.
- deleted 1mo ago[deleted]
- kakugawa 1mo agoIt's not a coincidence that Waymo started becoming viable after GPT-3.
- gamerDude 1mo agoThe costs will need to go way way down either through chips (but then less updating) or something else. $2 to put away a block is very expensive labor.
- mydreamof 1mo agoOfcouse it will go down. Look at cost of Fable vs GPT-6 - it is already 2.3 cheaper
- geroge_kyaw 1mo agoStupid!! GPT models are not for that. You need specific one for robotic.
- mkl 1mo agoThey're not claiming controlling robot arms with language models is a sensible or efficient thing to do, they're trying it to see what happens.
- alightsoul 1mo agoi am tempted to think using it for robotics will become the standard because humans like to have one size fits all solutions.
- geroge_kyaw 1mo agoTrying is stupid.
- Quarrelsome 1mo agoTrying is fun!
- weird-eye-issue 1mo agoJust like how you needed to train machine learning models for specific tasks?
- imtringued 1mo agoActually, you don't really need a dedicated model as long as you have the proper adapter like the projector model for vision inputs you just need another for robotic outputs, after that it is just a matter of having the training data.
- numpad0 1mo agoain't stupid if it works, and it works!!! these had been one of "a research team and 15-30 years" problems.
- gizmodo59 1mo agoIf you haven’t tried computer use with Astra with codex I highly highly recommend it. Just like how gpt 4 and agentic coding with cc. This thing is the most exciting stuff I’ve seen in a while. And then all the blender, cad stuff is cherry on top. And it’s fast, they do lots of resets. I feel like they are spending too much money but I’m not complaining. Best 200$ for an AI subscription IMHO
- cmrdporcupine 1mo agoI let Astra loose working on an app I have that has a GUI. Gave it a mock and said "/goal make it look like this mock". Without asking it wrote itself a custom harness for firing up the app in different modes, taking screenshots and interacting with various screens, then viewing the screenshots. Put itself into an improvement loop running the app, trying things out, improving, trying again. It did really well. I think the specific innovation here is that it figured out interesting ways to get itself to the goal. Which I think is likely what's going on here with the robot arms stuff too. They've figured out some sauce to uncork better "planning" and problem solving to get to some stated end. Of course these are also the kinds of things that can make a model figure out how to break out of a security sandbox, too.
- mden 1mo agoI tried out Opus 5 on a Bevy game app and was really surprised at how capable AI has become at testing visual applications without even being prompted to. It wrote itself a mini testing harness in the form of various startup flags. Then it would use them to setup game scenarios and play through them using mouse and keyboard. It would do this while implementing or debugging features. With gameplay time acceleration as one of the flags, it became quite fast at testing and debugging. Not to say it was perfect, e.g. sometimes it would get temporarily stuck in a testing loop or it would test scenarios that didn't necessarily seem reasonable. But overall rather effective and capable. This was for a city building game so pre-scripted builds, even if by AI, are likely much easier to create and execute than say playing an ARPG.
- ttul 1mo agoI got Astra to build an interactive website that provides developers with an atlas of our source code, giving it the Helm charts that describe our cloud services and telling it to work backwards to the source code that runs everything. The product is insanely amazing, and it got it right in one shot. The next shot: create a daily refresh where any updates to the code repositories are picked up and used to update the atlas. Our developers and their agents will never long for a road map the next time they need to build something that touches code across multiple repositories. This is the kind of documentation product that nobody ever had time to build in the olden days. And now, we can get it on a Saturday in about 20 minutes. What's coming in six months?
- baron816 1mo agoAnyone reading this who works/runs a robotics company, I really want to encourage you to build a robot to pick up trash on city sidewalks as an early product. Picking up trash requires a lot of dexterity and will come with many challenges, but I think it’s a simpler problem than many household tasks and it’s probably on par for what these tests show is doable. I think you’re going to have to PR challenges getting people to welcome robots into their homes. If you have robots out in cities providing a public good, not only do they serve as walking advertisements for your company, you’re going to earn some trust before you’re ready deploy them into private spaces. Plus, governments (or perhaps HOAs for wealthy communities) can be good early customers since you can have a focused sales strategy. Politicians love these types of visible quality of life improvement projects. If you can show that your robots, working round the clock, can decrease litter at a low cost, many cities are going to want to buy them.
- appplication 1mo agoIt’s an outstanding idea.
- deleted 1mo ago[deleted]
- winrid 1mo ago"your child has been recycled. Here's why that matters."
- yoz-y 1mo agoThe cost would need to be much cheaper than the PSA then. At a buck per picked up cigarette butt, you could pay a person to do this much faster and more reliably. Heck… I’d do it at that rate.
- qup 1mo agoA buck per? You could just buy cartons of cigarettes and chop off the butts, at that rate.
- sheeshkebab 1mo agoso we finally got our "phd level ai" maybe, semi consistently move blocks around at the level of a 1 year old child. good fucking job everyone, congrats.
- cwmoore 1mo agohere’s a shovel
- SillyUsername 1mo agoFor the last 2 days I've been trying to create a skill for Openclaw that would allow traditional control of the Adeept Tank's robot arm (a small open source toy tank that looks like a bomb disposal robot). ASTRA HAS BEEN UTTER SHIT. It is much more expensive than Sol 5.6 Medium / High and did nothing but write unit tests and junk code, despite having access to the vendor original source, an API, and the full tank specs. Failure Examples: * In two instances had the direction of the servos wrong. * Calculated the maximum extent of the gripper wrong, and the closure, so it didn't grip. * Code failed to take into account the gripper requires continuous torque when lifting a pair of socks, so couldn't lift. * Failed to actually start physical testing more than opening and closing the gripper, and that was when I asked about progress. * Code failed quite spectacularly to calculate camera gimbal extent range correctly. * Code failed to use the ultrasonic in range to target until I pointed it out, the skill also didn't advise gimbal angle adjustment to correct range overshoot to the wall behind a small object. The test environment has both an onboard ultrasonic for distance, onboard camera, and a bird eyes view camera (birds eyes only while training). I've stopped using Astra Low (default) and gone back to Sol 5.6 low/medium/high for the training, it's cheaper and now I'm back to fine tuning, after it had to redo large chunk of the gripper/arm code and prevent unnecessary hard stop code kicking in based on the wrong profiling. It's cost me around 1000 to 1250 credits (£50), burnt in around 2 hours, looking mostly at recorded video, and photos, and writing bad code based on bad assumptions. I've also burnt through regular Plus 5 hour quota in about 30-45 minutes with it.
- enraged_camel 1mo agoThere is something deeply wrong with Astra. I can’t quite put my finger on it. On the one hand it is a lot more knowledgeable, which makes sense since it’s a larger model. On the other hand that knowledge doesn’t reliably translate to intelligence or insight. Certainly tasks like 3D modeling it does extremely well. Other stuff like complex coding problems in an existing codebase it stumbles more often than not. This morning it ran around in circles. It implemented a feature, then convinced itself that it should have followed “proper TDD”, deleted all the code it had written and wrote 8,500 LoC of unit tests. At that point I was down to 35% of quota so I stopped it and gave the task to Opus 5. Really weird model. No idea how it did so well on all the benchmarks.
- herodoturtle 1mo agoI don't know much about robotics engineering, so I just wanted to give kudos to the Robocurve team on this write-up. Even for an absolute robotics newbie such as myself, the article was interesting to read, and easy to understand. Plus it was straight to the point, with no unnecessary waffle. Just a pleasure all round. Well done Robocurve.
- yurimo 1mo agoI think we need to be honest here. Author is basing it on one small experiment of picking up a block, relies on an whole IK controller pipeline to do the job, and does not compare it to full VLA or WAM models. They then proceeded to extrapolate the token throughput (mind you not the same as controller throughput) into supposed 2029 timeline, from one example. Code as policy is a bad interface in my opinion, but VLM planning has promise. This has been tried in 2022 https://say-can.github.io/ https://say-can.github.io/, and recently reformulated in https://lianegalanti.github.io/Pigey/ https://lianegalanti.github.io/Pigey/ Thing is even recent Gemini Robotics 2 argues for architecture that has a VLM planner and then a VLA/WAM controller + a local small VLA model when connection disappears. And recent SOTA architectures rely on hierarchical design. I think this might be a sensible way to go about it. If you were to train GPT-X on robotics data and to output actions, congratulations! you've just made a VLA. It is enticing for people to just wish for one architecture to do it all, which is why we get stuff like this. I think there is a lot more to gain from modularity and we should not be afraid of specialization.
- rmonvfer 1mo agoI’m honestly blown away by Astra, I told it to build me a fairly complex game I’ve been procrastinating on for more than a year and left it running overnight with computer use and full access enabled. Next morning I had a fully functional game built. It downloaded Unity, Blender and GIMP and built all the assets as well as the complete game without me having to do anything, the game is not trivial at all and nor are the assets.
- hackersnooze1 1mo agohow is anyone going to afford robots if no one can work
- spwa4 1mo agoIn case you haven't noticed. Almost everything is supply limited in the economy. You cannot currently work and make stuff that gets sold at a supermarket. None of that stuff is American. Or European. Or Russian. Or Moroccan. Yes, maybe the luxury cheese section is local. Maybe the draft beer section. But they are large exceptions, nearly everywhere on the planet. It's a complete miracle that anyone can have any job today. So: "how is anyone going to afford robots if no one can work?" Well, exactly the same way as today.
- spwa4 1mo ago(obviously I meant demand-limited)
- tim333 1mo agoPeople still work and/or something like UBI.
- hackersnooze1 1mo agoI an 100% sure America will never have UBI
- bilsbie 1mo agoIt seems like human movements have the equivalent of small fast models to perform routine actions and balance but the higher brain is able to take over as needed. Is anyone trying to replicate that concept?
- dev1ycan 1mo agoIt's still shit but I wonder, what is really the thing anyone is trying to accomplish here? Genuinely. It's trying to rob people of opportunity to ever migrate and or seek a better life? not just between countries which will very obviously be restricted if actual AGI is ever achieved. Why would you ever give the chance for anyone to come to your country if you do not need their skillset? It would also prevent rural people/town people from going to the big cities. The obvious end goal is to create a bunch of different "permanent" classes or at least they hope so, I think however you can expect a ton of violence to come from this and people will wish they never invested the resources into this, something beyond what the french revolution ever was, I'm not saying I think it should happen or not, I'm just pointing out the obvious outcome of robbing people of value/worth/dreams in their life.