21 ms·

Getting into robotics as a software engineer

179 points by bouk 3y ago

ih 3y ago
I'm just getting into robotics with a background in software and I ended up choosing the Isaac Sim platform/ecosystem (https://docs.omniverse.nvidia.com/isaacsim/latest/index.html https://docs.omniverse.nvidia.com/isaacsim/latest/index.html) and Jetbot (https://jetbot.org/master/ https://jetbot.org/master/) since they seem suited for neural networks and reinforcement learning. So far getting up and running with the simulation side hasn't been too bad and there is a model for the Jetbot in Isaac Sim since it's also by Nvidia. I haven't started on the hardware side so can't speak on that. The downside for some might be it's proprietary and Jetbot itself is a bit out of date and nearing end of life support.
- m00x 3y ago
  You'll be better served with a more capable platform. You can either 3d print it, or you can pick up something like the WaveRover from Waveshare. It's a solid metal frame with geared motors and differential drive. For ~$100 you won't get much better. You just need to pop in 3x18650 batteries in the chassis and connect a Jetson nano to the UART pins. It has an ESP32 inside, so you can also program that, but it comes programmed out of the box. You can control it with this python library https://github.com/msanterre/wave_rover_serial https://github.com/msanterre/wave_rover_serial Isaac SIM is a terrific simulator, you won't get much better.
  - ih 3y ago
    Cool, thanks for the advice!
Animats 3y ago
This might be a good time to get into robotics. For a long time, industrial robots were really dumb. Then there was a round of false enthusiasm for "intelligent robots" (Rethink Robotics went down that rathole.) Now, at last, compute power, vision, and AI are cheap enough that you can get something done. The components are much better. Batteries are better. Motors are better. Radio communications work. Cameras are cheap. Short-range LIDAR is affordable. Navigation systems work. Robotics work used to require using a lot of time building custom solutions for those problems. Now you can just order components. Here's an idea I'd like to see revisited. Back in the 1980s, someone built a pair of small forklifts that operated as a team. These were little things, about half a meter cubed, with maybe 50cm of lift. Individually, they couldn't do much. But a pair working together could pick up and move a couch, with one robot lifting each end. In the 1980s, the researchers had trouble coordinating two mobile robots. Communications alone were a big problem. Not today. There are many material handling tasks where one small robot isn't enough, and a big machine the size of a forklift is too bulky. But teams of small robots might work. I still have a small robot arm on my desk, but it's not connected to anything.
eschneider 3y ago
Nice article. As someone who's also gotten into robotics from the software side, I'd also suggest learning a bit about the hardware side. If you're doing anything with custom boards and board bring-up, you'll need to be able to read a schematic and data sheet at a basic level. Being able to use an oscilloscope and logic analyzer is also _very_ useful. None of this stuff is terribly hard, but it's nice to know what to expect.
- Animats 3y ago
  Yes. The number of people who both know C++ (not C) and how to use an oscilloscope is surprisingly small. Electronics at robotic speeds isn't that hard. Most signals are audio bandwidth or lower. You rarely need all the elaborate design techniques needed when you get into the MHz-Ghz range.
  - FirmwareBurner 3y ago
    >The number of people who both know C++ (not C) and how to use an oscilloscope is surprisingly small. Statements like these make my eyes roll. The difficult part in embedded, and what makes the difference between a n00b spending 2 months finding a hardware/firmware issue and a graybeard finding it in under 2 days, is not how to use an oscilloscope or how to write "clean" c++, it's knowing where exactly to put the oscilloscope probes and what exactly to look for on the oscilloscope screen, and how to make the bug/edge case reproducible. You don't have unlimited scope probes, trace memory and time to probe every single signal on the board and stare at blank signals not knowing what to look for, for weeks, while th clock is ticking. Tinkering with STM32s and Raspberry Pis at home for fun, doesn't prepare you for the issues you'll encounter in debugging production devices (especially battery powered or RF) you'll need to ship on time and on budget. Knowing how to use a scope and how to write c++ is only 10% of a successful product. The other 90% is blood, sweat and tears hunched over for hours/days over boards and breakpoints to find out your race condition comes from a cheap crystal oscillator operating out of spec and not from your software. Especially that at small companies and start-ups you don't have the budgets and HW lab equipment that the likes of Apple or Qualcomm can afford to brute force your way to the solution, so you need to be very shrewd and clever with your debugging to make the most out of your limited resources.
blah-yeah 3y ago
Make magazine has a great magazine on getting into robotics via the Battle Bots type of robots. I think it was an issue from 2022 or 2023 (not enough energy currently to track down the issue in their catalog) Here's an article on their site about the topic: https://makezine.com/tag/battlebots/ https://makezine.com/tag/battlebots/
- ashconnor 3y ago
  Reminds me of the UK TV show Robot Wars: https://en.wikipedia.org/wiki/Robot_Wars_(TV_series) https://en.wikipedia.org/wiki/Robot_Wars_(TV_series)
bella014 3y ago
[dead]
chris_st 3y ago
I've occasionally thought it would be fun (not necessarily productive!) to do something like the experiments folks have done with Genetic Algorithms, having simulated robots learn to walk, etc. in a simulated physics environment. The interesting bit would be to do it with real legs, sensors, servos, etc., to try to build up a "naive physics" library that can deal with balance, etc., by learning from physical experience rather than starting with equations of motion. That may be the most number of times I've typed "etc." in a paragraph.
- coderenegade 3y ago
  System Identification is a fairly common strategy for developing a controller. Essentially, you learn the dynamics model from recorded data, and then use that as a simulator to iteratively improve the controller. The devil is in the details though, as you can run into issues with simulation fidelity (due to missing data), which can produce something that doesn't translate to real life performance.
- imtringued 3y ago
  https://m.youtube.com/watch?v=17NrtKHdPDw https://m.youtube.com/watch?v=17NrtKHdPDw
  - mkoubaa 3y ago
    I had seen that. The assumption seems to be that simulations are simplistic, and then most of what remains shows practical ways for real world training to work (while acknowledging that it's hard to do)
- flutas 3y ago
  I've thought something similar but the reverse (if I'm understanding you correctly). Using simulation to train a DIY spot on something novel and see how good I could get the performance in the real world. Of course that's a dream of mine one day and not an active project.
- mdorazio 3y ago
  You're describing something similar to what TRI has been working on for several years and seems to be making progress on: https://www.tri.global/news/toyotas-robots-are-learning-do-housework-copying-humans https://www.tri.global/news/toyotas-robots-are-learning-do-h...
manuel_w 3y ago
This is completely unrelated to the article the OP posted other than it's also related to robotics. But it's the coolest robot project I found recently. And since readers of this thread are likely to be interested in robotics (I am!), it might of interest to some: https://www.allesblinkt.com/projects/round-about-four-dimensions/ https://www.allesblinkt.com/projects/round-about-four-dimens... Hope this is not considered too off-topic here.
- deleted 3y ago
  [deleted]
- kaycebasques 3y ago
  The original post is kinda lame so we should just hijack this thread into a general robotics appreciation discussion. That cube is amazing. Thanks for sharing. The aesthetic is so polished. Some true 21st century art right there. This made the rounds a few months ago but worth posting here again. So inspiring. https://www.youtube.com/watch?v=bO-DWWFolPw https://www.youtube.com/watch?v=bO-DWWFolPw
msadowski 3y ago
My journey is a little bit opposite to the author’s: I studied robotics and started learning programming by myself to create software for robots. There are two observations I have that can be useful to people making the software->robotics journey. * Agile for software-hardware is hard if not impossible. For software it can be reasonable, but it’s really hard to iterate on both hardware and software at the same time. It’s easier if the hardware designed is locked or the process involves waterfall on the hardware side. * I often found that people that come from pure software to robotics don’t have control experience and something that can be easily solved with a PID controller end up being a custom code that ends up being way more complicated than needed.
- fake-name 3y ago
  I'd go one further and say that basically agile in hardware is fundamentally impossible in any way people commonly use agile. If you have any custom hardware, you are basically stuck with a turn time of at minimum a day or two for any changes (often more, weeks + for new PCBs is common if you're not throwing huge amounts of money at people). In this context, any process that depends on rapid small iterations is basically impossible, because each iteration just takes too much time.
  - gregwebs 3y ago
    When hardware design can be software emulated then iteration can be more agile. There's a story how NVIDIA was the first one to do this for GPUs- this was done out of desperation because they were out of money and had to ship quickly. They didn't have time and money to do any revisions so they just shipped what they had done in software emulation even though some of the features were defective. https://www.acquired.fm/episodes/nvidia-the-gpu-company-1993-2006 https://www.acquired.fm/episodes/nvidia-the-gpu-company-1993...
    - FirmwareBurner 3y ago
      Absolutely all chip designs get software emulated/simulated for rigorous testing before being sent to the fab for production. It seems like the publication turned what is industry standard into a sensationalistic article. The only thing Nvidia did different was rolling directly to tapeout with their simulated design without any intermediate prototypes which was indeed a risky move but not unheard of for cash strapped semi startups.
bemusedthrow75 3y ago
This is very interesting. But quite a lot of CS people -- well at least I hope it is not just me -- end up on the web development trajectory despite or because of having less than good enough understanding of e.g. trigonometry, geometry and calculus... all of which start to matter a lot when you are starting to make things; especially things that move and consume electricity. I am a good programmer but a weak mathematician. I have a 3D printer now, and I'm starting to learn FreeCAD and microcontroller stuff actively rather than just reading about it, and run up against my maths weakness all the time. I have recently discovered the three books Joan Horvath and Rich Cameron wrote: https://www.makershed.com/products/make-trigonometry-print https://www.makershed.com/products/make-trigonometry-print https://www.makershed.com/products/make-calculus-print https://www.makershed.com/products/make-calculus-print https://www.makershed.com/products/make-geometry https://www.makershed.com/products/make-geometry
- bouk 3y ago
  Some trig and geometry is definitely useful—but it's quite limited to be honest. For most things you can use existing algebra libraries and for most things someone else has figured it out before!
  - tonyarkles 3y ago
    I don’t know that I agree, even at the simple project level. If the problem you’re trying to solve has been solved exactly before and you can find a good reference to copy then you might be ok (for example the inverted pendulum problem). But in my experience you end up solving a related problem instead and that’ll often require re-deriving the equations of motion with whatever quirk is required. The other part that’ll catch you pretty hard if you don’t have strong trig and linear algebra going for you is that debugging is going to be really hard. I admittedly have been deep in positioning-and-attitude control for 5 years now, but the ability to look at a transformation matrix or quaternion and mentally grok exactly what it means odd going to be incredibly useful when you’re trying to figure out why your system isn’t doing what you think it should be.
  - DragonStrength 3y ago
AYBABTME 3y ago
I often feel the same urge to work on soft/hard stuff. Not sure why I don't actually do it. Although the current stuff I work on is very cool on its own and not Candy Crush. But it's really a scourge on this world that the top tier pay in the industry is to shove ads in front of people. So I do physical things in my personal projects and then come to realize that the potential return on investment (time spent vs. expected money) would be peanuts in comparison to pure software. I would love to know of hardware related work that has decent margins. Or at least enough margins to justify paying a good salary to enough engineers to sustain them.
- bemusedthrow75 3y ago
  > But it's really a scourge on this world that the top tier pay in the industry is to shove ads in front of people. This cannot be said enough, IMO. The web software world's endless quest for the shallow and lucrative is exhausting.
tonmoy 3y ago
As someone who works in making physical things, I disagree with the premise that the world of bits and bytes does not have impact. We will always need to manage resources including humans at a large scale and web apps are the best way to do that.
mdorazio 3y ago
Really wish the author would have commented on salary. I personally think the reason more people don't end up on the hardware/robotics side is that the questionable-value fintech/adtech/socialtech pure software side of the world pays so much better and most developers follow the money.
- edge17 3y ago
  Definitely agree here. I have been making this transition and currently work on life science lab automation robotics. It definitely takes intentionality because if you're good at software it's easy to get steered towards lucrative but higher level places in the stack.
- bouk 3y ago
  There are certainly hardware companies that value software enough to pay well
  - giancarlostoro 3y ago
    I am assuming anything mission critical.
  - FirmwareBurner 3y ago
    But few companies do, usually American ones, and their open positions are usually less in number than software and they're only in certain regions. If you don't live there you're shit out of luck.
- DoctorDabadedoo 3y ago
  Robotics is very niche and the market is dominated by early stage startups (since most of them go out of business a few years in), so salaries are average unless you are working specific jobs for FAANG. Job hoping usually means moving elsewhere, since working close to the hardware makes it much easier, which in turn means having a good picture of what is a competitive salary sometimes is not obvious. source: I work in robotics. AMA.
  - 3y ago
richrichie 3y ago
“ An app is not going to house people, feed them or put them on Mars. We can’t solve climate change with smarter software” Yes it can and we can. Software has been a critical component of much progress we have made over the last few decades. I understand the difficulty with the abstract many people have. Some may think Hilbert spaces are not real. For many they are as real as any space around them. Some live in them !
- AnarchismIsCool 3y ago
  Ad-tech isn't feeding the poor or curing cancer, it's just giving everyone depression.
  - richrichie 3y ago
    [dead]
moffkalast 3y ago
> Also, if you’re not already, you should get into writing Rust. The powerful type system and safety is incredible. The build tooling and ecosystem is much better than C++ et al. Rust for embedded is not there yet, but improving all the time. Meh, maybe in a decade. ROS has no official support for it either. Robotics is still very much a C++ town and will probably remain as such for a long while as long as people remain stubbornly set in their ways. If you apply as a Rust only dev to a robotics company nobody will take you seriously.
andrew_eu 3y ago
What would be a good starting platform for a programmable drone? The article suggests playing around with a Raspberry Pi Pico, but this is a bit bare bones. Are there any kits, or entry level programmable drones on the market with a reasonable toolchain to program sequences?
- xrd 3y ago
  I came here to ask the same question. Anyone?
- jacquesm 3y ago
  A Raspberry Pi Pico has a large multiple of the kind of compute that cruise missiles have so should be more than plenty for a programmable drone.
  - andrew_eu 3y ago
    While true, it also doesn't include any of the hardware needed to move around. I'm sure it would be possible to build a drone that includes it as the microcontroller, but that creates a substantial barrier to actually getting something off the ground. I have looked at the Ryze Tello [0] before, and it seems like a decent entry-level device, but I'm not sure whether it's actually a huge pain to develop for. I'd like something to be extendable _like_ a Pico, for instance if I want to install additional sensors, but all of these platforms are quite foreign to me. [0] https://www.ryzerobotics.com/tello https://www.ryzerobotics.com/tello
    - jacquesm 3y ago
      With the price of these you could stick a few in there (there would be a weight penalty, they are not exactly zero grams though for their weight the amount of compute is impressive but you then would also need more power). Depending on what kind of drone you want to build (powered fixed wing, glider, quad, hex or octocopter) that might be more or less of an issue. Given the power draw and weight of an octocopter I don't think it would be much of an issue there but on a quadcopter, especially a very small one that might be prohibitive. But I'd focus on getting it to work before optimizing for size and weight.
hcks 3y ago
“You should base your life decisions on the koolaid flavour you prefer”
tester756 3y ago
If only anything closer to hardware wasnt paying this bad... A lot of SEs leave industries like semiconductors for web dev jobs that pay waay better
choonway 3y ago
The best way to get into it without investing money in all that hardware is to play with a simulator. My preference is CoppeliaSim - due to it's user friendliness. https://www.coppeliarobotics.com/ https://www.coppeliarobotics.com/ But take your pick there are many to choose from. https://www.sciencedirect.com/science/article/abs/pii/S1569190X22001046 https://www.sciencedirect.com/science/article/abs/pii/S15691...
AnarchismIsCool 3y ago
Some concepts that you'll need to get familiar with: Real time operating systems. Less fancy than they sound but the devil is in the details. Robots need things to happen at a certain speed and at the right time so we have a type of scheduler (that can be patched into the linux kernel) that sacrifices absolute throughput to try and guarantee tasks start inside a particular window. Funny enough, if you've done game development and recognize that everything needs to happen inside 1/60th of a second or better, you know some of the hard parts here. Memory mapped addresses. C is scary but ultimately fairly simple. Once you get the hang of doing silly things with pointers and arrays, the next step is dealing with microcontrollers. You probably wonder how they do anything without an operating system though, and the answer is memory mapped IO. They have a fully flat memory space, starting with 0x0 and going up from there. That space usually contains basically everything from your stack, heap, flash storage, and all the peripherals like GPIO, I2C, SPI, serial, and so on. You can literally do things like int* x = 0x12345678; *x = 0x1; to turn on an LED because the device is listening for changes to that address to set the output state. There's a ton of other stuff, but these are the gateways to understanding the space you're dealing with at a basic level.
- iamcreasy 3y ago
  Ah, I remember Casey Muratori speaking having desktop OS where everything is memory mapped. I remember reading a book that explains the low level Arduino schematic paired with C code that does exactly what you are describing. Is there any such book for modern microcontrollers?
  - AnarchismIsCool 3y ago
    https://www.st.com/resource/en/programming_manual/pm0214-stm32-cortexm4-mcus-and-mpus-programming-manual-stmicroelectronics.pdf https://www.st.com/resource/en/programming_manual/pm0214-stm... I'm only half kidding, the programming manuals are pretty good these days. Honestly there's more in common with the arduino and a modern uC than there is different. If you want to use the knowledge in anger you need to learn linker scripts, makefiles, gcc, gdb and jtag so you can deploy and debug code on boards that don't have self-flashing bootloaders and FTDI baked in. There's also some tribal knowledge about how ISRs work like keeping ISRs as short as possible (no print statements) but I don't know what books cover that.
NordSteve 3y ago
If you have a team nearby, go work with a FIRST Robotics Competition team as a mentor/volunteer. Great community, you'll learn a lot, and make great connections if you want to get into this area as a career.
- bouk 3y ago
  Really wish this existed in the Netherlands when I was growing up!
kaycebasques 3y ago
The article itself was a bit too holier-than-thou for me but I want more robotics content here on HN so let's hijack this thread and share robotics passion!! My own shitty first foray into robotics is an RPi that I can talk to [2]. If you didn't see "the coolest robot I've ever built" you gotta watch that... so inspiring [3] Latent Space has a robotics demo day coming up, pretty curious to see what comes out of that [4] Some stream-of-conscious thoughts about why I'm drawn to robotics: * The maker / hacker / homebrew communities that are basically just using robots to express art. Maker Faire, Burning Man, etc. * The satisfaction of writing code and seeing something physical happen. Last week I was trying to figure out how to get a shitty third party Amazon robot hat [5] to actually do something useful so I was iterating through the GPIOs and I somehow made it actually smoke. I'm weirdly proud of messing up so badly that my hardware actually smoked! * The joy of demystifying hardware and learning all the layers of abstraction just within hardware [2] https://www.biodigitaljazz.net/blog/STTTGTS.html https://www.biodigitaljazz.net/blog/STTTGTS.html [3] https://news.ycombinator.com/item?id=38162881 https://news.ycombinator.com/item?id=38162881 [4] https://lu.ma/latent-space-final-frontiers https://lu.ma/latent-space-final-frontiers [5] A Xmas present from my sweet wife, really touching that she's encouraging me to actually pursue my interest in robotics
tamimio 3y ago
As long as software folks don’t bring their shenanigans (agile/scum (ahmm scrum) approach, daily stand up, ticket based management, ship products within 2 weeks, among others) into the robotics/engineering world, you are welcome. I have seen many cases of software “project or engineering managers” ruin a whole department because they were trying to force such stupid approaches, it’s always ironic seeing some are trying to force agile for example even though the whole idea of agile is being..agile.

179 points by bouk 3y ago

ih 3y ago

I'm just getting into robotics with a background in software and I ended up choosing the Isaac Sim platform/ecosystem (https://docs.omniverse.nvidia.com/isaacsim/latest/index.html https://docs.omniverse.nvidia.com/isaacsim/latest/index.html) and Jetbot (https://jetbot.org/master/ https://jetbot.org/master/) since they seem suited for neural networks and reinforcement learning. So far getting up and running with the simulation side hasn't been too bad and there is a model for the Jetbot in Isaac Sim since it's also by Nvidia. I haven't started on the hardware side so can't speak on that. The downside for some might be it's proprietary and Jetbot itself is a bit out of date and nearing end of life support.

m00x 3y ago
You'll be better served with a more capable platform. You can either 3d print it, or you can pick up something like the WaveRover from Waveshare. It's a solid metal frame with geared motors and differential drive. For ~$100 you won't get much better. You just need to pop in 3x18650 batteries in the chassis and connect a Jetson nano to the UART pins. It has an ESP32 inside, so you can also program that, but it comes programmed out of the box. You can control it with this python library https://github.com/msanterre/wave_rover_serial https://github.com/msanterre/wave_rover_serial Isaac SIM is a terrific simulator, you won't get much better.
- ih 3y ago
  Cool, thanks for the advice!

Animats 3y ago

This might be a good time to get into robotics. For a long time, industrial robots were really dumb. Then there was a round of false enthusiasm for "intelligent robots" (Rethink Robotics went down that rathole.) Now, at last, compute power, vision, and AI are cheap enough that you can get something done. The components are much better. Batteries are better. Motors are better. Radio communications work. Cameras are cheap. Short-range LIDAR is affordable. Navigation systems work. Robotics work used to require using a lot of time building custom solutions for those problems. Now you can just order components. Here's an idea I'd like to see revisited. Back in the 1980s, someone built a pair of small forklifts that operated as a team. These were little things, about half a meter cubed, with maybe 50cm of lift. Individually, they couldn't do much. But a pair working together could pick up and move a couch, with one robot lifting each end. In the 1980s, the researchers had trouble coordinating two mobile robots. Communications alone were a big problem. Not today. There are many material handling tasks where one small robot isn't enough, and a big machine the size of a forklift is too bulky. But teams of small robots might work. I still have a small robot arm on my desk, but it's not connected to anything.

eschneider 3y ago

Nice article. As someone who's also gotten into robotics from the software side, I'd also suggest learning a bit about the hardware side. If you're doing anything with custom boards and board bring-up, you'll need to be able to read a schematic and data sheet at a basic level. Being able to use an oscilloscope and logic analyzer is also _very_ useful. None of this stuff is terribly hard, but it's nice to know what to expect.

Animats 3y ago
Yes. The number of people who both know C++ (not C) and how to use an oscilloscope is surprisingly small. Electronics at robotic speeds isn't that hard. Most signals are audio bandwidth or lower. You rarely need all the elaborate design techniques needed when you get into the MHz-Ghz range.
- FirmwareBurner 3y ago
  >The number of people who both know C++ (not C) and how to use an oscilloscope is surprisingly small. Statements like these make my eyes roll. The difficult part in embedded, and what makes the difference between a n00b spending 2 months finding a hardware/firmware issue and a graybeard finding it in under 2 days, is not how to use an oscilloscope or how to write "clean" c++, it's knowing where exactly to put the oscilloscope probes and what exactly to look for on the oscilloscope screen, and how to make the bug/edge case reproducible. You don't have unlimited scope probes, trace memory and time to probe every single signal on the board and stare at blank signals not knowing what to look for, for weeks, while th clock is ticking. Tinkering with STM32s and Raspberry Pis at home for fun, doesn't prepare you for the issues you'll encounter in debugging production devices (especially battery powered or RF) you'll need to ship on time and on budget. Knowing how to use a scope and how to write c++ is only 10% of a successful product. The other 90% is blood, sweat and tears hunched over for hours/days over boards and breakpoints to find out your race condition comes from a cheap crystal oscillator operating out of spec and not from your software. Especially that at small companies and start-ups you don't have the budgets and HW lab equipment that the likes of Apple or Qualcomm can afford to brute force your way to the solution, so you need to be very shrewd and clever with your debugging to make the most out of your limited resources.

blah-yeah 3y ago

Make magazine has a great magazine on getting into robotics via the Battle Bots type of robots. I think it was an issue from 2022 or 2023 (not enough energy currently to track down the issue in their catalog) Here's an article on their site about the topic: https://makezine.com/tag/battlebots/ https://makezine.com/tag/battlebots/

ashconnor 3y ago
Reminds me of the UK TV show Robot Wars: https://en.wikipedia.org/wiki/Robot_Wars_(TV_series) https://en.wikipedia.org/wiki/Robot_Wars_(TV_series)

bella014 3y ago

[dead]

chris_st 3y ago

I've occasionally thought it would be fun (not necessarily productive!) to do something like the experiments folks have done with Genetic Algorithms, having simulated robots learn to walk, etc. in a simulated physics environment. The interesting bit would be to do it with real legs, sensors, servos, etc., to try to build up a "naive physics" library that can deal with balance, etc., by learning from physical experience rather than starting with equations of motion. That may be the most number of times I've typed "etc." in a paragraph.

coderenegade 3y ago
System Identification is a fairly common strategy for developing a controller. Essentially, you learn the dynamics model from recorded data, and then use that as a simulator to iteratively improve the controller. The devil is in the details though, as you can run into issues with simulation fidelity (due to missing data), which can produce something that doesn't translate to real life performance.
imtringued 3y ago
https://m.youtube.com/watch?v=17NrtKHdPDw https://m.youtube.com/watch?v=17NrtKHdPDw
- mkoubaa 3y ago
  I had seen that. The assumption seems to be that simulations are simplistic, and then most of what remains shows practical ways for real world training to work (while acknowledging that it's hard to do)
flutas 3y ago
I've thought something similar but the reverse (if I'm understanding you correctly). Using simulation to train a DIY spot on something novel and see how good I could get the performance in the real world. Of course that's a dream of mine one day and not an active project.
mdorazio 3y ago
You're describing something similar to what TRI has been working on for several years and seems to be making progress on: https://www.tri.global/news/toyotas-robots-are-learning-do-housework-copying-humans https://www.tri.global/news/toyotas-robots-are-learning-do-h...

manuel_w 3y ago

This is completely unrelated to the article the OP posted other than it's also related to robotics. But it's the coolest robot project I found recently. And since readers of this thread are likely to be interested in robotics (I am!), it might of interest to some: https://www.allesblinkt.com/projects/round-about-four-dimensions/ https://www.allesblinkt.com/projects/round-about-four-dimens... Hope this is not considered too off-topic here.

deleted 3y ago
[deleted]
kaycebasques 3y ago
The original post is kinda lame so we should just hijack this thread into a general robotics appreciation discussion. That cube is amazing. Thanks for sharing. The aesthetic is so polished. Some true 21st century art right there. This made the rounds a few months ago but worth posting here again. So inspiring. https://www.youtube.com/watch?v=bO-DWWFolPw https://www.youtube.com/watch?v=bO-DWWFolPw

msadowski 3y ago

My journey is a little bit opposite to the author’s: I studied robotics and started learning programming by myself to create software for robots. There are two observations I have that can be useful to people making the software->robotics journey. * Agile for software-hardware is hard if not impossible. For software it can be reasonable, but it’s really hard to iterate on both hardware and software at the same time. It’s easier if the hardware designed is locked or the process involves waterfall on the hardware side. * I often found that people that come from pure software to robotics don’t have control experience and something that can be easily solved with a PID controller end up being a custom code that ends up being way more complicated than needed.

fake-name 3y ago
I'd go one further and say that basically agile in hardware is fundamentally impossible in any way people commonly use agile. If you have any custom hardware, you are basically stuck with a turn time of at minimum a day or two for any changes (often more, weeks + for new PCBs is common if you're not throwing huge amounts of money at people). In this context, any process that depends on rapid small iterations is basically impossible, because each iteration just takes too much time.
- gregwebs 3y ago
  When hardware design can be software emulated then iteration can be more agile. There's a story how NVIDIA was the first one to do this for GPUs- this was done out of desperation because they were out of money and had to ship quickly. They didn't have time and money to do any revisions so they just shipped what they had done in software emulation even though some of the features were defective. https://www.acquired.fm/episodes/nvidia-the-gpu-company-1993-2006 https://www.acquired.fm/episodes/nvidia-the-gpu-company-1993...
  - FirmwareBurner 3y ago
    Absolutely all chip designs get software emulated/simulated for rigorous testing before being sent to the fab for production. It seems like the publication turned what is industry standard into a sensationalistic article. The only thing Nvidia did different was rolling directly to tapeout with their simulated design without any intermediate prototypes which was indeed a risky move but not unheard of for cash strapped semi startups.

bemusedthrow75 3y ago

This is very interesting. But quite a lot of CS people -- well at least I hope it is not just me -- end up on the web development trajectory despite or because of having less than good enough understanding of e.g. trigonometry, geometry and calculus... all of which start to matter a lot when you are starting to make things; especially things that move and consume electricity. I am a good programmer but a weak mathematician. I have a 3D printer now, and I'm starting to learn FreeCAD and microcontroller stuff actively rather than just reading about it, and run up against my maths weakness all the time. I have recently discovered the three books Joan Horvath and Rich Cameron wrote: https://www.makershed.com/products/make-trigonometry-print https://www.makershed.com/products/make-trigonometry-print https://www.makershed.com/products/make-calculus-print https://www.makershed.com/products/make-calculus-print https://www.makershed.com/products/make-geometry https://www.makershed.com/products/make-geometry

bouk 3y ago
Some trig and geometry is definitely useful—but it's quite limited to be honest. For most things you can use existing algebra libraries and for most things someone else has figured it out before!
- tonyarkles 3y ago
  I don’t know that I agree, even at the simple project level. If the problem you’re trying to solve has been solved exactly before and you can find a good reference to copy then you might be ok (for example the inverted pendulum problem). But in my experience you end up solving a related problem instead and that’ll often require re-deriving the equations of motion with whatever quirk is required. The other part that’ll catch you pretty hard if you don’t have strong trig and linear algebra going for you is that debugging is going to be really hard. I admittedly have been deep in positioning-and-attitude control for 5 years now, but the ability to look at a transformation matrix or quaternion and mentally grok exactly what it means odd going to be incredibly useful when you’re trying to figure out why your system isn’t doing what you think it should be.
- DragonStrength 3y ago

AYBABTME 3y ago

I often feel the same urge to work on soft/hard stuff. Not sure why I don't actually do it. Although the current stuff I work on is very cool on its own and not Candy Crush. But it's really a scourge on this world that the top tier pay in the industry is to shove ads in front of people. So I do physical things in my personal projects and then come to realize that the potential return on investment (time spent vs. expected money) would be peanuts in comparison to pure software. I would love to know of hardware related work that has decent margins. Or at least enough margins to justify paying a good salary to enough engineers to sustain them.

bemusedthrow75 3y ago
> But it's really a scourge on this world that the top tier pay in the industry is to shove ads in front of people. This cannot be said enough, IMO. The web software world's endless quest for the shallow and lucrative is exhausting.

tonmoy 3y ago

As someone who works in making physical things, I disagree with the premise that the world of bits and bytes does not have impact. We will always need to manage resources including humans at a large scale and web apps are the best way to do that.

mdorazio 3y ago

Really wish the author would have commented on salary. I personally think the reason more people don't end up on the hardware/robotics side is that the questionable-value fintech/adtech/socialtech pure software side of the world pays so much better and most developers follow the money.

edge17 3y ago
Definitely agree here. I have been making this transition and currently work on life science lab automation robotics. It definitely takes intentionality because if you're good at software it's easy to get steered towards lucrative but higher level places in the stack.
bouk 3y ago
There are certainly hardware companies that value software enough to pay well
- giancarlostoro 3y ago
  I am assuming anything mission critical.
- FirmwareBurner 3y ago
  But few companies do, usually American ones, and their open positions are usually less in number than software and they're only in certain regions. If you don't live there you're shit out of luck.
DoctorDabadedoo 3y ago
Robotics is very niche and the market is dominated by early stage startups (since most of them go out of business a few years in), so salaries are average unless you are working specific jobs for FAANG. Job hoping usually means moving elsewhere, since working close to the hardware makes it much easier, which in turn means having a good picture of what is a competitive salary sometimes is not obvious. source: I work in robotics. AMA.
- 3y ago

richrichie 3y ago

“ An app is not going to house people, feed them or put them on Mars. We can’t solve climate change with smarter software” Yes it can and we can. Software has been a critical component of much progress we have made over the last few decades. I understand the difficulty with the abstract many people have. Some may think Hilbert spaces are not real. For many they are as real as any space around them. Some live in them !

AnarchismIsCool 3y ago
Ad-tech isn't feeding the poor or curing cancer, it's just giving everyone depression.
- richrichie 3y ago
  [dead]

moffkalast 3y ago

> Also, if you’re not already, you should get into writing Rust. The powerful type system and safety is incredible. The build tooling and ecosystem is much better than C++ et al. Rust for embedded is not there yet, but improving all the time. Meh, maybe in a decade. ROS has no official support for it either. Robotics is still very much a C++ town and will probably remain as such for a long while as long as people remain stubbornly set in their ways. If you apply as a Rust only dev to a robotics company nobody will take you seriously.

andrew_eu 3y ago

What would be a good starting platform for a programmable drone? The article suggests playing around with a Raspberry Pi Pico, but this is a bit bare bones. Are there any kits, or entry level programmable drones on the market with a reasonable toolchain to program sequences?

xrd 3y ago
I came here to ask the same question. Anyone?
jacquesm 3y ago
A Raspberry Pi Pico has a large multiple of the kind of compute that cruise missiles have so should be more than plenty for a programmable drone.
- andrew_eu 3y ago
  While true, it also doesn't include any of the hardware needed to move around. I'm sure it would be possible to build a drone that includes it as the microcontroller, but that creates a substantial barrier to actually getting something off the ground. I have looked at the Ryze Tello [0] before, and it seems like a decent entry-level device, but I'm not sure whether it's actually a huge pain to develop for. I'd like something to be extendable _like_ a Pico, for instance if I want to install additional sensors, but all of these platforms are quite foreign to me. [0] https://www.ryzerobotics.com/tello https://www.ryzerobotics.com/tello
  - jacquesm 3y ago
    With the price of these you could stick a few in there (there would be a weight penalty, they are not exactly zero grams though for their weight the amount of compute is impressive but you then would also need more power). Depending on what kind of drone you want to build (powered fixed wing, glider, quad, hex or octocopter) that might be more or less of an issue. Given the power draw and weight of an octocopter I don't think it would be much of an issue there but on a quadcopter, especially a very small one that might be prohibitive. But I'd focus on getting it to work before optimizing for size and weight.

hcks 3y ago

“You should base your life decisions on the koolaid flavour you prefer”

tester756 3y ago

If only anything closer to hardware wasnt paying this bad... A lot of SEs leave industries like semiconductors for web dev jobs that pay waay better

choonway 3y ago

The best way to get into it without investing money in all that hardware is to play with a simulator. My preference is CoppeliaSim - due to it's user friendliness. https://www.coppeliarobotics.com/ https://www.coppeliarobotics.com/ But take your pick there are many to choose from. https://www.sciencedirect.com/science/article/abs/pii/S1569190X22001046 https://www.sciencedirect.com/science/article/abs/pii/S15691...

AnarchismIsCool 3y ago

Some concepts that you'll need to get familiar with: Real time operating systems. Less fancy than they sound but the devil is in the details. Robots need things to happen at a certain speed and at the right time so we have a type of scheduler (that can be patched into the linux kernel) that sacrifices absolute throughput to try and guarantee tasks start inside a particular window. Funny enough, if you've done game development and recognize that everything needs to happen inside 1/60th of a second or better, you know some of the hard parts here. Memory mapped addresses. C is scary but ultimately fairly simple. Once you get the hang of doing silly things with pointers and arrays, the next step is dealing with microcontrollers. You probably wonder how they do anything without an operating system though, and the answer is memory mapped IO. They have a fully flat memory space, starting with 0x0 and going up from there. That space usually contains basically everything from your stack, heap, flash storage, and all the peripherals like GPIO, I2C, SPI, serial, and so on. You can literally do things like int* x = 0x12345678; *x = 0x1; to turn on an LED because the device is listening for changes to that address to set the output state. There's a ton of other stuff, but these are the gateways to understanding the space you're dealing with at a basic level.

iamcreasy 3y ago
Ah, I remember Casey Muratori speaking having desktop OS where everything is memory mapped. I remember reading a book that explains the low level Arduino schematic paired with C code that does exactly what you are describing. Is there any such book for modern microcontrollers?
- AnarchismIsCool 3y ago
  https://www.st.com/resource/en/programming_manual/pm0214-stm32-cortexm4-mcus-and-mpus-programming-manual-stmicroelectronics.pdf https://www.st.com/resource/en/programming_manual/pm0214-stm... I'm only half kidding, the programming manuals are pretty good these days. Honestly there's more in common with the arduino and a modern uC than there is different. If you want to use the knowledge in anger you need to learn linker scripts, makefiles, gcc, gdb and jtag so you can deploy and debug code on boards that don't have self-flashing bootloaders and FTDI baked in. There's also some tribal knowledge about how ISRs work like keeping ISRs as short as possible (no print statements) but I don't know what books cover that.

NordSteve 3y ago

If you have a team nearby, go work with a FIRST Robotics Competition team as a mentor/volunteer. Great community, you'll learn a lot, and make great connections if you want to get into this area as a career.

bouk 3y ago
Really wish this existed in the Netherlands when I was growing up!

kaycebasques 3y ago

The article itself was a bit too holier-than-thou for me but I want more robotics content here on HN so let's hijack this thread and share robotics passion!! My own shitty first foray into robotics is an RPi that I can talk to [2]. If you didn't see "the coolest robot I've ever built" you gotta watch that... so inspiring [3] Latent Space has a robotics demo day coming up, pretty curious to see what comes out of that [4] Some stream-of-conscious thoughts about why I'm drawn to robotics: * The maker / hacker / homebrew communities that are basically just using robots to express art. Maker Faire, Burning Man, etc. * The satisfaction of writing code and seeing something physical happen. Last week I was trying to figure out how to get a shitty third party Amazon robot hat [5] to actually do something useful so I was iterating through the GPIOs and I somehow made it actually smoke. I'm weirdly proud of messing up so badly that my hardware actually smoked! * The joy of demystifying hardware and learning all the layers of abstraction just within hardware [2] https://www.biodigitaljazz.net/blog/STTTGTS.html https://www.biodigitaljazz.net/blog/STTTGTS.html [3] https://news.ycombinator.com/item?id=38162881 https://news.ycombinator.com/item?id=38162881 [4] https://lu.ma/latent-space-final-frontiers https://lu.ma/latent-space-final-frontiers [5] A Xmas present from my sweet wife, really touching that she's encouraging me to actually pursue my interest in robotics

tamimio 3y ago

As long as software folks don’t bring their shenanigans (agile/scum (ahmm scrum) approach, daily stand up, ticket based management, ship products within 2 weeks, among others) into the robotics/engineering world, you are welcome. I have seen many cases of software “project or engineering managers” ruin a whole department because they were trying to force such stupid approaches, it’s always ironic seeing some are trying to force agile for example even though the whole idea of agile is being..agile.