5 ms·
This is a really cool concept. The scene from the video where it is identifying bricks from a pile seems like a crazy awesome computer vision application. Does
by tacLog 5y ago
This is a really cool concept. The scene from the video where it is identifying bricks from a pile seems like a crazy awesome computer vision application.
Does anyone have any references to how they managed that brick identifying algorithm?
- elephantum 5y agoSeveral neural networks, lots of hard work in data labelling, more hard work in data synthesis, even more hard work in dataset curation.. et voila! Source: it was me with the team who did all of that.
- c9fc42ad 5y agoThis is really cool! Do you have any in depth posts describing the ML behind this?
- elephantum 5y agoNot yet, but it seems that we will talk about it
- elephantum 5y agoTo dig deeper, there are couple of interesting nuances there: - extreme number of objects on single photo, typical number of visible pieces in large pile is 1500-2000 - extreme number of classes in multi-class classification, there are ~1000 most common Lego bricks and up to 30000 classes if you include rare bricks and different patterns - really hard data labelling: one photo can take up to a 5 work days to label
- brutus1213 5y agoVery cool! How do you deal with occlusion? What kind of NN are you using? Would an off-the-shelf model like Yolo/SSD/FasterRCNN work for this or did you have to devise some new tricks?
- tacLog 5y agoI am really curious about the data labelling. Do you mean that you took photos of piles of Legos and labeled them by hand. Or individual Legos from different angles? Also you mentioned data synthesis. How would this be possible? Unless your suggesting that you rendered photo realistic piles of Legos and used trained on them because if that is the case, please do a write up of the project. I can't imagine more interesting way to generate training data.
- lgrebe 5y agoSuper neat! With all this hard work, how is it free? Am I / my data being sold or is it just a “student tech demo” with an Apple dev account ?
- caf 5y agoIt seems like "go viral, get acquired by Lego" would be a reasonable game plan.
- wjnc 5y agoYea, this. An app like this is a total and legal potentially huge competitor to Lego. I’d wager Lego will happily pay 100 M€+ if this would never come to fruition in order to keep the new box with book flow not slowing due to this idea. The cat is out of the box however. This is disruption in optima forma.
- wjnc 5y agoSince downvotes are not my thing, I’ll explain my logic. The value of Legos is return customers. Build, break, drop it in the pile and never build again. (This with great fun obviously, but my experience is no box stays separate in the long run.) Any outside disruption that massively increases the replay value of boxes (perhaps even find that old box from among 20k pieces?) is a threat to Lego. They make about 1 € billion a year on 6 € billion gross. You have to take the lifetime value of the disruption, hence my 100 M€ estimate. It is an estimate that says the repeat value of Lego will rise for the customer with a fall in repeat sales for Lego in the 1% range.
- caf 5y agoI tend to disagree. Lego's main competitor isn't their own old sets; it's alternate entertainment like XBox or YouTube. I would estimate that any innovation like this that encourages people to pull out and play with their Lego strongly increases the chances that they'll sell new sets to those people.
- zby 5y agoCongrats! I have been working (or rather I tried to find someone competent to work on an app like this) but it is still in proof of concept phase.