6 ms·

Given that this approach needs a manual written by somebody who already understands how to assemble a lego kit, this is not practically useful. The person who w

Given that this approach needs a manual written by somebody who already understands how to assemble a lego kit, this is not practically useful. The person who wrote the manual can also specify a set of machine-readable instructions (for example, in a programming language). In short, this looks cool, but it's pointless.

soraki_soladead 4y ago
Many more people can put together a manual of step-by-step pictures than can program.
- YeGoblynQueenne 4y ago
  Yes, but if the step-by-step procedure is known, it can be programmed manually and does not need to be machine-learned from examples. It doesn't have to be the same person who writes the manual and does the coding. An approach like the one in the article above would be useful if it could derive a procedure we don't know how to code manually.
outsidetheparty 4y ago
Couldn’t possibly disagree more; that’s akin to saying that feature identification is not useful, because if we really wanted to we could just deliver a list of identifiable features along with every photograph. The point of this, obviously, is not to make it easier for machines to construct LEGO sets. It’s just a really clever problem space to do research into improving machine vision and image interpretation: it’s complicated enough to run into meaningful challenges, but constrained enough to make progress achievable. (The “Minecraft” dataset mentioned sounds like a simpler baby-steps version of the same, since there’s a much more limited set of possible shapes and connections to work with.) Plus they get to be the fun lab with all the LEGOs in it
- YeGoblynQueenne 4y ago
  >> Couldn’t possibly disagree more; that’s akin to saying that feature identification is not useful, because if we really wanted to we could just deliver a list of identifiable features along with every photograph. These are not arbitrary photographs but images of legos. The features needed are few and finite and there's no need for machine vision to find them. This is pointless and has no real-world usefulness, other than to look like "the fun lab with all the legos", like you say.
  - outsidetheparty 4y ago
    Again: you somehow appear to have missed the point that this is a research project, not a SaaS proposal. It is not intended to have "real world usefulness", it's intended to advance the science of feature identification and image recognition. You know: science. You're a PhD student yourself, which makes it especially confusing that you appear to believe research has no value in its own right. You are of course correct that these are not arbitrary photographs. Again: it's a domain containing enough complexity to offer meaningful challenges, but constrained enough to allow achievable progress. > The features needed are few and finite and there's no need for machine vision to find them. Finite, sure. Few, not so much; there are thousands of distinct types of LEGO block. If you're going to try to make the case that these diagrams can just be trivially parsed into the corresponding parts, orientation, and connections, while also accounting for occlusion of elements and model rotation across diagrams... well, you're probably going to have to try a lot harder to make that case than by just stating it as fact.
    - YeGoblynQueenne 4y ago
      >> Again: you somehow appear to have missed the point that this is a research project, not a SaaS proposal. It is not intended to have "real world usefulness", it's intended to advance the science of feature identification and image recognition. You know: science. You're a PhD student yourself, which makes it especially confusing that you appear to believe research has no value in its own right. Your comment is aggressive and a personal attack, and for this reason I'm not engaging you further.
visarga 4y ago
If only the persons making invoices and receipts would have specified a set of machine readable values for the fields, a whole task in NLP would have disappeared.
- deleted 4y ago
  [deleted]
- YeGoblynQueenne 4y ago
  Why NLP? Reading (hand-written) receipts and identifying field contents is a machine vision problem, not an NLP problem. Anyway this is a much easier problem than either NLP or handwritten digit recognition and the solution is already known, so there's no need for machine learning. Ordinary, manual programming can do the job just as well. For a bit of perspective, industrial robots are programmed by hand and they don't build stuff with legos. Edit: so what kind of non-lego application would you expect of a model that can put together lego assembly plans from lego assembly manuals? I'm curious.
  - visarga 4y ago
    It's a problem in the middle between CV and NLP. It used to be solved with LSTM sequence tagging models, now solved with combinations of CNN and transformer with 2d coordinate embeddings. It's far from being solved, not even receipts work 99% correct, real scores between 89..92%. Manual programming solutions are very bad, they usually do regex matching or template matching. If you want to extract 5-10 fields from a document then you'll have at least one or more errors per page, unfortunately not "automating" too many documents out of 100 yet, with any method.
    - YeGoblynQueenne 4y ago
      To be honest, 89-92% sounds too optimistic for a real-world application. But I was talking about applications of the lego assembly model. My intuition is that lego is a special case: the pieces are made to interlock with each other and there are only a small number of ways in which two pieces can lock together. Compare this with building a prefab house, for example, or a car, where elements must be bound together with nails, mortar, soldering, or glue etc, in essentially unpredictable ways. I say that the approach described is pointless because I can't see any real-world applications of extracting assembly plans from a visualisation of an already-known assembly plan. Even less so when the object assembled is just a toy. And it's been possible to assemble toys using machine vision, given a pre-determined set of instructions, from the 1970's already: (Donald Michie presenting the Edinburgh Research Purpose General Assembly Robot, a.k.a. "Freddy"): https://youtu.be/03p2CADwGF8?t=1668 https://youtu.be/03p2CADwGF8?t=1668 More info of Freddy here https://www.aiai.ed.ac.uk/project/freddy/ https://www.aiai.ed.ac.uk/project/freddy/ Freddy was tediously hand-programmed with a full plan but it used vision and touch to find the pieces of the toy it assembles in the video.

soraki_soladead 4y ago

Many more people can put together a manual of step-by-step pictures than can program.

YeGoblynQueenne 4y ago
Yes, but if the step-by-step procedure is known, it can be programmed manually and does not need to be machine-learned from examples. It doesn't have to be the same person who writes the manual and does the coding. An approach like the one in the article above would be useful if it could derive a procedure we don't know how to code manually.

outsidetheparty 4y ago

Couldn’t possibly disagree more; that’s akin to saying that feature identification is not useful, because if we really wanted to we could just deliver a list of identifiable features along with every photograph. The point of this, obviously, is not to make it easier for machines to construct LEGO sets. It’s just a really clever problem space to do research into improving machine vision and image interpretation: it’s complicated enough to run into meaningful challenges, but constrained enough to make progress achievable. (The “Minecraft” dataset mentioned sounds like a simpler baby-steps version of the same, since there’s a much more limited set of possible shapes and connections to work with.) Plus they get to be the fun lab with all the LEGOs in it

YeGoblynQueenne 4y ago
>> Couldn’t possibly disagree more; that’s akin to saying that feature identification is not useful, because if we really wanted to we could just deliver a list of identifiable features along with every photograph. These are not arbitrary photographs but images of legos. The features needed are few and finite and there's no need for machine vision to find them. This is pointless and has no real-world usefulness, other than to look like "the fun lab with all the legos", like you say.
- outsidetheparty 4y ago
  Again: you somehow appear to have missed the point that this is a research project, not a SaaS proposal. It is not intended to have "real world usefulness", it's intended to advance the science of feature identification and image recognition. You know: science. You're a PhD student yourself, which makes it especially confusing that you appear to believe research has no value in its own right. You are of course correct that these are not arbitrary photographs. Again: it's a domain containing enough complexity to offer meaningful challenges, but constrained enough to allow achievable progress. > The features needed are few and finite and there's no need for machine vision to find them. Finite, sure. Few, not so much; there are thousands of distinct types of LEGO block. If you're going to try to make the case that these diagrams can just be trivially parsed into the corresponding parts, orientation, and connections, while also accounting for occlusion of elements and model rotation across diagrams... well, you're probably going to have to try a lot harder to make that case than by just stating it as fact.
  - YeGoblynQueenne 4y ago
    >> Again: you somehow appear to have missed the point that this is a research project, not a SaaS proposal. It is not intended to have "real world usefulness", it's intended to advance the science of feature identification and image recognition. You know: science. You're a PhD student yourself, which makes it especially confusing that you appear to believe research has no value in its own right. Your comment is aggressive and a personal attack, and for this reason I'm not engaging you further.

visarga 4y ago

If only the persons making invoices and receipts would have specified a set of machine readable values for the fields, a whole task in NLP would have disappeared.

deleted 4y ago
[deleted]
YeGoblynQueenne 4y ago
Why NLP? Reading (hand-written) receipts and identifying field contents is a machine vision problem, not an NLP problem. Anyway this is a much easier problem than either NLP or handwritten digit recognition and the solution is already known, so there's no need for machine learning. Ordinary, manual programming can do the job just as well. For a bit of perspective, industrial robots are programmed by hand and they don't build stuff with legos. Edit: so what kind of non-lego application would you expect of a model that can put together lego assembly plans from lego assembly manuals? I'm curious.
- visarga 4y ago
  It's a problem in the middle between CV and NLP. It used to be solved with LSTM sequence tagging models, now solved with combinations of CNN and transformer with 2d coordinate embeddings. It's far from being solved, not even receipts work 99% correct, real scores between 89..92%. Manual programming solutions are very bad, they usually do regex matching or template matching. If you want to extract 5-10 fields from a document then you'll have at least one or more errors per page, unfortunately not "automating" too many documents out of 100 yet, with any method.
  - YeGoblynQueenne 4y ago
    To be honest, 89-92% sounds too optimistic for a real-world application. But I was talking about applications of the lego assembly model. My intuition is that lego is a special case: the pieces are made to interlock with each other and there are only a small number of ways in which two pieces can lock together. Compare this with building a prefab house, for example, or a car, where elements must be bound together with nails, mortar, soldering, or glue etc, in essentially unpredictable ways. I say that the approach described is pointless because I can't see any real-world applications of extracting assembly plans from a visualisation of an already-known assembly plan. Even less so when the object assembled is just a toy. And it's been possible to assemble toys using machine vision, given a pre-determined set of instructions, from the 1970's already: (Donald Michie presenting the Edinburgh Research Purpose General Assembly Robot, a.k.a. "Freddy"): https://youtu.be/03p2CADwGF8?t=1668 https://youtu.be/03p2CADwGF8?t=1668 More info of Freddy here https://www.aiai.ed.ac.uk/project/freddy/ https://www.aiai.ed.ac.uk/project/freddy/ Freddy was tediously hand-programmed with a full plan but it used vision and touch to find the pieces of the toy it assembles in the video.