4 ms·

For those with experience with ML in Julia I’d love to get some advice. I have a little business that sells 3D printed jewelry (https://lulimjewelry.com https:/

For those with experience with ML in Julia I’d love to get some advice. I have a little business that sells 3D printed jewelry (https://lulimjewelry.com https://lulimjewelry.com). My biggest seller is customers engraving their or their loved ones fingerprint on the ring. Most of those prints come in needing manual cleanup, which I can usually do in a few minutes. I’d love to train a ML algorithm to do this, and I’ve been building up the before and after pictures over time using my manually cleaned up customer fingerprint images. Can anyone give me suggestions or pointers on the sort of algorithm that may be best suited for this task? Just something to get me started down the correct path would be very helpful.

tomrod 6y ago
ML practitioner here. Step 1: write out the steps you do to manually clean up. Step 2: Depending on: (1) how many samples you have, (2) what is defined as "before" and "after", (3) what performance level is acceptable, there are many algorithms to choose from. Transform learning may be the target, reinforcement learning or CNN as well. It really depends on Step (1) and (2.1).
- doctoboggan 6y ago
  For a given poor quality print I usually take it on my iPad and retrace either the whole print or just a portion. I usually just use the pencil tool in white and run it between the ridges of the print to give good separation and definition to the final engraving. Depending on the quality of the print (some of them are hardly more defined that a low res ink smudge) I may have to use a good amount of artistic liberties on where I think the pattern was going. Using my knowledge of what a print may look like I can usually do a decent job. Thats why I thought this might be a good task for a ML algorithm since I can show it what a bad print looks like and what a decent cleaned up version of that print should look like.
  - tomrod 6y ago
    So you're denoising by identifying where the patterns should be, but there is some smudge? If so, you may be able to skip ML altogether and use a deterministic software program to classify light versus dark areas (what you're doing with your pencil). ML might help with mapping smudges to the right shape/sizing. That said, if you have a ton of examples, then yeah an image recognition task might help.
  - tomrod 6y ago
    Also: https://towardsdatascience.com/fingerprint-denoising-and-inpainting-using-fully-convolutional-networks-e24714c3233 https://towardsdatascience.com/fingerprint-denoising-and-inp...
    - doctoboggan 6y ago
      Wow, this is basically exactly what I needed, thanks! Seems like the key word I was missing in my searches was "inpainting"
      - tomrod 6y ago
        Glad to hear it.
        doctoboggan 6y ago
        I got their model running with the included training data and it’s quite impressive. Some of my more extreme poor examples don’t produce reasonable results but many do.
        tomrod 6y ago
        That's awesome! Happy scaling!
neolog 6y ago
Sounds like you want to convert raster images to vector. https://vectormagic.com/ https://vectormagic.com/ is a commercial version.
- doctoboggan 6y ago
  Thats not actually my problem. For a good quality print I can do this with no issue. (I use imagetracer.js in my web app) but the problem I have is many prints that come through smudged or otherwise damaged. In those cases I have to retrace them, sometimes taking a decent amount of artistic license using what I know about how other prints looks.
  - ramraj07 6y ago
    How many samples do you have of smudged prints that have been corrected? Can you also artificially smudge prints to create a training set? Then an interesting solution might be possible!
    - doctoboggan 6y ago
      I currently only have about 100 fixed up image sets. One of the other commenters pointed me to a blog that mentioned this software for generating realistic finger print images: https://dsl.cds.iisc.ac.in/projects/Anguli/ https://dsl.cds.iisc.ac.in/projects/Anguli/ which could then be artificially smudged. There seems to be a dataset out there somewhat that has already done this (https://competitions.codalab.org/competitions/18426 https://competitions.codalab.org/competitions/18426) but I can't find out where to download it.
      - ramraj07 6y ago
        I would advise against completely synthetic prints, you want to model your training data as close to reality as possible! Can't you just blur other finger prints you have?
syntaxing 6y ago
Depends on what you are cleaning. I have a hunch you wouldn't even need anything ML related. Just the right computer vision pipeline using OpenCV should get you really close. If you really want to use ML/DL because its cool and fun, I'm pretty sure Autoencoders will do what you need [1]. If you're lucky, the denoising autoencoder should do the clean up automatically. Alternative is to use a VAE which goes really far compared to GAN since its much more sampling efficient and easier to train. I think(?) it's what was used in the original deepfake. [1] https://www.mygreatlearning.com/blog/autoencoder/ https://www.mygreatlearning.com/blog/autoencoder/
- tomrod 6y ago
  You're right on the VAE. Depending on output quality, my experience has been fewer samples are needed for VAE training.
- teruakohatu 6y ago
  I agree. For cleaning up a fingerprint I would think you could get good enough using Opencv. People have been cleanly vectoring raster images long before deep learning. If you can automate it using a combination of Gimp filters + Inkscape, you probably can do it pretty easily convert that process into code.
  - doctoboggan 6y ago
    I can and do simply rasterize images for some of my clean and proper fingerprints. The problem I am having is with smudges, incomplete prints, or other artifacts. In those cases I trace over and fix the print sometimes taking a decent amount of artistic license depending on the underlying quality of the image.
    - bryanrasmussen 6y ago
      maybe a quick mvp would be to detect smudged images, and separate those out for manual handling, and the non smudged automatically rasterized.
- 6y ago
hantusk 6y ago
I would go for decrappify: https://www.fast.ai/2019/05/03/decrappify/ https://www.fast.ai/2019/05/03/decrappify/ GAN's are hard to train for a new ML practitioner, VAE can have blurry results if you are not using skip-connections like in U-NET. It sounds like you have training examples (images pre and post processing), but would benefit from an approach using transfer learning since you don't have that many. Good luck!
- ampdepolymerase 6y ago
  How do skip links help with blurriness? I thought they are mainly to prevent disappearing gradients?

tomrod 6y ago

ML practitioner here. Step 1: write out the steps you do to manually clean up. Step 2: Depending on: (1) how many samples you have, (2) what is defined as "before" and "after", (3) what performance level is acceptable, there are many algorithms to choose from. Transform learning may be the target, reinforcement learning or CNN as well. It really depends on Step (1) and (2.1).

doctoboggan 6y ago
For a given poor quality print I usually take it on my iPad and retrace either the whole print or just a portion. I usually just use the pencil tool in white and run it between the ridges of the print to give good separation and definition to the final engraving. Depending on the quality of the print (some of them are hardly more defined that a low res ink smudge) I may have to use a good amount of artistic liberties on where I think the pattern was going. Using my knowledge of what a print may look like I can usually do a decent job. Thats why I thought this might be a good task for a ML algorithm since I can show it what a bad print looks like and what a decent cleaned up version of that print should look like.
- tomrod 6y ago
  So you're denoising by identifying where the patterns should be, but there is some smudge? If so, you may be able to skip ML altogether and use a deterministic software program to classify light versus dark areas (what you're doing with your pencil). ML might help with mapping smudges to the right shape/sizing. That said, if you have a ton of examples, then yeah an image recognition task might help.
- tomrod 6y ago
  Also: https://towardsdatascience.com/fingerprint-denoising-and-inpainting-using-fully-convolutional-networks-e24714c3233 https://towardsdatascience.com/fingerprint-denoising-and-inp...
  - doctoboggan 6y ago
    Wow, this is basically exactly what I needed, thanks! Seems like the key word I was missing in my searches was "inpainting"
    - tomrod 6y ago
      Glad to hear it.
      - doctoboggan 6y ago
        I got their model running with the included training data and it’s quite impressive. Some of my more extreme poor examples don’t produce reasonable results but many do.
        tomrod 6y ago
        That's awesome! Happy scaling!

neolog 6y ago

Sounds like you want to convert raster images to vector. https://vectormagic.com/ https://vectormagic.com/ is a commercial version.

doctoboggan 6y ago
Thats not actually my problem. For a good quality print I can do this with no issue. (I use imagetracer.js in my web app) but the problem I have is many prints that come through smudged or otherwise damaged. In those cases I have to retrace them, sometimes taking a decent amount of artistic license using what I know about how other prints looks.
- ramraj07 6y ago
  How many samples do you have of smudged prints that have been corrected? Can you also artificially smudge prints to create a training set? Then an interesting solution might be possible!
  - doctoboggan 6y ago
    I currently only have about 100 fixed up image sets. One of the other commenters pointed me to a blog that mentioned this software for generating realistic finger print images: https://dsl.cds.iisc.ac.in/projects/Anguli/ https://dsl.cds.iisc.ac.in/projects/Anguli/ which could then be artificially smudged. There seems to be a dataset out there somewhat that has already done this (https://competitions.codalab.org/competitions/18426 https://competitions.codalab.org/competitions/18426) but I can't find out where to download it.
    - ramraj07 6y ago
      I would advise against completely synthetic prints, you want to model your training data as close to reality as possible! Can't you just blur other finger prints you have?

syntaxing 6y ago

Depends on what you are cleaning. I have a hunch you wouldn't even need anything ML related. Just the right computer vision pipeline using OpenCV should get you really close. If you really want to use ML/DL because its cool and fun, I'm pretty sure Autoencoders will do what you need [1]. If you're lucky, the denoising autoencoder should do the clean up automatically. Alternative is to use a VAE which goes really far compared to GAN since its much more sampling efficient and easier to train. I think(?) it's what was used in the original deepfake. [1] https://www.mygreatlearning.com/blog/autoencoder/ https://www.mygreatlearning.com/blog/autoencoder/

tomrod 6y ago
You're right on the VAE. Depending on output quality, my experience has been fewer samples are needed for VAE training.
teruakohatu 6y ago
I agree. For cleaning up a fingerprint I would think you could get good enough using Opencv. People have been cleanly vectoring raster images long before deep learning. If you can automate it using a combination of Gimp filters + Inkscape, you probably can do it pretty easily convert that process into code.
- doctoboggan 6y ago
  I can and do simply rasterize images for some of my clean and proper fingerprints. The problem I am having is with smudges, incomplete prints, or other artifacts. In those cases I trace over and fix the print sometimes taking a decent amount of artistic license depending on the underlying quality of the image.
  - bryanrasmussen 6y ago
    maybe a quick mvp would be to detect smudged images, and separate those out for manual handling, and the non smudged automatically rasterized.
6y ago

hantusk 6y ago

I would go for decrappify: https://www.fast.ai/2019/05/03/decrappify/ https://www.fast.ai/2019/05/03/decrappify/ GAN's are hard to train for a new ML practitioner, VAE can have blurry results if you are not using skip-connections like in U-NET. It sounds like you have training examples (images pre and post processing), but would benefit from an approach using transfer learning since you don't have that many. Good luck!

ampdepolymerase 6y ago
How do skip links help with blurriness? I thought they are mainly to prevent disappearing gradients?