8 ms·

As a non-ML practitioner who has an interest in the field, I have a question I was hoping someone could answer. Looking at the page this links to, it gives pre

As a non-ML practitioner who has an interest in the field, I have a question I was hoping someone could answer. Looking at the page this links to, it gives pretrained models for "common use cases", among other things. Can you actually do anything interesting or novel with those? I feel like any meaningful new use of ML will not be possible with such "beginner building blocks" and you would just wind up with a weaker version of existing offerings. Could someone give me any examples of meaningful ways this can be used?

sk0g 6y ago
Re-training! The generalised models are... Generalised. They have learned to pick up details from a vast, varied data set. You can reuse that, and fine tune to the model to your specific data set, which could save you days worth of computation, and also decreases the data set size requirement on your end.
memexy 6y ago
There is a process called transfer learning where you take a model that has been trained on a large data set and then modify the last few layers of the neural network by training them on your specific/smaller data set. Not sure if there are any good resources for beginners but I learned about the technique from fast.ai videos.
ml_basics 6y ago
Here is an example thing I made using one of the pretrained mdoels (MediaPipe Facemesh): https://github.com/paruby/snake-face/ https://github.com/paruby/snake-face/ I'm sure that many people would not class this as interesting or novel, but it demonstrates that at least some of the building blocks can actually be used to make actual things.
pabloem 6y ago
I don't think pretrained models are "beginner building blocks". They provide a kind of "base model" that can be fine tuned to your specific need. The advantage of not training the full model means you can save many resources (in design time and computation). There are many boring but meaningful tasks in which this can be used. For example, I'm sure many industries could be benefit from image classification for very specific cases (e.g., fruit categorization). In those cases, you are not interested in the classification of "general" objects such as car, person, bike or horse (as provided by MobiletNet pretrained model), but you can use that model as base to classify different categories of fruit. Therefore, you are right that they might not be very useful to build new ML algorithms or network architectures. But they are useful to build specific (and novel) uses of current neural networks.
fxtentacle 6y ago
You are correct that all commoditized AI currently cannot do anything completely novel. They ship a network to convert landscape photos to landscape paintings, for example. You can retrain that with limited effort and knowledge by using their Collab notebook. That way, you can make an AI to convert photos of cats to photos of dogs. But you wouldn't be able to create an AI to convert between cats and rabbits, for example, because those are too different for that AI architecture to tolerate. Similarly, they ship a classifier to detect objects of some categories based on surface texture. You can retrain that to detect different categories of things. But if someone paints a chair to look like elephant skin, the AI will be confused and short of inventing a new architecture yourself, there's nothing you can do to fix it. Plus in any case, you will need to have large high quality datasets for your retraining. The biggest cost in building a bird classification app isn't the one ML employee that creates the final AI, it's hiring biologists to correctly tag thousands of photos to train the AI on. That's why many AI research nowadays tries to offload the data generation to random people on the internet. For example, recaptcha, or crawling Flickr for that faces dataset. Edit: since it was brought up in the comments, the retraining that I mentioned can be done only on parts of the model, which is then called transfer learning and can save you some computing time. But, you'll need to determine if the first feature layers of the original model are suitable for your new classification task. Even experts sometimes guess wrongly here, so as a beginner, I'd say you can only pray and hope.
- dheera 6y ago
  "But you wouldn't be able to create an AI to convert between cats and rabbits" I reckon you probably actually could with some of the more recent results.
  - fxtentacle 6y ago
    I am not aware of any model on the TensorFlow Hub which could do that. I also believe that I haven't read a paper with a suitable architecture yet. Which one are you referring to?
    - jaredtn 6y ago
nl 6y ago
Sure. The innovation often comes from how you use the model, not trying to build new ones (although that happens too). They have pose detection: https://github.com/tensorflow/tfjs-models/tree/master/posenet https://github.com/tensorflow/tfjs-models/tree/master/posene.... One day during a talk with someone who works in aged care I worked up a demo of fall detection using this.
- memexy 6y ago
  That sounds pretty interesting. Do you have notes you can share?
- mncharity 6y ago
  And some years back I use posenet in node.js, just for its head tracking, while exploring 3D UIs. Because it was fairly robust, at least as long as an eye was visible. Even now, it degrades more gracefully with heads turned away than some face nets. Though its positions are a bit noisy, which hurt multi-camera-parallax 3D pose estimation.
darepublic 6y ago
There is a difference between consuming models and training them yourself -- I could be wrong but with tensorflow.js I think it is much more geared toward consuming models in the browser, rather than training novel ones yourself. Seems like you can customize an existing model via transfer learning, which may or may not be good enough for your usecase, but even then I feel like basically the training side of it is second rate. But you could potentially make _predictions_ with a really sophisticated/cutting edge model in JS -- its not the prediction part that takes enormous compute, it is the training part
- memexy 6y ago
  You can also build and train models in js. This is one of the features I'm looking at. Currently there is a lot of information websites are throwing away or they're using centralized aggregation mechanisms to provide personalization but Tensorflow.js changes that dynamic. I think small AI models that run completely in the browser and provide personalization by learning from how the user interacts with a given website are the future. This empowers the user and puts them in charge of how their data is used. The example I mentioned previously to demonstrate this was about ranking new submissions on HN: https://news.ycombinator.com/item?id=23407549 https://news.ycombinator.com/item?id=23407549. I'll quote the relevant part > TensorFlow.js is a pretty nifty piece of software and it's underutilized. If the model parameters can be stored in IndexedDB then users could train TensorFlow.js based site augmentation to suit their own needs. For example, what if HN had a TensorFlow.js model for ranking new submissions based on the user's preferences? This model could be trained like a spam filter and would eventually learn the types of articles that someone likes to see but they would be in charge of the model's evolution and so would be empowered to use it however they saw fit. Maybe I don't care about politics then my model parameters would eventually converge on downgrading all political posts and the more technical submissions would rise to the top based on how I upvoted new and front page submissions.
  - darepublic 6y ago
    Cool idea, but why not have that client side model operate across websites -- if I switch from hacker news to reddit why not carry over that data to figure out what I will like on the separate platform?
whymauri 6y ago
It depends on your definition of "novel." I've seen some of these base models for tfjs used in several neat hackathon projects ranging from music generating web apps to "hot dog, not a hot dog." I've seen tfjs run simple RL game agents in-browser, too. The models are re-trained, tweaked, or slightly augmented - but not too much, since you only have 24 hours.

sk0g 6y ago

Re-training! The generalised models are... Generalised. They have learned to pick up details from a vast, varied data set. You can reuse that, and fine tune to the model to your specific data set, which could save you days worth of computation, and also decreases the data set size requirement on your end.

memexy 6y ago

There is a process called transfer learning where you take a model that has been trained on a large data set and then modify the last few layers of the neural network by training them on your specific/smaller data set. Not sure if there are any good resources for beginners but I learned about the technique from fast.ai videos.

ml_basics 6y ago

Here is an example thing I made using one of the pretrained mdoels (MediaPipe Facemesh): https://github.com/paruby/snake-face/ https://github.com/paruby/snake-face/ I'm sure that many people would not class this as interesting or novel, but it demonstrates that at least some of the building blocks can actually be used to make actual things.

pabloem 6y ago

I don't think pretrained models are "beginner building blocks". They provide a kind of "base model" that can be fine tuned to your specific need. The advantage of not training the full model means you can save many resources (in design time and computation). There are many boring but meaningful tasks in which this can be used. For example, I'm sure many industries could be benefit from image classification for very specific cases (e.g., fruit categorization). In those cases, you are not interested in the classification of "general" objects such as car, person, bike or horse (as provided by MobiletNet pretrained model), but you can use that model as base to classify different categories of fruit. Therefore, you are right that they might not be very useful to build new ML algorithms or network architectures. But they are useful to build specific (and novel) uses of current neural networks.

fxtentacle 6y ago

You are correct that all commoditized AI currently cannot do anything completely novel. They ship a network to convert landscape photos to landscape paintings, for example. You can retrain that with limited effort and knowledge by using their Collab notebook. That way, you can make an AI to convert photos of cats to photos of dogs. But you wouldn't be able to create an AI to convert between cats and rabbits, for example, because those are too different for that AI architecture to tolerate. Similarly, they ship a classifier to detect objects of some categories based on surface texture. You can retrain that to detect different categories of things. But if someone paints a chair to look like elephant skin, the AI will be confused and short of inventing a new architecture yourself, there's nothing you can do to fix it. Plus in any case, you will need to have large high quality datasets for your retraining. The biggest cost in building a bird classification app isn't the one ML employee that creates the final AI, it's hiring biologists to correctly tag thousands of photos to train the AI on. That's why many AI research nowadays tries to offload the data generation to random people on the internet. For example, recaptcha, or crawling Flickr for that faces dataset. Edit: since it was brought up in the comments, the retraining that I mentioned can be done only on parts of the model, which is then called transfer learning and can save you some computing time. But, you'll need to determine if the first feature layers of the original model are suitable for your new classification task. Even experts sometimes guess wrongly here, so as a beginner, I'd say you can only pray and hope.

dheera 6y ago
"But you wouldn't be able to create an AI to convert between cats and rabbits" I reckon you probably actually could with some of the more recent results.
- fxtentacle 6y ago
  I am not aware of any model on the TensorFlow Hub which could do that. I also believe that I haven't read a paper with a suitable architecture yet. Which one are you referring to?
  - jaredtn 6y ago

nl 6y ago

Sure. The innovation often comes from how you use the model, not trying to build new ones (although that happens too). They have pose detection: https://github.com/tensorflow/tfjs-models/tree/master/posenet https://github.com/tensorflow/tfjs-models/tree/master/posene.... One day during a talk with someone who works in aged care I worked up a demo of fall detection using this.

memexy 6y ago
That sounds pretty interesting. Do you have notes you can share?
mncharity 6y ago
And some years back I use posenet in node.js, just for its head tracking, while exploring 3D UIs. Because it was fairly robust, at least as long as an eye was visible. Even now, it degrades more gracefully with heads turned away than some face nets. Though its positions are a bit noisy, which hurt multi-camera-parallax 3D pose estimation.

darepublic 6y ago

There is a difference between consuming models and training them yourself -- I could be wrong but with tensorflow.js I think it is much more geared toward consuming models in the browser, rather than training novel ones yourself. Seems like you can customize an existing model via transfer learning, which may or may not be good enough for your usecase, but even then I feel like basically the training side of it is second rate. But you could potentially make _predictions_ with a really sophisticated/cutting edge model in JS -- its not the prediction part that takes enormous compute, it is the training part

memexy 6y ago
You can also build and train models in js. This is one of the features I'm looking at. Currently there is a lot of information websites are throwing away or they're using centralized aggregation mechanisms to provide personalization but Tensorflow.js changes that dynamic. I think small AI models that run completely in the browser and provide personalization by learning from how the user interacts with a given website are the future. This empowers the user and puts them in charge of how their data is used. The example I mentioned previously to demonstrate this was about ranking new submissions on HN: https://news.ycombinator.com/item?id=23407549 https://news.ycombinator.com/item?id=23407549. I'll quote the relevant part > TensorFlow.js is a pretty nifty piece of software and it's underutilized. If the model parameters can be stored in IndexedDB then users could train TensorFlow.js based site augmentation to suit their own needs. For example, what if HN had a TensorFlow.js model for ranking new submissions based on the user's preferences? This model could be trained like a spam filter and would eventually learn the types of articles that someone likes to see but they would be in charge of the model's evolution and so would be empowered to use it however they saw fit. Maybe I don't care about politics then my model parameters would eventually converge on downgrading all political posts and the more technical submissions would rise to the top based on how I upvoted new and front page submissions.
- darepublic 6y ago
  Cool idea, but why not have that client side model operate across websites -- if I switch from hacker news to reddit why not carry over that data to figure out what I will like on the separate platform?

whymauri 6y ago

It depends on your definition of "novel." I've seen some of these base models for tfjs used in several neat hackathon projects ranging from music generating web apps to "hot dog, not a hot dog." I've seen tfjs run simple RL game agents in-browser, too. The models are re-trained, tweaked, or slightly augmented - but not too much, since you only have 24 hours.