12 ms·
Teachable Machine: Teach a machine using your camera, live in the browser
- lelima 9y agoYou can solve problems using Machine learning without coding from a while ago.. azure machine learning have this features from more than a year ago. I've solve regression, classification and recommendation problems with it and the best part is it deploys an web service with a few clicks.
- thanksgiving 9y agoBut you would need to have: 1 a working phone 2 a valid credit card To use azure which places a too high bar on students. I mean I've tried to argue for graduated restrictions so basically students with .edu emails should be able to do some things without entering a credit card number but the fact that it is not possible suggests this isn't a priority for azure. Google says this finds on your browser so there's little infrastructure cost for this demo, right?
- jlian 9y agoThere is a student offer that doesn’t require credit card https://azure.microsoft.com/en-us/offers/ms-azr-0144p/ https://azure.microsoft.com/en-us/offers/ms-azr-0144p/
- marksomnian 9y agoBut IIRC it doesn't include ML tools.
- shostack 9y agoCan you clarify on what you did with it? I'd love to start dabbling in solving problems with ML, but am a bit intimidated by getting started. Is it fairly easy for a novice to do the things you did?
- celim307 9y agoPretty neat! Good overview without overwhelming right off the bat. Would be cool if they showed off common pitfalls like over fitting, or even segued into general statistics!
- amelius 9y agoI don't have a camera here. Did anyone try it? How does it work?
- gfredtech 9y agothere are 3 default classes, so you train according to each class(e.g. hand waving, sitting still, etc) you take examples of each(using your camera). you map the input data from your camera to some output data(e.g. if i used the green button to take photos of me waving), display a GIF of a cat that's waving. instead of a GIF you can use sound too
- IanCal 9y agoSurprisingly well! It's a really well put together demo & tutorial. I held a pen up next to me and held the green button. Then did the same with a mouse. It would flick between the two if I was holding nothing, so I held the orange button for a bit while holding nothing. Worked pretty much every time. Training is fast enough with a few hundred images per class that I didn't notice any delay.
- amelius 9y agoWhat do you mean exactly by "held the green button"? I can't run the demo here (browser not capable enough, and no camera) and I'm getting really curious what this is about.
- icc97 9y agoWatch the demo video in the link, it explains the green button
- amelius 9y agoThanks!
- IanCal 9y agoAh sorry. So there are three coloured buttons. When you hold one, the site takes a series of photos from your webcam, and assign them to that "class". Then it'll train and start classifying your video input live. It's a pretty neat way of creating a reasonable training set of 3 classes.
- StavrosK 9y agoDoes anyone know what this uses under the hood? I loved the demo, but I would like a similarly easy way to get started locally with Python, for example. Is there an ML library that can easily start capturing images from the webcam so you can play around with training a model?
- hnarayanan 9y agoYes, there is! http://www.wekinator.org http://www.wekinator.org (It is not a Python application but a Java application, but still as fun!)
- StavrosK 9y ago> a Java application, but still as fun! Your unbridled optimism even in the face of reality has inspired me to give this a shot, thank you! It seems to be based on Weka, so it should be good. On a first run, I don't really see how to record images from the webcam, it just says "waiting for samples". I'll play around some more and hopefully figure it out, thanks again. EDIT: Ah, there's a detailed walkthrough which seems to work well!
- icc97 9y agoIt says it in the video: https://deeplearnjs.org https://deeplearnjs.org
- greggman 9y agobe aware, at least in Chrome, once you give teachablemachine.withgoogle.com permission to use you camera, unless you revoke that permission is has permission to use your camera without further permission including from iframes. In other words every ad from and analytics from Google could start injecting camera access. I wish chrome would give the option to only give permission "this time" and I wish it didn't allow camera access from cross domain iframes.
- jamesmishra 9y agoIf this happened, the Google Chrome tab would show a camera. Many webcams have adjacent LEDs that identify that they are activated. Google could theoretically release compromised versions of Google Chrome and only use the permission on devices where webcam LEDs are unlikely (e.g. smartphones), but this is going deep into tin-foil-hat territory.
- greggman 9y agothat's not helpful. the pictures would already be taken and uploaded to servers without my permission reguardless of whether or not I wanted my picture taken or what's visible (contracts, trade secrets, people in various states of undress) . also this isn't about Google spying. it's about Chrome's bad camera permission model. any company can abuse it
- addedlovely 9y agoGood to know, but thankfully easy to remove permissions from the settings.
- azinman2 9y agoBut won’t it be on just that FQDN alone? Google analytics and ads are served from a totally different domain. What’s the actual concern here?
- amigoingtodie 9y agoSo you are contending we are secure via DNS?
- deleted 9y ago[deleted]
- peepopeep 9y agoAm I the only paranoid one who thinks this is just Google's way of capturing millions of faces in their database? Or did Apple beat them to it?
- danso 9y agoWhat did Apple beat them to? FaceID is said to not upload data off-device.
- xyrnoble 9y agoFacebook beat them to it... that's the whole reason for tagged images imo. Then they can relate identities with each other and with exif gps data to track their movements over time.
- colmvp 9y agoYeah it's a little hilarious how people just keep giving Facebook more and more data to experiment with.
- melling 9y agoHN discussions tend to devolve into rants about privacy. There are a lot of repeated discussions that occur here. They overwhelm the discussion about the actual technology https://h4labs.wordpress.com/2017/09/27/groundhog-day-amazon-updates-echosomeone-rants-about-privacy/ https://h4labs.wordpress.com/2017/09/27/groundhog-day-amazon...
- gtufano 9y agoThat's because the privacy implication of the technology should be part of the discussions on the technology... technology is not neutral, the way its used and the privacy implications are significant.
- xyrnoble 9y agoI can solve the privacy problem by not using their products? I disagree: https://en.m.wikipedia.org/wiki/FBI–King_suicide_letter https://en.m.wikipedia.org/wiki/FBI–King_suicide_letter Also, my own personal privacy is less secure if it's a relative inconvenience for employers. If everyone but me gives up their privacy then there's more pressure on me to follow suit. The argument even doubles back on itself. If these comments aren't interesting to you... don't read them. Embrace tree-style collapsible comments.
- mschuster91 9y agoHmm. I wonder if one could train this with dick pics and embed into popular messenger apps client-side... "this picture was classified as a penis", to counter morons sending their dick as first message.
- melling 9y agoHow long before I can teach my computer gestures that are mapped to real computer functions? For example, scroll up/down, switch apps, save document, cut/copy/paste, etc. One could probably map each gesture to a regular USB device that acts as a second keyboard and mouse? The hard part is identifying enough unique gestures?
- wildebaard 9y agoYou mean a device such as the Leap Motion Controller? http://store-eur.leapmotion.com/products/leap-motion-controller http://store-eur.leapmotion.com/products/leap-motion-control... They seem to offer a VR headset add-on these days, but I've only ever seen the 'basic' controller in action, which worked okay-ish.
- melling 9y agoI think that’s infrared but the same idea. That never quite worked. Also, Leap didn’t continue to refine their hardware for consumers. They have next generation hardware that’s going directly into VR headsets, but you can’t buy it. https://www.engadget.com/2014/08/28/leap-motion-s-next-sensor-is-designed-specifically-for-virtual-r/ https://www.engadget.com/2014/08/28/leap-motion-s-next-senso...
- wildebaard 9y agoSeems to me that you can buy the SDK[0] (which is not much more than a Controller and a bracket to hold in against your VR headset of choice), so at least they've made some progress since 2014. [0]: http://store-eur.leapmotion.com/products/universal-vr-developer-bundle http://store-eur.leapmotion.com/products/universal-vr-develo...
- BatFastard 9y agoUsing the Leap Motion on an Oculus is magic. Much better than hand held controllers. When I use it I don't get sick from VR.
- 9y ago
- f00_ 9y agothis is really cool, openframeworks-esque in browser javascript if you like this I would highly recommend looking at openframeworks. the interactive browser part excites me want to try to make something with deeplearn.js
- eggie5 9y agoi bet it's fine tuning an ImageNet CNN
- netcraft 9y agoWhat makes it non-mobile? Is it something about the expected performance of the JS? or are there apis being used im not thinking about?
- nsthorat 9y agoIt works on mobile, it's just slow. Every time we read and write from memory we have to pack and unpack 32 bit floats as 4 bytes without bit shifting operators >.>
- white-flame 9y agoIsn't that what ArrayBuffers can do for you at nearly the same amortized speed as C unions?
- nsthorat 9y agodeeplearn.js author here... We do not send any webcam / audio data back to a server, all of the computation is totally client side. The storage API requests are just downloading weights of a pretrained model. We're thinking about releasing a blog post explaining the technical details of this project, would people be interested?
- amelius 9y agoYes please! :) And some quick questions: What network topology do you use, and on what model is it based (e.g. "inception")? What kind of data have you used to pretrain the model?
- nsthorat 9y agoWe're using SqueezeNet (https://github.com/DeepScale/SqueezeNet https://github.com/DeepScale/SqueezeNet), which is similar to Inception (trained on the same ImageNet dataset) but is much smaller - 5MB instead of inception's 100MB - and inference is much much quicker. The application takes webcam frames and infers through SqueezeNet, producing a 1000D logits vector for each frame. These can be thought of as unnormalized probabilities for each of ImageNet's 1000 classes. During the collection phase, we collect these vectors for each class in browser memory, and during inference we pass the frame through SqueezeNet and do k-nearest neighbors to find the class with the most similar logits vector. KNN is quick because we vectorize it as one large matrix multiplication. I'll go deeper in a blog post soon :)
- amelius 9y agoInteresting! I'm curious why you've used a different classification algorithm on top of a neural network. I would expect that a neural network on top of a pretrained network could give similar results, with the benefit of simpler code. Is performance the reason? Anyway, I'm looking forward to your blog post.
- nsthorat 9y agoTraining a neural network on top would require a "proper" training phase, and finding the right hyperparameters that work everywhere turned out to be tricky. Actually, this is what we did originally, in the blog post we'll try to show demos of each of the approaches and explain why they don't work. KNN also makes training "instant", and the code much much simpler.
- crypticlizard 9y agoThe value-add for this demo is amazing, it's going to be many people's first approachable experience of ml, or things just like this will be. I expect a lot more of this stuff to appear in UI/UX. It's fun, intuitive, and a game changer away from dumb screens to fully interactive machines with their own knowledge graph.
- irascible 9y agoThe paranoia in this thread is crazy. It's client side, people, and if google wanted to steal your precious fluids, they wouldn't need this to do it.