9 ms·
Cloud Video Intelligence API
- sna1l 10y agoI wonder if Snapchat is/will become a large user of this service? Depending on the average response time of this API, Snapchat could get much better ad targeting analyzing their Stories content. I imagine that they have something similar in house that they run since it is pretty vital to their core business, but you never know.
- HappyTypist 10y agoSnapchat is big enough to do it in house without paying Google to donate their data for their NN. For volume consumers, Google should be paying Snapchat to learn from their data.
- wastedhours 10y agoThey have just committed to spending $1bn on GCP though, so not unrealistic that they'd leverage some of the other suite of tools.
- discordance 10y agoIt's actually $2bn [1], $400 million per year over 5 years. 1: http://www.recode.net/2017/2/2/14492026/snap-ipo-2-billion-contract-with-google-cloud http://www.recode.net/2017/2/2/14492026/snap-ipo-2-billion-c...
- ganfortran 10y agoThis API is not cheap
- Jach 10y agoHow long until someone runs an attack like https://arxiv.org/abs/1609.02943 https://arxiv.org/abs/1609.02943 and provides the model for free?
- nl 10y agoThis field moves quickly. Embedding Watermarks into Deep Neural Networks https://arxiv.org/abs/1701.04082 https://arxiv.org/abs/1701.04082
- jonknee 10y agoSnapchat is required to spend buckets of money with Google...
- kneel 10y agoThey're more than likely on this already. Any photo you save in the app is already categorized by content.
- skewart 10y agoI'm curious about how much use these general-purpose computer vision APIs are actually getting. How many companies out there really want to sift through a lot of photos to find ones that contain "sailboat"? I'm inclined to think a lot more companies would want to find "one of these five different specific kinds of sailboats performing this action", which is definitely not among the tens of thousands of predefined labels that Google, and Amazon, offer with their general purpose models. High-quality custom model training as a service seems much more compelling.
- timc3 10y agoIt depends on the data set and the images in question. Of course taking as an example for a sport’s clothing company where their digital assets are mostly related to their products a general purpose API might not get the subtle differences between two similar shoes, or clothing lines from different seasons. But it might be enough to help catalog that one image or video has a sports person in, and the other is a fashion shoot or product shot.
- djloche 10y agoone immediate need is NSFW flagging, esp. things that might indicate abuse.
- danso 10y agoThis is a feature that Microsoft's computer vision API offers (in contrast to AWS Rekognition and other services): https://www.microsoft.com/cognitive-services/en-us/computer-vision-api https://www.microsoft.com/cognitive-services/en-us/computer-...
- discordance 10y agoContent moderator is what you're looking for: https://www.microsoft.com/cognitive-services/en-us/content-moderator https://www.microsoft.com/cognitive-services/en-us/content-m...
- jpatokal 10y ago
- timc3 10y agoI have been on the beta program for this and generally the results in our testing have been very good. I particularly like how granular the data can get.
- komali2 10y agoWhat do you guys plan to do with it? Another poster mentioned how it seems hard to imagine a business that has a model around "finding the sailboats in this batch of pics."
- timc3 10y agoWe build systems that let our customer manage their video, and usually in large quantities. Some of this video spans a very long period of time and the only description that they have of it is in the filename. Getting proper structured metadata from content has traditionally been expensive as it has required humans, sometimes trained as librarians, so providing the ability to extract some meaning from video becomes valuable. Even for systems that have trained librarians, it can still help to have the ability to have a system that highlights the general content so they can further refine it and bring it inline with a taxonomy. Google Vision API also helps because the metadata can be placed against timecode, further helping the ability to search for particular subjects within a video that might otherwise not be found easily from quickly browsing the videos. There are other use cases depending on the customers need (such as a customer making sure that certain things are not in video about to go to air), but none of it has to do with marketing.
- ASpring 10y agoThe business prop is easy and already hinted at by another user here. People post millions of hours of themselves in natural settings on Snapchat. If you can recognize their settings (objects, environments) and cluster/categorize users then you can target advertising even more intrusively than Google et al already do.
- aub3bhat 10y agoI think there is a need for a comprehensive system for image and video data analytics. Much like how we today have relational databases (postgres, MYSQL) and full text search engines (lucene/Solr). The approach Google or Amazon have been taking which involves providing a "tagging" API is frankly unimaginative. I am working on Deep Video Analytics an Open Source Visual Search and Analytics platform for images and videos. The goal of Deep Video analytics is to become a quickly customizable platform for developing visual & video analytics applications, while benefiting from seamless integration with state or the art models released by the vision research community. Its currently in very active development but still well tested and usable without having to write any code. https://github.com/AKSHAYUBHAT/DeepVideoAnalytics https://github.com/AKSHAYUBHAT/DeepVideoAnalytics https://deepvideoanalytics.com https://deepvideoanalytics.com
- timc3 10y agoI would be interested to know more about this, particularly the database and what you plan to do with it in the future (I am thinking the license on the GitHub project is obviously restrictive for a purpose at the moment).
- aub3bhat 10y agoSorry about the license, I am trying to reach a beta version within a month along with a system-description paper that outlines the long term vision behind building such a system. At that point I plan on relaxing the license. There are certain constraints such as making sure that all underlying models are correctly licensed. Also FAISS which I use is licensed by Facebook under an explicit non-commercial license.
- bitmapbrother 10y agoIt was really entertaining listening to Fei-Fei Lee talk about AI and ML at Google Cloud. If you get the chance check it out on YouTube. I especially liked how she referred to video as once being the "dark matter" of vision AI.
- icy_as_blue 10y agoWould you mind share a youtube link?
- davidcgl 10y agoDirect link to her talk: https://youtu.be/j_K1YoMHpbk?t=2h38m40s https://youtu.be/j_K1YoMHpbk?t=2h38m40s
- wyc 10y agoI think the most commercially successful application of computer vision has been quality-control devices (citation needed). Agriculture is very interested in CV for a return-optimization technique known as precision farming. Manufacturers pay for inspection of production throughout the pipeline. To predict where a mass-market CV could be successful, I think we should look for industries with similar problems but cannot currently afford a bespoke custom modeling solution.
- madenine 10y agoI feel like the places where CV always proves most useful are places where either we need more eyes than would be financially viable, or we need eyes in places its hard/unsafe/expensive to have them. The tipping point on training+pay for a human vs an ML/CV system for a lot of tasks is coming down. I saw a Microsoft spotlight last fall on a company that's using drones + CV to inspect power infrastructure in Scandinavia. As much as the tech to do that has cost, its probably cheaper than sending out helicopters with specialists all the time to check power line towers for wear/damage.
- nostrademons 10y agoOr we need eyes that can drive a physical response faster & more accurately than a human can react. This is the whole robotics/drone/hardware market. Right now, most of these efforts work at the same speed humans work (probably because we still need to monitor them to make sure they're doing the right thing). But imagine self-driving cars with no traffic lights, because the cars can react & communicate fast enough to avoid a collision without any macro-scale signaling. Manufacturing processes where molten metal is shaped directly into whatever shape desired, where computers do real-time calculations of the effect of gravity & cooling rate on the position of the material. Airliners that never land and can take on passengers anywhere, because passengers ascend in a personal drone that mates with the mothership under computer control. Unfortunately CVaaS isn't really suitable for this, because the network delays transmitting the data to a datacenter will outweigh any speed benefits of computerizing it. But I could see a big market for services that let you train a model in the cloud at high-speed, and then download the computed model for execution on a local GPU or TPU.
- soared 10y agoSounds similar to a company I worked with that took security camera footage from restaurants and identified employee theft and process inefficiencies.
- ar15saveslives 10y agoCorrect me if I'm wrong, but this is just a frame-by-frame labeling. You can download whatever pre-trained CNN, pass individual frames through it and get the same result.
- skewart 10y agoTrue. But then you have to deploy and maintain that CNN yourself. The value prop is similar to, say, Twilio. Though, arguably, it's easier to run your own pre-trained CNN than it is to replicate the telephony, VoIP, and video conferencing stuff Twilio provides. Also, presumably Google is hoping that they can continue to train and improve their CNN so that it's always just a little better than the best free-to-download ones.
- ar15saveslives 10y agoYes, but I mean if you analyze video as individual frames, you can't declare "we do video analysis", because video is completely different domain. There're papers like [1] where CNN output is used as input for RNN, which performs deeper context analysis. Results aren't exciting, though. [1] http://cs231n.stanford.edu/reports2016/221_Report.pdf http://cs231n.stanford.edu/reports2016/221_Report.pdf
- robotresearcher 10y agoI don't know if they are doing it right now in this API, but Google and plenty of others have demonstrated recognition of actions in videos. I expect they'll add it later if it's not there right now. At the very least in this release they are identifying scene changes, which is a dynamic property.
- pgodzin 10y agoIt also mentions verbs like swimming so it may do those across several frames? Not sure how much more accurate that is than frame-by-frame
- ar15saveslives 10y ago
- hartator 10y agoIt's awesome, but I can't really see any application beside content filtering and supericial content classification.
- timc3 10y agoIt can help more than superficially, it can become a starting point for humans to go and refine classification into a controlled taxonomy. The Google implementation can also detect when different people are speaking which is useful - though it takes someone to tag who is who. As I mentioned in another post, it can also highlight where things are happening in a video - for instance a 2hr video where a scene suddenly appears. Like in a nature video rushes where an animal appears for only a few seconds. Other types of video analysis can also detect problems in the video/audio, such as dropped frames, noise or colour gamut issues.
- zitterbewegung 10y agoNot the first https://clarifai.com https://clarifai.com has a similar service .
- chimtim 10y agowhat is the "video" bit here? This is just running image recognition on a bunch of frames.
- catshirt 10y agohow do you know the implementation details? it would be completely naive to implement it that way, considering there is an entirely new attribute video applies over images which of course is "time". I don't know shit about ML- talking out of my ass here- but I'd be surprised if the algorithms didn't account for changes over time or canonical entity recognition (is this the same boat that was in the last image)?
- chimtim 10y agoThe linked press release shows an animal is detected -- tiger etc. It does not say tiger running or hunting, which is where the time component would have been used.
- timc3 10y agoI have seen it detect that a car is drifting..
- catshirt 10y agothe press release says: > nouns such as “dog,” “flower” or “human” or verbs such as “run,” “swim" or “fly” that out of the way... i suspect you wouldn't need video to detect those things... and the screenshot you're referring to is an specific application of the API... not a kitchen sink: > It can even provide contextual understanding of when those entities appear; for example, searching for “Tiger” would find all precise shots containing tigers across a video collection in Google Cloud Storage.
- ramramanathan 10y agoWe are talking about our underlying tech at Next conference - https://goo.gl/3ihXth https://goo.gl/3ihXth We are clearly using frame level annotations, but we also have additional models to aggregate visual and additional information to provide aggregate level entities at the shot level or video level. PM at Google
- imh 10y agoThe demo picture they chose is interesting. It's obviously a tiger, and is identified as such with only 90% probability. I appreciate the difficulty of the problem and how big of a success it is to achieve even that level of confidence, but that low level of confidence really shows how far we are from being able to simply trust computer vision. Still useful from an information retrieval perspective, I expect.
- muzakthings 10y agoYou realize that softmax scores aren't probabilities, right? It's just a relative measure of confidence, scaled such that they all sum to 1.0.
- chipperyman573 10y agoYou can't add any of the numbers in the picture to equal 1.0 (or 100)
- visarga 10y agoWhat you need to do is to take the top prediction and see how accurate it is compared to a test set. The scores on the picture represent confidence not accuracy.
- imh 10y agoThat's kinda true, but (regularization aside) for standard loss functions it's minimized at the point it's well calibrated, right? Given the scores in the image (97% animal, 90% tiger, etc) they seem to be binary classifiers e.g. "is this a tiger?" So of all scores in the neighborhood of 90%, 90% should be "yes it is," making it a measure of confidence compatible with probability. Please someone correct me if I'm wrong, but I'm pretty sure that's how it works, just like how logistic regression gives you a probability.
- kneel 10y agoCronenberg inception porn is coming
- frakkingcylons 10y agoAs a Cloud Prediction API user, it makes me a bit uneasy to see it left out of the image of their product suite. Is it effectively in maintenance mode now? I feel like TensorFlow is overkill for what I need and my use case doesn't fit into image/speech/video detection.
- torechudi 10y agoLogan full movie online free https://www.linkedin.com/pulse/logan-full-hd-movie-2017-online-putlocker-lee-bease https://www.linkedin.com/pulse/logan-full-hd-movie-2017-onli...
- jrullman 10y agoI've never seen spam like this on HN before...
- tambourine_man 10y agoIt amazes me how smart these guys at google are, and yet, they can't design a mobile site if their lives depended on it: http://imgur.com/bXGuNfL http://imgur.com/bXGuNfL
- chrisweekly 10y ago^ lol, too true
- cobookman 10y agoIf you could share with me what mobile phone / web browser you used that produced the styling issue, I'll be sure to pass it on to the relavent people within google so that it gets resolved. Also if you send me an email at bookman@google.com I'll be sure to update you as to when the styling errors are resolved. (Disclaimer, I work for google cloud)
- tambourine_man 10y agoiPhone SE, iOS 10.2.1 Also, see my previous comment on a similar issue with Google Cloud Calculator: https://news.ycombinator.com/item?id=13729466 https://news.ycombinator.com/item?id=13729466
- cobookman 10y agothanks. Submitted your issue. Along with your google cloud calculator issue. Feel free to email me if you'd like to be cc'd on the status.
- cobookman 10y agoA code change was just pushed out to fix this css styling bug, and is live in production. Sorry for the inconvenience, and please don't hesitate to forward any other bugs you find to me at bookman@google.com
- tyre 10y agoI think their model should take a second pass on the words and probabilities, independent of the video. Look at their example: Animal: 97.76% Tiger: 90.11% Terrestrial animal: 68.17% So we are 90% sure it is a tiger but only 68% sure it is a land animal? I don't think that makes sense. It could be that this is a weakness of seeding AI data with human inputs. I can believe that 90% of people who saw the video would agree that it is a tiger, while fewer would agree it is a terrestrial animal, because they don't know what terrestrial means.
- chrisballinger 10y agoPerhaps it's confused by the many images and videos of tigers swimming in water?
- kevin488 10y agoThanks a lot Life of Pi
- crack-the-code 10y agoIt's probably more likely that they want each output to be independent of the other. Certain features may be predominantly associated with a tiger, but not necessarily indicative of a terrestrial animal. If the 9.89% chance that they could have been wrong would have been the case, then that should not influence whether or not it was a terrestrial animal. In my opinion, the consumer of the output values should be able to rely on these fields independently, and make these associations themselves. Although I totally agree a second pass could be useful as a separate data set.
- njohnson41 10y agoStill, in any consistent way of assigning probabilities to events, if A implies B, then P(A) <= P(B). Neural network outputs are not probabilities. I think that's the main lesson here.
- thomasdullien 10y ago
- joaoaccarvalho 10y agoWhen you use these Google APIs, can Google keep/ use your data in any way?
- BrandonY 10y agoHere's a link to the Google Cloud Terms of Service: https://cloud.google.com/terms/ https://cloud.google.com/terms/ (you may be interested in section 5.2: Use of Customer Data). Disclaimer: I work for Google but am definitely not a lawyer and can't authoritatively speak for Google here.
- CRUDmeariver 10y agoIs there any storage-related cost (i.e. retreival or egress cost) when you call this on a file stored on Google Cloud Storage?
- vaiski 10y agoThere's alternative out there from a company called Valossa. More comprehensive than what Google is now offering. Https://val.ai
- mikeflynn 10y agoI've seen the Valossa offering and it is indeed impressive. Insane amounts of visual data on videos.
- jimmcslim 10y agoI wonder if you could use this to upload recordings from your DVR and have it determine the likely timecode of commercial breaks...