7 ms·
Meta open-sources multisensory AI model that combines six types of data
- seanlinehan 4y agoLink to the ImageBind site: https://imagebind.metademolab.com/ https://imagebind.metademolab.com/
- mkaic 4y agoAdditionally, here's a link the blog post about it, which I feel might actually be a better place for this post to link to instead of Verge's coverage of it: https://ai.facebook.com/blog/imagebind-six-modalities-binding-ai/ https://ai.facebook.com/blog/imagebind-six-modalities-bindin...
- deleted 4y ago[deleted]
- soulofmischief 4y agoAudio2image looks insane. Can't wait to see how this tech develops over time.
- derefr 4y agoAre multi-format embeddings — like this one, or simpler ones like text-to-image — inherently bijective? I.e. could a model like this be configured to use any of its trained sense data types as either an input or output? Or is there some bias in the way the training data is ingested that makes the model better at transforming in certain directions?
- tkellogg 4y agoLike laser eyes?
- jacobr1 4y agoI haven't yet read this paper, but other multimodal models have different layers for mapping the input/output of each modality to the common embedding space. So, to simplify, there is some high-dimensional coordinate that the audio sample for the morphemes for "frog," the text word "frog" and maybe the same in multiple languages like, "rana" - they all basically have the same point in the abstract feature space. The additional layers of the model then provide "translation"
- alhirzel 4y agoHas anyone found info on what the hardware requirements are to run this?
- capableweb 4y agoFor training, it probably looks very different, but the usage example in https://github.com/facebookresearch/ImageBind#usage https://github.com/facebookresearch/ImageBind#usage seems to use just one cuda device, so you might be able to run it on a consumer GPU
- bhoops 4y agoIt seems to me that we put this inside a boston dynamics robot and we have Terminator
- WinLychee 4y agoIn the quest to placate shareholders, one of the failing FAANGs will make a panicked announcement they have partnered with Boston Dynamics, to ensure AI is democratized and equitable for all (technically true when we're all on the run).
- renewiltord 4y agoWell, it's unlikely to be Google. They've sold them off once already.
- gremlinsinc 4y agonothing says equality when terminators are annihilating the human race, non-discriminatory.
- nprateem 4y agoHa ha they're trying to revive the metaverse with the one killer app that'll save it: immersive AI porn. Probably a good time to buy some meta shares in case they "pull it off"
- deleted 4y ago[deleted]
- throwuwu 4y agoI dunno, everyone pivoted to AI because it’s clearly a massive force multiplier but they’re all bringing those gains back to whatever they were working on previously. I’ve already seen some great uses of AI generated voices, scripts, textures, and sky spheres in VR. AI is also a huge resource for learning how to do things you’ve never attempted before. Shit’s going to be popping off all over the place this year and I don’t think it’ll stop.
- gloryjulio 4y agoIt's a bit late for that. They already almost tripled from the low point
- omeze 4y agoI see this sentiment of “the metaverse is dead” very often on HN but VR usage is at an all time high and meta has the leading hardware. Genuinely wonder: Why do you think the metaverse is dead, and by metaverse do you mean “horizon worlds”, VR, “metaverse apps” or something else?
- Auracle 4y agoI’ve seen many, many people mistake Horizon worlds for the metaverse. I guess it’s Meta’s fault for not being more clear, but I doubt those deriding the tech have actually seen anything directly from them anyways.
- nprateem 4y agoBecause Meta have had to drastically reduce the amount they're investing in their metaverse because they didn't have the user figures to reassure investors it wasn't just a waste of billions of dollars.
- digitcatphd 4y agoIt seems like this is a substantial step in Yann LeCun's approach to AGI https://www.shaped.ai/blog/yann-lecun-a-path-towards-autonomous-machine-intelligence https://www.shaped.ai/blog/yann-lecun-a-path-towards-autonom...
- flangola7 4y agoMaybe, I take anything LeCun says with many grains of salt. Much like Elon Musk, any time he says something that appears to make sense I find myself asking "what don't I know enough about, to know why he's wrong?"
- seydor 4y ago> text, audio, visual, movement, thermal, and depth. > It’s only a research project Nice try , zuckercop. Facebook policing and terminating real people is truly their natural evolution.
- deleted 4y ago[deleted]
- thrtythreeforty 4y agoOnce again, Facebook tries to have their cake and eat it too. This model is not open source, despite the claim. https://github.com/facebookresearch/ImageBind/blob/main/LICENSE https://github.com/facebookresearch/ImageBind/blob/main/LICE... The source is available to view and use under a non-commercial license. It's extremely frustrating that major AI players are trying to shoehorn a redefinition of a term that's well-understood, just to further their reputation.
- squokko 4y ago"Open source" is not "free software." This is absolutely open source and will enable a lot of academic, governmental, and nonprofit uses. Just because it doesn't allow someone else to make money from it doesn't make it not open source.
- thrtythreeforty 4y agohttps://opensource.org/osd/ https://opensource.org/osd/, specifically: > 6. No Discrimination Against Fields of Endeavor > The license must not restrict anyone from making use of the program in a specific field of endeavor. For example, it may not restrict the program from being used in a business, or from being used for genetic research. It's true that people are starting to make a distinction between "free software" and "open source," but generally the distinction is smaller than what you're saying. The broader class of programs whose code is available for viewing and (potential) use or modification subject to restrictions, has been called "source available," and I think that's a great description of what it is and sets expectations well. You may or may not have a license to use it, but you can see the source code. For example, the Business Source License is generally not considered open-source. It places restrictions on what you can do. I kind of hate being pedantic about this, but I also dislike the deliberate misuse of the term, especially by a company that is obviously quite familiar with publishing large amounts of high-quality software that's genuinely open source.
- mindcrime 4y agoJust because it doesn't allow someone else to make money from it doesn't make it not open source. Actually it does, per the OSD[1]. [1]: https://opensource.org/osd/ https://opensource.org/osd/
- revskill 4y agoThey should rename Meta to MeetAI now.
- sroussey 4y agoMetAI
- gumballindie 4y ago> Those opposed to open-sourcing, like OpenAI, say the practice is harmful to creators because rivals can copy their work OpenAI complaining about people copying their work is hilariously hypocritical.
- FeepingCreature 4y agoDid they actually say this?
- ajayyy 4y agohttps://cdn.openai.com/papers/gpt-4.pdf https://cdn.openai.com/papers/gpt-4.pdf > Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar. They did say "competitive landscape"
- yreg 4y agoIf that's it, then it's quite different to Verge's misquote imo.
- swyx 4y ago> Those opposed to open-sourcing, like OpenAI, so we're just saying the quiet part out loud now huh
- gumballindie 4y agoWhich is the quiet part?
- ChatGTP 4y agoTotally, I think open ai believes they will build an AGI and rule the world with importance before everyone realised they just steal and compress everyone’s stuff and sell it via hosted database.
- transformi 4y agoAny thoughts about possible applications of this model? Why do they made this source available?
- fnordpiglet 4y agoPrimary source HN discussion : https://news.ycombinator.com/item?id=35875700 https://news.ycombinator.com/item?id=35875700