31 ms·
Apple Vision predictive bio-feedback
- kasperni 3y ago> So, a user is in a mixed reality or virtual reality experience, and AI models are trying to predict if you are feeling curious, mind wandering, scared, paying attention, remembering a past experience, or some other cognitive state. And these may be inferred through measurements like eye tracking, electrical activity in the brain, heart beats and rhythms, muscle activity, blood density in the brain, blood pressure, skin conductance etc. I don't really know how I feel about this...
- stby 3y agoThis quickly becomes creepy beyond my wildest imaginations. Kill it with fire, please.
- ChildOfChaos 3y agoWhy? The more personal a device is and the closer it becomes to interfacing with who we are, the more it will need to do things like this to work, else it will just feel off.
- seydor 3y agoYou have emotions , the device does not. No matter how we pretend to ignore it, the ability to empathize and embody the emotions of others separates us from machines. Even if they emulate it, i know they are fake (yes, i am being a machine-racist here)
- TeMPOraL 3y agoThere's the good ol' bait-and-switch of tech industry you have to consider. New tech is promoted by emphasizing (and sometimes overstating) the humane aspects, the hypothetical applications for the benefit of the user and the society at large. In reality, these capabilities turn out to be mediocre, and those humane applications never manifest - there is no business in them. We always end up with a shitty version that's mostly useful for the most motivated players: the ones with money, that will use those tools to make even more money off the users. It's true with all new tech these days, and I'd be doubly worried when it comes to things like emotion detection. There are no good uses with strong business cases, but plenty of nefarious ones.
- KennyBlanken 3y agoPC makers selling TPM to everyone: "We'll make your computer more secure!" Reality: "We'll use it to store media encryption keys, so hollywood will let us stream media to your device."
- fsflover 3y agoI’m using TPM with free software and my own keys. It works fine.
- Jevon23 3y agoWhy do I want a device to be “personal” and “interface with who I am”?
- lrvick 3y agoExcept that device is not owned by you, or even accountable to the public. It is owned by a private corporation whose mission is to do whatever it takes to keep you making purchases to make stock price go up. I do not even carry a smartphone anymore and am happier for it.
- renegat0x0 3y agoin 2013 Edward showed that data are used by governments. We do not see it right now, but it does not mean it does not happen. Actually I think we do not see the scale of it, because most of us are not doing anything incorrect. It is only important for our politicians. Everybody is under surveillance. It is just we do not know the effects of it.
- croes 3y agoThat's the kind of personal data that shouldn't be on a device with an internet connection.
- polishdude20 3y agoYes if it was a standalone device which no outside party had control over then sure. But that's not practically what any tech device has been in the last 10 years. Everything phones home. And with the amount of valuable personal data this thing could accumulate, you can be sure it will be phoning home about it all the time.
- m4rtink 3y agoFully user controlled device running only free software ? Sure, agree totally! A totally closed hardware & software combination tethered to a corporate cloud ? NO WAY IN HELL!!
- piyh 3y agoGive it 30 years for the open source option to catch up.
- astrange 3y agoOne thing that's notable about devices that "only run free software" is that they actually don't, since people use web browsers on them, and the websites are not free software. You'd also have to never talk to another person with them, since other people are capable of manipulating your emotional state and can't be audited.
- thih9 3y agoThere is more: > Other tricks to infer cognitive state involved quickly flashing visuals or sounds to a user in ways they may not perceive, and then measuring their reaction to it.
- hdjjhhvvhga 3y agoAnd still, there will be people who will happily pay for it.
- thih9 3y agoNo guarantee that any of these will ever become part of the product though. The post is talking about past research work and about patents; Apple has lots of unused patents.
- makeitdouble 3y ago> I don't really know how I feel about this... But the headset of course does ! Otherwise, it could be interesting if as users we got access to all these measurements, for our personal use. That could open the door to many other applications that would be probably less creepy.
- viraptor 3y ago> could be interesting if as users we got access to all these measurements You can! It's not trivial, but not very hard either. eye tracking - OS projects getting this from a laptop camera exist and work well enough for assistive tech (i.e. it can replace your mouse today) electrical activity in the brain - there are fancy things like https://zeto-inc.com/device/ https://zeto-inc.com/device/ but there are basic options as well heart beats and rhythms, muscle activity, blood pressure, skin conductance - there's lots of cheap sensors for those
- OJFord 3y agoI assume GP means that the measurements would be exposed (at some helpful level of abstraction, plus also the raw data say) by the Apple Vision.
- viraptor 3y agoI know ; ) I wanted to show there are already alternatives for whose who are interested in experimenting with that data themselves (and at a fraction of the cost)
- seydor 3y agoI d think those are detectable from facial expression alone. There is also EEG as a possibility
- rmm 3y agoI wonder how much they can plug into the new “autocorrect” system in iOS? Ok user heart rate is low and at baseline, they probably were trying to say “I could give a flying duck”… Pupils dilated, heart rate racing, therefore “duck” may not be the right word..
- masklinn 3y agoGiven Siri’s reaction when it fucks up for the 5th time in a row and you tell it to fuck off… I would not bet on it.
- purple_ferret 3y agoThink of the possibilities. -A VR show can modify the jokes in real time if they know how funny you find them. -A VR casino can offer a 'full biometrics' table for the players who really think they know how to bluff -A VR porno can have your VR GF climax at the same time as you
- steve_adams_86 3y ago> VR GF This seems like an oxymoron. I get that it's a "virtual" girl friend which semantically qualifies the term girl friend, yet there's something immensely sad about the combination of these terms. VR sex bot seems more apt to me. It doesn't really matter and I don't mean to impose terminology on anyone; it's just an observation/reaction.
- manquer 3y agoSex bot implies sex in some form, that I don’t think is primary value of a such system even if it all available. The ability to form a connection and understand your needs perfectly and respond to that a companion is the real value Sex gets boring pretty fast . It is not that expensive to pay for in real life, a companion would be supportive, knowledgeable, empathetic and always available is very hard to pay and get or have otherwise . I don’t know why people find it sad , not everyone is able to find the perfect (or any) human partner /companion , we should be happy if we are all are able to connect to sentience does it matter if it is a human , dog or a machine ?
- steve_adams_86 3y agoI'm not opposed to it under all circumstances. What worries me is that people will do it under too many circumstances and fail to engage with the real thing. That seems like a tremendous loss of social opportunity which I'd argue is already occurring with technology today.
- croes 3y agoThink of the possibilities - they can track what you fear - they can track what you like - they can track what enrages you Agencies will love that data.
- reportgunner 3y agoReminds me of dreaming.
- fsloth 3y agoThe perfect Skinner box.
- makeitdouble 3y agoThis whole tweet had the same tone as when pop science crime investigation TV explains a criminal can be identified through its gait alone. It's always magic, except when the predictive model doesn't work and there is no corrective action possible on the user side. We're talking about physical reactions, so that means it's supposed to be the same through gender, age, race, health conditions from the algorithm's point of view...That's a lot of variables, and I wonder if Apple have analyzed this beyond the US population, assuming the device will be sold internationally at some point. Apple seems really confident about this as they are foregoing controllers on the default interfaces, but that also reminds me on how they were rumored to make a car without a driving wheel...fingers crossed, I guess.
- hdjjhhvvhga 3y agoAs usual, people will try to use workarounds. For missing ports on my Mac Book Pro, I use a hub. For connecting my AKG headset to my iPhone, I use a dongle. I guess sooner or later Apple will make available a kind of API for device makers to make it possible to interact with the content in ways different that eyeballs tracking.
- makeitdouble 3y agoYou're right. The same way face unlock doesn't work for some people and they inout a password every time, so it's not intractable. I expect people for who eye tracking fails to be stuck on third party controllers for the default interface (hopefully they're supported outside of games ?). The bummer though will be that this device is US only for its launch, so the first round of application might also not care at all about these kind of issues. Accessibility is something Apple usually cares about, but third party devs can be more casual about it.
- matt-attack 3y agoWell Apple is only expecting less than 1M units to be sold. Must such a niche product be designed for every variation of human? Seems reasonable that if you fall outside of the most common human forms by enough standard deviations then you're just out of luck.
- thih9 3y agoIt looks like this is mostly about patents. Apple has a history of doing lots of hardware and software research, not all of that gets to be part of an actual release. Is anything indicating that solutions mentioned in the post would actually become part of Apple Vision?
- Traubenfuchs 3y agoI wonder if him not having worked on Apple Vision for the last >1.5 years means they distanced themselves from his "vision, strategy and direction" or even gave up on it.
- justinator 3y agoJudging from his other tweets, he seems really into crypto and making art projects out of crypto tech. I can't say I really understand the art - and I went to school with him, had the same professors, etc. I'm just scratching my head when I read his tweets. But well: he's worth many millions and I still have housemates so I guess capitalism has crown him in the lead.
- sterlingcrispin 3y agobro I am not worth millions I'd delete my twitter account and go enjoy my life if I was
- deleted 3y ago[deleted]
- justinator 3y agoAlso he keeps telling companies to reach out to him about this tech so he can make that sweet sweet consultant money, so I'm guessing he's still onboard.
- graypegg 3y agoI would assume he’s advertising his own consulting availability as “an ex apple employee” by the verbiage he’s using.
- justinator 3y agoI think it's not only because he's an ex-apple employee, but an ex-apple employee that's worked on the patented tech on this specific product.
- kossTKR 3y agoI can't really grasp how enormous this project is. 5000 patents filed from Apple is crazy, and looking at the thing with all of these new both pieces of hardware and concepts, wow. How many people have been working on this, and for how long? The other threads here seems to have missed just how "huge" a projects this is. Is this also the lead up to the Apple car, or some other ecosystem?
- rmm 3y agoExactly. I know many people are upset that this will lead to patent soup resulting in less innovation or competition, which may very well be the case, but it shows also how much they haven’t just put high res glasses and a fast chip in a headset, but really tried to push the limits and leap past everyone with their first offering
- Dalewyn 3y ago>The other threads here seems to have missed just how "huge" a projects this is. The other threads are busy jumping on the Hate Apple bandwagon, is what it is. I have my own criticism of Apple's VR goggle, specifically that its form factor (albeit necessitated by today's technologies) drags the whole thing down in both the luxury/fashion and usage factors. But if I put that aside, the technology and innovative thoughts crammed into that thing is potentially groundbreakingly amazing if the marketing is truthful and accurate. Even the $3500 price tag, which yes is bloody expensive, doesn't sound unreasonable given most other "cheaper" VR goggles still require a mothership costing about the same anyway. I wish Apple success here, because the VR industry needs someone to come in and kickstart it back to something interesting and worthy of attention. ...I also admit I wouldn't mind seeing all the haters getting put out to dry for their noisy trouble, but maybe that's beside the point. :V
- lrvick 3y agoMost of us Apple Haters are not opposed to this type of technology existing. We are opposed to technology with this much potential for abuse being solely controlled by a profit maximizing patent troll. When the tech is open source and I can control what it does, then sign me up.
- seydor 3y agoI know we all hate each other but is this like a good direction for the future? Directing all our emotions to silicon chips? Maybe there is other things we should be building. Emotion recognition is an old field but (apart from autism research) the only application i can think of is emotion manipulation. How does one wake up in the morning and decide to work on consumer emotion recognition
- adinisom 3y agoPerhaps rubber hose resistance: "I'm sorry Seydor, it appears you're being coerced to unlock this data."
- throwaway290 3y agoWhich is especially fun if Seydor weren't coerced and does not truly help solve the rubber hose factor... Once rubber hose is in play this only means more suffering
- FabHK 3y agoBeatings will continue until your emotions improve.
- seydor 3y agoSiri, i want to sin
- steve_adams_86 3y ago> How does one wake up in the morning and decide to work on consumer emotion recognition I think two significant factors would be 1) they have the skills to make progress in the field and it's highly rewarded, and 2) they believe there could be real value in making progress. One or both of those could be enough to motivate a lot of people.
- mhh__ 3y agoValve have been interested into the biological side of VR and BCI for a while too
- seba_dos1 3y agoApparently they've been working on their own standalone headset (codenamed Deckard), and judging from how Valve Index and Steam Deck were done I'd say there's some significant chance they may get it right.
- noindiecred 3y agoOh so now the hot new thing is a computing device that actively tries to guess its user’s emotional state (using various methods, including introducing new stimuli) and then then CHANGES ITS BEHAVIOR based on the results! The Free Software people were right - a device this intimate needs to have publicly available, verifiable source code.
- NLPaep 3y agoThat’s Left 4 Dead’s “director”
- alpaca128 3y agoI already hate software that tries to guess my intentions from key inputs, now we're going to have the same "smart" behaviour expanded to my brain? I hope this can be turned off.
- sangnoir 3y ago> Oh so now the hot new thing is a computing device that actively tries to guess its user’s emotional state It also induces different states by way of imperceptible stimuli and measured the response. The tweet got increasingly dystopian the more I read; spine-chilling stuff. Had this been about the Quest, people in this thread would be reasonably apoplectic. Imagine what Saudi Arabia or a Gambling or News app could do with this. Fuck it, I'm going Thiel, anyone want to cofound a security startup, HMU (law enforcement experience advantageous) - we'll make the Apple Vision the best interrogation tool on earth. Fifth amendment? Pssh, the suspect won't need to say a single thing.
- ribosometronome 3y ago>It also induces different states by way of imperceptible stimuli and measured the response Are you sure? Am I missing that part? I see this: >Other tricks to infer cognitive state involved quickly flashing visuals or sounds to a user in ways they may not perceive, and then measuring their reaction to it. But that doesn't say they're inducing states but rather trying to figure out the state. Trying to understand if someone is mad does not seem quite as dystopian as trying to make someone mad.
- lrvick 3y agoI own 0 Apple products and will never give them money, but this post really helped reaffirm my desire to not have Apple products anywhere near me.
- dangus 3y agoThis post: “I’m a mercenary helping to make the thought police a reality, hire me.”
- vezuchyy 3y agoWait, how did I miss that it is now possible to tweet full-size essays?
- strictnein 3y agoI think it's a Twitter Blue feature.
- BasedAnon 3y agoRight... so this thing hooked up to a centralized AI, deciding what people see when, how often they see it, what they can express and how they feel... Do people really want to live in the matrix? I have a visceral reaction to this idea
- seba_dos1 3y agoThis could be exciting if the user was in control (FLOSS). The user is not, so it's nightmarish instead.
- mynameisvlad 3y agoWho said it’s hooked up to a centralized AI? Apple’s buzzword of the day yesterday was “on-device processing” and “on-device X” in general. They have invested heavily for years now into shipping ML chips in their devices and don’t seem to be stopping; if anything they’re increasing the use of it. Is there anything definitively claiming this would be any different?
- BasedAnon 3y ago> Who said it’s hooked up to a centralized AI? I'm talking about the infrastructure not this device necessarily
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- jodrellblank 3y agohttps://www.apple.com/newsroom/2023/06/introducing-apple-vision-pro/ https://www.apple.com/newsroom/2023/06/introducing-apple-vis... says "Where a user looks stays private while navigating Apple Vision Pro, and eye tracking information is not shared with Apple, third-party apps, or websites. Additionally, data from the camera and other sensors is processed at the system level, so individual apps do not need to see a user’s surroundings to enable spatial experiences."
- justinator 3y agoI went to art school with this guy.
- ugh123 3y agonext level humble-bragging. in post and his replies
- ripper1138 3y agoTotally. "10% of my life" sounds a lot more significant than "3.5 years"
- Dudester230602 3y ago"I was nearby when cool things happened, hire me now, need to pay Cupertino mortgage rate."
- astrange 3y agoI'm not sure anyone under the age of 70 has a Cupertino mortgage.
- fragmede 3y agoMaybe not on a $3 million 5-bedroom mansion in the hills, but with $500,000 1 and 2-bedroom condos, 20% down payment is $100,000 k, which is affordable by a high level engineer or product designer making $500,000/yr, especially if it's a power DINK couple making close to $1mm/yr. Whether they want to invest in housing there is another story, but let's not pretend the upper class is priced out of Cupertino. Eg https://www.zillow.com/homedetails/20488-Stevens-Creek-Blvd-APT-1113-Cupertino-CA-95014/64542887_zpid/ https://www.zillow.com/homedetails/20488-Stevens-Creek-Blvd-... recently sold.
- astrange 3y agoThere are not many of those. (Not many condos in the US in general for that matter, since they're pretty discouraged by housing laws.) Wonder how many 10s-100s of bids there were for it. Anyway, I'm not accusing Cupertino residents of being upper class, I'm accusing them of being old.
- vincnetas 3y ago"All of these details are publicly available in patents, and were carefully written to not leak anything." This part makes me sad about patents and original idea behind them. This actually is the opposite of original idea why people came up with the idea of patenting something. To spread the knowledge and to award inventors. Not to stifle the competition.
- wil421 3y agoIt’s not the patent that’s preventing him from speaking it’s the NDA he signed with Apple.
- FabHK 3y agoThe sad fact is not that the author cannot speak about the invention (because of the NDA), but that the patent was carefully written not to disclose anything substantial about the invention.
- MichaelZuo 3y agoWhy is that sad? Partial information seems better then remaining 100% a trade secret.
- vkou 3y agoNo, it's worse. The point of patent protection is that you teach everyone how to build the thing that's being protected, in exchange for a monopoly. Not actually teaching people how to build the thing that's being protected, but still having a monopoly on it is the worst of all worlds. If you aren't going to teach us how to build something, don't expect patent protection for it. I'd rather you keep it as a trade secret, good, go do it, it's a free country, but then we can just reverse engineer it. If I don't have an NDA with you, I have no reason to keep your trade secret... secret.
- MichaelZuo 3y ago
- DoreenMichele 3y agoSocial stuff is best approached from a statistical confidence model that allows for "This might be the one in ten times my assumption is wrong." I wonder if/how they incorporate that. Because a lot of social drama comes from humans betting on the statistically likely thing in social situations and it being wrong rather than checking or hedging their bet. ("I shall assume the white male is in charge. Oops. Now I've offended the woman or person of color who is actually in charge.")
- riffic 3y agotweet should have been a blog post.
- hoherd 3y agoAt least it wasn't a tweet thread!
- FabHK 3y agoOr a picture of a pdf.
- Brendinooo 3y agotweet is a blog post.
- gopuck16 3y agomore like a brag post
- renewiltord 3y agoThis is pretty cool. Eager to see what it feels like in reality. The risk with prediction is misprediction. Causes a lot of frustration.
- Bukhmanizer 3y agoThis is a pretty shockingly bad statement. Not only is it a complete exaggeration, it’s not even a good exaggeration. Saying “I worked on mind-reading features” for Apple’s new VR headset is just creepy.
- deleted 3y ago[deleted]
- nbzso 3y agoI really miss Steve Jobs.:)
- prashp 3y agoMaybe he was fired in 2021 for being too modest.
- joe5150 3y agoNot hard to imagine how tools to predict and manipulate your emotions might be useful in a device that even in the wildest dreams of its developers is mainly designed to provide an immersive environment for taking Teams calls and working on spreadsheets.
- thorum 3y agoPublic patents or not, can't imagine Apple is going to be thrilled with this guy for announcing to the world that their new product is secretly reading everyone's minds.
- sterlingcrispin 3y agoHey I appreciate the general lack of cruelty so far in the comments, sometimes HN can be pretty rough. The weird phrasing in the tweet is because I can't talk about any of the work I did due to NDA's besides my job description and those patents. But I had a pretty unusual role all things considered. I was also trying to clarify in the tweet that these aren't meant to be interpreted as unannounced existing features or capabilities of the product. But rather, research I did and contributed to, that was made public through patents. There's a ton of interesting preexisting research literature about neurotechnology research for VR. Both in terms of using it for biofeedback, and using it as a device just for studying the brain. If that sounds interesting browse through what's out there via google scholar. Also I said I spent 10% of my life working on it, realizing that was a goofy way of putting it. But it's a standout thing in my life and it was mostly a self reflection on what a trip being alive is, not trying to brag about it, mostly just sharing what I did. It's an unusual product and I had an unusual role Anyway it's a cool device and hopefully people enjoy it.
- ethanbond 3y agoCongrats on the launch! Seems like a really fun project to work on. And thanks for sharing what you can :)
- 9dev 3y agoI for one got the 10% phrase like you seem to have intended it, and it actually got me to think about how much time I spent in previous employments relative to the entire span of my life, which was an interesting venue to go down, so thank you! (Also for the insights on your work, that sounds like an immensely fascinating experience!)
- practice9 3y ago> how much time I spent in previous employments relative to the entire span of my life, which was an interesting venue to go down Same for me! Didn't even notice any weird phrasing (the tweet got immensely popular though, so nitpickers inevitably arrived). It was also nice to know people are working on getting non-invasive neurotech into the mainstream - very exciting
- lumb63 3y agoThis begs the question: what is Apple’s real play with Apple Vision? It sounds like there’s a real possibility that, even at a $3500 price point for hardware, the consumer is still the product. What else can their system determine about someone’s brain? What is going to be done with that data?
- ShamelessC 3y agoHopefully it at least means I can type out the “eyes rolling” emoji by simply rolling my eyes.
- KaiserPro 3y agoThe eye tracking required to get that level of accuracy is really high. Normally you need a bunch of camera like sensors that operate different to rolling/global shutter, plus a hefty calibration routine to get a good map of the eyeball. Even then it only really works looking dead ahead with your eye wide open. so for the moment, that stuff is firmly in the lab. So whats the real play? to replace phones with glasses that know you better than you know your self. To bake all that you see and do into <300 megs a day, and make it searchable in a way thats useful to you (not as in where was I at 22:04 on the 4th) as in "when did I last see x, what colour y did they say they liked?" but without having to ask that question. Imagine a really really good personal assistant, that could answer 99.5% of the boring questions asked you with authority. That is the level of AI they (and google and facebook) want
- astrange 3y agoNobody wants to do anything with anyone's data. The idea that your data is valuable is largely an excuse to feel self important. Facebook, for instance, makes money by showing you ads, not with "your data".
- euroderf 3y ago> Other tricks to infer cognitive state involved quickly flashing visuals or sounds to a user in ways they may not perceive, and then measuring their reaction to it. Is this like how in the old days the movie theater flashed a single film frame of BUY POPCORN YUM.
- exitb 3y agoSoo… is this real? Did those features really make it into the product as of now? I don’t recall any of these details being mentioned. > measurements like eye tracking, electrical activity in the brain, heart beats and rhythms, muscle activity, blood density in the brain, blood pressure, skin conductance etc. The headset does not appear to track most of these things.
- serf 3y agothis whole release feels extraordinarily 'Apple' , to me. Apple takes an idea and staples as many 'side-grade' style functionalities to it as possible in order to sell it , rather than just offering a really well-performing product within the niche. The iPhone was a remarkably bad phone -- it was more awkward to hold than any other phone at the time (now they're ALL awkward), and the reception was terrible. The provider it was locked to at the time was one of the worst in North America at launch; but the iPhone had features that people went nuts for which drove sales beyond the phone offering. I feel like this VR headset will be similar. It's a mobile device with a 2 hour battery life -- something that everyone in the world has been saying would make a device dead on arrival for the mobile market; but it has all of these little side-grade features that will attract purchasers that are interested in the Apple product itself but not necessarily VR. As a consumer i'm glad -- I may have never been a big iPhone fan, but the Android reactionary effort produced so much value that I can hope that this field will see something similar once it proves the market feasibility. I can't wait for my 8 hour/8k res/quarter-priced white-label VR headset clone in a few years.
- dahwolf 3y agoSo Apple can tell how you bio-physically respond to dynamic input. No worries though, we'll show a blend of an Apple logo and lock, which means all is fine. Go back to sleep, but we'll let you know if it was a proper sleep.
- kcrwfrd_ 3y agoI’m currently undergoing a course of neurofeedback therapy… the biofeedback tech in the Apple Vision sounds like it could have some pretty cool applications for such things.
- stevebmark 3y agoDemystifying this concept: The mechanism is almost certainly just measuring pupil dilation, and maybe eye focus, if that's something you can track with an eye facing camera. Source: Thinking Fast and Slow, where the initial inspiration for the book was an experiment noting that human's pupils reflexively dilate when they engage their "system two," aka the "slow brain," metaphorical part of their mind.
- MagicMoonlight 3y agoLmao that sales pitch trying desperately to get a job
- AlexanderTheGr8 3y agoSorry for hijacking this thread for a question about personal curiosity but does anyone know of an affordable way of measuring pupil distance? I want to measure my pupil distance while browsing social media. The reasoning is my reading in "Thinking Fast and Slow" that pupils dilate when we see something we find interesting / something we like / when we are thinking and vice-versa. I want to put it to the test and it seems like finally the consumer tech is close to making it possible. Does quest support pupil tracking? I did some cursory research but couldn't find any reference for it. Industry pupil tracking headsets are way too expensive; My last hope is that someone will jailbreak vision pro...