17 ms·
Turning a MacBook into a touchscreen with $1 of hardware (2018)
- spidermonkey23 6mo agoUsing an external webcam is that not more than $1? cool project though; reminds me of how you could use a Wii remote to create a interactive whiteboard.
- jbverschoor 6mo agoInternal with a small mirror
- als0 6mo agoStill an amazing hack today and I love it. However, I heard Apple are developing a touch screen MacBook this year, and I simply don't get why they're doing that. I don't know what's worse, the ergonomics or the fingerprints.
- bardackx 6mo agoi used to have a bad touch screen laptop like 7 years ago and back then it made the mobile development more pleasant (both native and web based)
- Etheryte 6mo agoSo you mean they're developing the iPad, an insanely popular device, and you're not sure why they would make such a device?
- nsxwolf 6mo agoI have been around touch screen Windows laptops for I don’t know how many years now, and I have never felt even the slightest compulsion to touch the screen.
- bigfishrunning 6mo agoIt might be a generational thing; my kids get touchscreen laptops from their school, and they interact with them almost exclusively by touching the screen. I agree, I'd much rather use a mouse (or even better, a trackball; i wish most laptops still had those)
- t1234s 6mo agoIs there a coating you can apply to the glass to help with smudge marks?
- rappatic 6mo agoOleophobic coating is standard on phones and tablets, which is part of why they don’t pick up fingerprints as easily. Some brands offer coating you can DIY yourself (eg ProofTech OLEOPEL) but these seem mostly designed for phone screens. I don’t know whether they’d be as effective on laptop screens
- dotancohen 6mo agoAnd this is why modern screens (and eyeglasses) should be cleaned with a damp microfiber cloth and no aggressive cleaners. These coatings are fragile. I do carefully clean the nosepads with soapy water, however.
- anandkulkarni 6mo agoBrilliant!
- solfox 6mo agoLove it! I appreciate the ethos of doing more with existing hardware. Adding an actual touchscreen would add real COGs to a macbook, and many potential failure points. Using the existing camera hardware + software seems to produce a "good enough" result for most people for casual use. I'm sure with some time and eng, Apple could make the "hack" shippable. But it doesn't earn product managers the big big bonuses, so it'll never happen.
- us321 6mo ago[dead]
- egypturnash 6mo agoI wonder how well this would work with my bright blue fingernails that are about .5" longer than my finger. I then wonder how much recalibration I would have to do when one of them broke and I was poking directly at the screen.
- Synthetic7346 6mo agoYou could still use the keyboard and track pad
- jojobas 6mo agoYou chose to suffer not just with touchscreens. That said, it would probably work just fine.
- egypturnash 6mo agoIt's honestly kind of amazing how many things seem to never be tested by anyone with long fingernails. My favorite was the time I picked up a video game controller and physically could not use it because the buttons were in a stylish little recess that meant my thumbnail would have to pass through the plastic housing before I could get the button to make contact.
- jojobas 6mo agoWhy would product designers make products worse for most due to self-inflicted conditions of the few?
- egypturnash 6mo agoYou do realize that a significant portion of the population is encouraged to grow their nails long, right? Women exist.
- phatskat 6mo agoEven as a non-woman I sometimes like to have my nails done up nice, and sure it’s my _choice_ just as it’s my choice not to purchase products that are fine excluding a chunk of the market. Out of curiosity, do you remember what controller? We are pretty much all DualSense in this house but I’d like to be aware in case we go shopping for new ones at any point
- brcmthrowaway 6mo agoChecking this profile of a random hacker in 2018, of course they are now working on AI.
- anishathalye 6mo agoI was working on AI in 2018 too :) At that time, I was quite interested in adversarial examples and ML security.
- ForOldHack 6mo agoI think I could do this for less than 15 cents: four small peices of double sided tape, and the tiny mirror, and two hair pins... but the software? Priceless.
- nothrowaways 6mo agoTouch screens are not pleasant for laptops. I prefer not to have them.
- gavinsyancey 6mo agoYou don't have to use it.
- nothrowaways 6mo agoyou will accidentally touch the screen more often that you think.
- criddell 6mo agoAs long as there's a way to maintain the current display density, that would be just fine. However, like on Windows, I suspect macOS would increase the tap target size on lots of the touchable elements. Even if I don't use the touchscreen, I would still have to pay the touch target real estate tax in my applications.
- hntway90904 6mo agoThat's a fair ask. My dream would be a simple toggle in something like control center for macOS that can flip between "touch mode" and "desktop mode" with most of the under the hood stuff being the same and just UI changes for the task. No doubt this would create new hurdles for software devs but again I'm dreaming here. Windows 10 actually had this with "tablet mode" in the notification center but I think they already soured people on the touch Windows thing by this point. I think Apple could reasonable do it better if they had the will but they'd much rather you buy and iPad for touch and a mac for desktop and everyone who doesn't want an extra device for certain use cases is left out in the rain.
- dataflow 6mo agoNote on Windows you can disable the touch device and it goes back to the old density. Don't know if Macs support it.
- 6mo ago
- dotBen 6mo agoI wouldn't want a touchscreen MBP even if it was free, anyone else feel similar? I don't get the draw - we already optimize for keyboard commands to avoid living our fingers over to a touchpad. Why would I want to start clicking on my screen? If you're using your computer for tasks (rather than entertainment) and you're not a visual designer, I don't get why Apple are apparently going to be putting them into the new MBP line later this year.
- eitally 6mo agoI very much would want a touchscreen for my use cases.
- post-it 6mo ago> I don't get why Apple are apparently going to be putting them into the new MBP line later this year. Apple has apparently being going to put a touchscreen in a laptop every year since the iPad came out, and it's never materialized.
- xmprt 6mo agoPreviously, those were rumors from enthusiasts who wanted to see it. Now it's an internal leak so there's a lot more credibility to those rumors.
- hu3 6mo agoNo, there were internal "leaks" in the past as well. https://archive.ph/xOgtp https://archive.ph/xOgtp
- jacobgkau 6mo ago> January 11, 2023 > Based on current internal deliberations, the company could launch its first touch-screen Mac in 2025 Even if it didn't come to pass, just a few years ago is a more relevant leak than the every-year-since-the-iPad-released "rumors."
- long-time-first 6mo agoThis is amazing. They should start to install upward looking cameras to implement this officially.
- JaredCampbell 6mo agoIt feels real because of the dirty touchscreen.
- parpfish 6mo agoI don’t want to encourage people touch my screen and smudge things up.
- rox_kd 6mo agoNeeeeveeeer!!! please let macbooks be as they are .. why would I ever choose to put fingers on that beautful screen ... I don't get it!
- bigyabai 6mo ago> why would I ever choose to put fingers on that beautful screen ... I don't get it! I'm not sure, but bare in mind that the iPad is almost as large of a market as the Mac at this point, and the iPhone has long surpassed Mac revenue. Touching your computer is a very popular sentiment among the grimy-handed public.
- platevoltage 6mo agoThere is a reason why the MacBook hasn't been replaced by an iPad with a keyboard and trackpad attached to it. Very different use case.
- hoherd 6mo agoYou're right. Those are completely separate millions of dollars to be made. Best not let them overlap at all, even if some of us intuitively touch our laptop screens to scroll a window now and then, especially when carrying it around.
- platevoltage 6mo agoThey overlap plenty where it makes sense. There was a time where Mac users were worried that Apple was going to merge MacOS and IpadOS. Nobody wants that.
- bigyabai 6mo agoWhat is that reason? Clearly Apple can put desktop chips in the iPad, their desktop apps already support ARM, and the iPad hardware can absolutely run macOS. What is the advantage of iPadOS, instantially?
- ianberdin 6mo agoI was laughing so much. Thank you. Unexpected tech!
- callamdelaney 6mo agoThe reason we buy macbooks is because they aren't touchscreens.
- mlvljr 6mo ago[dead]
- Jabrov 6mo agoWhat a super neat application of computer vision. Cool writeup. Thanks for sharing the code and making it open source too!
- joaohaas 6mo agoAs other people mentioned this is obviously not something I would want in my notebook... but I can still appreciate the cool tech! I can also definitely see this kind of thing being used in things budget outdoor displays, specially if the UI is made to accommodate the lack of accuracy, and the camera is positioned on the side (since these displays are usually vertical).
- starkparker 6mo agoDifficult to capture reflections across a large screen while also dealing with outdoor lighting, glare, and moisture. The touchscreen part isn't usually what makes outdoor signage expensive compared to IP65, temperature control, and a secure housing, all of which would still need to apply here. This looks like a neat option for retrofitting, and I suspect it'd work for some non-screen glass applications too. A combined IR/visible light solution would be interesting too, since I suspect those are complimentary (IR touch has issues with radiant light, while this wouldn't; this would have issues with low/no light, while IR wouldn't).
- wek 6mo agoThis is cool. Simple prototype. Is it dependent on lighting ... what if you are outside or backlit or glare etc...?
- coef2 6mo agoPeople use their laptops under various lighting conditions. I can imagine it would be difficult (or likely impossible) to bring this PoC to a solid production level technology. It looks like a fun project though.
- allenu 6mo agoThis project brings back memories. I worked somewhere over 20 years ago where we were working on something just like this (touch displays using cameras). The biggest challenge was definitely the lighting conditions as you mentioned. We tried to rely on natural light but it was too unreliable. Darker skin tones were harder to pick up, and then you had issues with random reflections, light and shadow being cast on the screen, etc., which would make the system detect spurious fingers and touches. We also had algorithms to detect finger shape to detect location of the pointer and when you were touching the screen. I saw way too many videos of fingers touching screens back then, so it's funny to see similar video clips here.
- coef2 6mo agoIt sounds like a fun project. I worked on a vision based biometric system that used near infrared light as its light source. NIR was supposed to be more stable than natural light but we still experienced issues similar to yours. We found that certain problems appeared at different times of day, and the system also struggled to handle the diversity of people.
- dhbradshaw 6mo agoDon't love touchscreens that much. But I did love my Toshiba Satellite. It was like writing on paper! Down with capacitive screens and long live Active Digitizers!
- wildrhythms 6mo ago"We've done tons of user testing on this, and it turns out it doesn't work. Touch surfaces don't want to be vertical. It gives great demo but after a short period of time, you start to fatigue and after an extended period of time, your arm wants to fall off. it doesn't work, it's ergonomically terrible." -Steve Jobs, 2010 https://www.businessinsider.com/steve-jobs-touch-screen-mac-2010-10 https://www.businessinsider.com/steve-jobs-touch-screen-mac-...
- trash_cat 6mo agoNobody forces you to use touchscreen exclusively?
- onion2k 6mo agoIf that's the case then the interface will remain a compromise that has to work for both point and touch, and ends up being suboptimal for both. Touch input necessitates bigger hit targets, lots of additional negative space around important buttons rather than groups, and a much lower information density on anything that has a click/touch handler like a list in order to avoid accidental presses. Apps need to be written with a touch UI in mind to work well.
- cosmic_cheese 6mo agoIt will likely compromise anti-glare performance too, since an oleophobic treatment will be required for the screen to not instantly be covered in a haze of fingerprints. For someone who has no use for touch this is a strict downgrade. The only way touch on MacBooks can make a shred of sense is if it’s a non-default option in the configurator, much like the current nanotexture matte option.
- ThoAppelsin 6mo agoThe line of thinking here applies to accessibility features as well. Luckily, it is uncool to say those things against the accessibility features, and there are even many good preemptive positive arguments, such as “a good design with accessibility in mind benefits everyone”. And those same arguments do apply for touch-enabled design as well; a good design with touch input in mind can benefit all.
- BugsJustFindMe 6mo agoIgnoring whether touchscreen laptops are actually a good idea, I "OOF"ed out loud at this line. > Filter for skin colors and binary threshold Skin has an extremely broad range of colors that are also lighting dependent. I'd have gone with background subtraction.
- zuhsetaqi 6mo agoI always say, people who want a touchscreen on their Laptop never used a really good trackpad. I never missed a touchscreen on my MacBook but when I do something on someone else’s Windows Laptop I often prefer to touch the screen because the trackpad is just terrible.
- deleted 6mo ago[deleted]
- DubOfWeek 6mo agoI don't even imagine how I would be tapping on the screen of my MacBook, not because of its form factor or design, but because of the macOS system itself. It's a different story with the iPad where you can do it endlessly...
- DeathArrow 6mo ago[dead]
- p0w3n3d 6mo agoI almost hear that screen cracking when accidentally closing macbook with the camera inside... I love Mac since I started using it in 2020, but boy this hardware is fragile. I am scared that I will be held accountable for fixing a broken screen of my work MBP
- voidUpdate 6mo ago> "Filter for skin colors and binary threshold" Which skin colours? The image below that has a lot of colours that I'd associate with darker skin colours, and they're not included in the triggering zone. I'd be interested to see some data on hoe well it works with someone who has dark skin
- vjay15 6mo agosooo clever
- lumpo 6mo ago[dead]
- Zobat 6mo agoMy previous work provided laptop had a touchscreen and I miss it (for the record, the screen didn't fold 180). It was useful about once a week and I completely forgot about it the rest of the time. Two primary use cases. Sitting on the train with the laptop in my actual lap it was often more convenient to reach for the screen instead of the trackpad, especially when I had someone sitting next to me on the right and I didn't want to stab them in their ribcage with my elbow so I could reach the trackpad. Second use case was often scrolling while reading, for some reason (phone-scrolling-indoctrination I guess) it felt natural to scroll using finger on screen. The screen was never my primary pointing device but it was always an option. I think it was annoying a handful of times during the two years I had it, you point at something on the screen and end up clicking something.
- 83457 6mo agoReminds me of Johnny Lee's Wii-mote projects... http://johnnylee.net/projects/wii/ http://johnnylee.net/projects/wii/