4 ms·
Attention is all you have
- Onavo 12d agoClick bait title :/ I thought it's a machine learning article
- kevinmgranger 12d agoWhat about the title implied that?
- blinkymach12 12d ago"Attention is all you need" is a famous ML paper. Though given the content of the article I don't think the author was intentionally referencing that.
- fragmede 12d agoWe could call doing such a thing, referencing a seminal work with a similar title that's so close that you couldn't to it that have missed it, yet somehow you did, as an AliceGG's Mistake. How could you write 600 words on the topic and not mention the paper otherwise?
- arrowleaf 12d agoWhy would AliceGG need to reference the paper? Anyone who is vaguely aware of the LLM architecture knows about the paper title and it being a huge advancement in machine learning. This is an article about recommendation algorithms hijacking our attention and LLMs ruining the internet... it's pretty clear the article title is a nod to the paper?
- mjmsmith 12d agoFor the same reason that you could tell a joke without feeling the need to explain it.
- deleted 12d ago[deleted]
- sublinear 12d agoGiven the tone of the blog otherwise, I think it was intended to both grab attention and subtly imply that AI is irrelevant with feigned ignorance.
- recursive 12d agoMachine learning makes the doom scroll more effective/worse. That's the connection I inferred.
- econ 12d agoThat is exactly what the article is about.
- Contortion 12d agoOn the other hand, the article was exactly what I expected from the title.
- jamesgill 12d ago"There’s one lesson the Tetris effect teaches us: whatever you focus on long enough will end up shaping your thoughts." Yes. And--change your thoughts, change your life: https://thinkhuman.com/change-your-thoughts-change-your-life/ https://thinkhuman.com/change-your-thoughts-change-your-life...
- arrowleaf 12d agoWhy the 'but'? The article is about becoming intentional in your thoughts and not letting them be hijacked by algorithms.
- alansaber 12d agoThis is why I like books. No attempt at attention hijacking. Wheras every other modern app tries to claw your eyes out.
- robertclaus 12d agoOur book club has actually noted that more and more mainstream books are written for low attention span. Repeating details ad nauseum and simplifying the plot significantly. So I don't think books are immune to this automatically.
- tombert 12d agoI am not sure that that's recent. Most books, even before the internet, were kind of low-effort shit. You've always had terrible borderline-porn romance novels at grocery stores and terrible pulp novels and the like. We just don't read terrible novels from 100 years ago; whatever we read from that era now has survived the test of time. The same cannot be said about anything released in the last couple years.
- deleted 12d ago[deleted]
- pixl97 12d agoAlso, quantity is a quality in and of itself. Grocery store softcore porn was still very expensive in time, money, and space. The amount of energy required to move a book around is monumental in comparison to how much energy it takes to move the same number of bits of information contained in the book. The speed at which just the information moves (in modern digital networks) is nearly instantaneous. The space the digital book carries is not measurable by the human senses. It was nearly impossible for most people to afford to fill every waking moment with a book in front of them, bar a few addicts. Now there is a river of information actively fighting for your eyeballs at every moment. And how the meme-gene works, the most addictive types will rise as the victor.
- popalchemist 12d ago
- rsinerate 12d ago[dead]
- cube00 12d ago> You didn’t really have one big app that you could open and order it to entertain you. Instead, you had a few dozen of bookmarks to websites, each with a specific idea in mind. This overlooks how common custom homepages such as Lycos, Yahoo! and MSN were and how the browsers pushed these as their default home pages. While they may not be as addictive as "the algorithm" of today, they were still packed with clickbait images/links to attract your attention and ads to drain your wallet. MSN: https://web.archive.org/web/20021108094511/http://www.msn.com/ https://web.archive.org/web/20021108094511/http://www.msn.co... My MSN: https://web.archive.org/web/20021107081016/http://my.msn.com/ https://web.archive.org/web/20021107081016/http://my.msn.com...
- ReturnoftheHack 12d ago[dead]
- redanddead 12d agoMicrosoft is still hawking this same beat decades later
- markus_zhang 12d agoI think this needs a strategic change. Unless you have a clear cut life objective and you are really into it, everything else you try to improve are at most tactical wins that will be lost to time. And those small wins actually make the end frustration worse because you thought you won strategically this time. I’ll also going to declare that doom scrolling-ish working on non-impactful personal projects is as BAD as doom scrolling twitter.
- econ 12d agoAlternatively we can have a committee distribute attention over things that are useful to the country/humanity/the economy etc.(can't think of the right word) Say, if someone next door is looking for an employee with exactly your expertise and experience and they are willing to pay for it the job offer is worthy of being at the top of your feed. That the person might not like seeing the perfect offer is less important than filling the hole in the economy.
- markus_zhang 12d agoI think HN users are usually hostile to committees, even the C++ one!
- aeturnum 12d agoOn the social science side, The Ecology of Attention[1] talks about attention as an ecosystem and the final economic frontier for capitalism. It envisions a world where every moment of our attention is fought over by commercial entities, which feels pretty prescient at the moment. [1] https://www.perlego.com/book/1536286/the-ecology-of-attention-pdf https://www.perlego.com/book/1536286/the-ecology-of-attentio...
- charcircuit 12d ago>Before recommendation algorithms where a thing, you had to decide what you would be doing on the computer. This is just the illusion of choice. Not only that attention was already being monetized. Digital advertising didn't start with algorithmic feeds.
- daniel31x13 12d ago> Instead of picking what we want to see we let other people decide what is supposed to be good for us. 100% agree, and this is exactly why I disable any sort of algorithmic recommendations on the platforms I’m active on. I do this either through the app itself or by using those platforms on the web and hiding distracting elements with an extension like uBlock or AdGuard. My final feed consists only of channels/profiles I directly follow. Oh, and there’s also RSS… I really recommend everyone do the same. It really clears your head.
- pixl97 12d agoThere are a few particular problems with this. Let's go with the most extreme example I can think of. You walk up to the interface of an infinite digital library. In this library there are an infinite amount of books that you'd like to read, or even contain anything coherent for you to read. The problem is there is also an infinite amount of books you don't want to read, can't read, or aren't even coherent information. And this second infinity is so much bigger than the first infinity that it is infinitely improbable that you will ever pick out a coherent book. Everything you do in a coherent informational world is directed by sets of biases. Algorithmic recommendations are just another one of the tools used to traverse this infinite world, and if you understand the biases they impart you can use these tools successfully.
- titzer 12d agoOk but why "find me a book that explains how car insurance works" no find book that explains how car insurance works but find exactly one car insurance company? ...because advertising
- siriusastrebe 12d agoI think in the next five years the internet will change drastically. Walled gardens, invite only, and companies that verify you're a human (I.E. use government identification). There is now no way to tell if somebody online in a human or a robot. Even video/zoom/voice is no longer a distinguishable.
- rpozarickij 12d ago> verify you're a human There could be a technology that cryptographically ensures that each letter/character came from a physical keypress/touch from a human when you want to interact with a human online in written form, but then again a robot could also be pressing the physical switches or interacting with a touchscreen. Perhaps this technology could handle copying/pasting too as long as the text originally came from a human. I hope we'll develop more robust mechanisms for detecting non-human generated content (including text) that aren't only enforceable by law (e.g. labeling AI-generated content in the EU). Apple Reference Image is one of the steps in this direction.
- pixl97 12d agoAt the end of the day I see no path to human victory in a digital world. It's kind of like a human moving to a planet with a nitrogen atmosphere. We have to wear a space suit to survive. No matter how much we slim it down, it's still a system on top of our existing systems. The internet is the same. We have to put a digital suit on our analog meat to explore a digital world that is becoming crowded with evolving digital natives, and those natives are rapidly becoming smarter and better than us in these spaces.
- rvshchwl 12d agoI cut out Social Media earlier this year, and it's been one of the best decisions I've made. Part of why I started was remembering that we didn't have doom-scrolling apps before smartphones. The change has been tremendous since I've been trying to go back to more "intentional" media. When I find myself distracted and wanting to pick up my phone, I realize there isn't really anything on it for me to do and am forced to just put it away. So I've started getting into the habit of keeping a book on me wherever I go. I've read 4 books this year!
- zeroonetwothree 12d agoHN is social media :)
- hypfer 12d agoYes. This pedantic gotcha that misses the point is amazing proof of that. Thanks for demonstrating.
- astura 12d agoYou're on social media right now, Hacker News is social media.
- vegnus 12d agoYeah, forums were social media too, but I dont remember people having such a negative reaction to forums as they do now to what replaced them, largely.
- hnlmorg 12d agoForums have their share of negativity too. HN included. There was always this curve where a forum would be fun because it was a small group of likeminded individuals. And as it got larger, more trolls came in, more newbies with different interests, and others that might not have shared the same cultural mentality that started that forum. And the forum would begin to shed its original members who draw the crowd. Then what’s left is people who have less interest in the community spirit. And that forum starts to shrink again as more people become disinterested and more infighting happens. Good moderation can help but nonetheless I’ve seen this happen across dozens of forums. In IRC channels, newsgroups and so on too. So it’s not even a phenomena specific to HTTP message boards. Some communities even have a terms for it like “the eternal September”. To be clear, this is a separate problem to the addictive design of modern platforms. That’s a whole other level negativity. But people seem to forget that the internet could be a cesspit even before Zuckerberg poisoned the well.
- afinlayson 12d agoWe don't have protection from attention hijacking algorithms the same way workers didn't have protection until unions became a thing. AI can do more for consumers than Unions ever could (because it'll be more uniform). Currently there's no motivation (monetary or otherwise) to make this tech easily available to the masses - and when it does the bad actors and dark pattern wizards will miss the good ole days when only the select few had access to these patterns.
- ddaniel10 12d agoI really liked to read this. i had this idea in mind but its nice to be remind of it
- econ 12d agoThe Mosaic browser (1993) had full text history search. We then got a bookmark system that was every bit as terrible as a web directory. It stayed that way. The delicious search revenue made organizing websites uninteresting. That obscure website you enjoyed a decade ago but don't even remember, they had lots of traffic like you. No point updating or keeping it online. You can't have rss in Firefox but here is a Facebook like button in your address bar in stead. I've tried to maintain the bookmark menu but I rarely use it since everything is dead. Why aren't browsers storing a text version of the bookmark? Did people in 1993 have more resources than us? Should I be afraid it grows to a few GB over the decades? A good few dead websites have a backup some place but there is no automation to find it. If you had a string of text from a page you might be able to search for it. If the page found is highly similar we might automate the process to have alternative location for bookmarks with a nice warning dialog.
- abc3354 12d agoRelated : Hister https://news.ycombinator.com/item?id=49743097 https://news.ycombinator.com/item?id=49743097
- bflesch 12d agoYour chain of thoughts is dangerous, because once you realize that can you can cache website content locally, you might also want to share this cache with your friends so that traffic is reduced ;)
- cesarsk 12d agoDo you guys know if there is an extension that textifies most of the internet?
- fwip 12d agoThere's a lot of reader-mode extensions out there - I'm not sure which ones are good nowadays.
- weinzierl 12d agoW3C's Amaya browser [1] was as much an editor as it was a browser. Production and consumption were supposed to be symmetrical. This was the promise of the web. Instead we turned it into a worse version of television, where you still only consume passively what is fed to you while being watched and analysed. [1] https://dev.w3.org/Amaya/amaya/AmayaPage.html https://dev.w3.org/Amaya/amaya/AmayaPage.html
- StilesCrisis 12d ago> Do you want to watch a video? YouTube knows you like cooking and art streams. But why not also recommend a few clips about the stock market bubble, global warming, and the war in Iran. I'm always fascinated when people claim YouTube pushes content like this. It's an extremely common take, but I don't think it's true. My experience has been that if I stop clicking something, the algorithm soon stops recommending it. It occasionally brings back a blast from the past--e.g. I watched a sumo competition once, and once in a blue moon it recommends another. But you can downvote bad recommendations, and I simply don't get recommendations for ideas I've never engaged with before. FWIW I'm not saying the algorithm is great. My feed is absolutely worthless right now, but it's all goofy Japanese content because one of my kids is extremely into that right now. There's zero political content because she isn't showing any interest in it. Heck, most of the stuff isn't even in English anymore.
- SoftTalker 12d agoIt's actually quite easy. If you watch something and you don't want more of it, delete it from your watch history. If it's particularly bad, you can block the channel entirely. I do this a lot for obvious AI slop content. Only subscribe to channels that are worth watching. Now my YouTube feed is almost too boring -- it's only showing me stuff in a pretty narrow set of interests and channels. I see very little outrage bait, celebrity news, "influencers," or politics. And hide "Shorts" entirely. They will reappear after a few weeks, but you can hide them again.
- bad_username 12d agoSame. I don't think I remember the last time I came across actually "FAR right propaganda" recommended by an algorithm. Could be that the author and I differ in what "far right" means.
- danbush 12d agoI wrote [something extremely similar](https://blog.scoopy.zone/posts/attention/ https://blog.scoopy.zone/posts/attention/) a bit ago. It is much harder to accomplish, but like the article says, you can still experience an Internet like the past.
- tripleee 12d agoPeak internet was the brief period of time when it was available but we weren't permanently connected. It was an intentional decision to log on, do what you wanted to do, and then disconnect.
- TacticalCoder 12d agoI've got two LANs at home, one of them I disconnect from the Internet by pulling a single ethernet cable (in a "try to hack this" way). It's honestly very refreshing to have stuff I self-host and be operational without needing 24/7 Internet connection. I'm running such a setup since years and, IMO, it helps me focus.
- analog8374 12d agoAttention is the axis of reality. Focus on a thing and it gets realer. Ignore a thing and it fades away.
- evo_9 12d agoAs logic stands, you couldn't meet a man who's from the future But logic broke; as he appeared, he spoke about the future [Pre-Chorus] "We're not going to make it," he explained how the end will come "You and me were never meant to be part of the future" [Chorus] All we have is now All we've ever had is now All we have is now All we'll ever have is now [Verse 2] I noticed that he had a watch and hat that looked familiar He was me, from a dimension torn free of the future [Pre-Chorus] "We're not going to make it," he explained how the end will come "You and me were never meant to be part of the future" [Chorus] All we have is now All we've ever had is now All we have is now All we'll ever have is now Flaming Lips - All We Have Is Now
- seydor 12d agothe most intentional internet was made by people, not algorithms now that we have cheap spam filtering, why don't we go back to forums
- pixl97 12d agoFor the same reasons we left forums in the first place. Spam was just a very small part of why people left forums. Size of the community and network effects was one of them. A constant stream of activity on any subject you wanted was yet another. Not having to log in to 40 different places yet another.
- inshard 12d agoI use SelfControl for Mac and set the down time to around 15 days. No Youtube no news sites etc. Except HN. It's helped a lot, after a week you can notice measurable change in attention and habit and productivity. SelfControl lets you add 48 hrs to your blocked time repeatedly, so you can get to a 15 day mark pretty easily through repeated mouse clicks.
- jolt42 12d agoI don't think social media is inherently bad. For me, the measure is whether I'm wallowing in consuming rather than doing. Although, yes, it's easy to get lost in consuming, doing takes much more intent/attention.
- pixl97 12d agoI mean it's no different saying a river is inherently bad. We need them for water for survival. What is inherently bad is the maelstrom of a raging torrent seeking to pull you under so no one ever sees your body again. This is what a lot of the platforms have evolved into.
- apparent 12d ago> LinkedIn will bury any relevant career news between the opinion of complete strangers. It is surely just a coincidence that those strangers happen to be shilling whatever Microsoft is invested in at the moment. The browser extension LinkOff makes the feed much more bearable, even useful. [1] Also, though I dislike the posts that LI pushes into my feed, I've never gotten the sense that this was motivated by a desire to boost Microsoft's investments. Perhaps I just wasn't paying attention, or am not cynical enough? 1: https://chromewebstore.google.com/detail/linkoff-filter-and-custom/maanaljajdhhnllllmhmiiboodmoffon https://chromewebstore.google.com/detail/linkoff-filter-and-...
- blfr 12d agoThis is a story of enshittification but... I don't see any of it and neither should anyone who reads HN: * adblocking is simple both in browser and on the dns level, * revanced youtube allows you to even skip sponsored segments, regular ads don't deserve a mention, you will see none, * reddit hasn't been good in forever, it's why we're here, somehow Twitter/Spotify/YouTube/TikTok algos work just fine for me with this one simple trick: I watch the good stuff, and skip/block/mute the not-so-good stuff. This works less well on LinkedIn but it's a Microsoft product, what did you expect, really. If you're like me, ie. on Hacker News daily, the modern Internet is a never ending series of wonders with virtually none of the inconvenience. And if on some absolute edge, you must use a website that sucks, both Claude and ChatGPT can drive a browser nowadays.
- andrewmutz 12d agoI agree wholeheartedly with the sentiment expressed in this post. Our attention is extremely valuable (both to us and to advertisers), and consumers today broadly squander it. For consumer digital products, the only business model that seems to work is advertising, because consumers are cheap and refuse to pay money for software. But the result is that they instead trade something far more valuable for the software, which is their attention. The products that get built, as a result, are supremely well-engineered to attract and retain attention, to the detriment of their users. Most of the "news" that reaches us these days is not at all an attempt to actually educate us about the state of the world. Instead its just a collection of inflammatory and enraging facts that are chosen to optimize for engagement (to hold your attention) In theory, someone could rebuild an ecosystem where people stopped squandering their attention and monetized it with money, but this would require consumers to stop chronically undervaluing their time and overvaluing their money.
- pixl97 12d agoAddiction is an extreme form of attention grabbing. We've moved from chemical addicticants to informational ones.
- omgwhoisthisher 12d agoAttention is all you _are_
- Krei-se 12d agoMy antidote to this was 100% selfhosting from home as you are now on par with the platforms and have a choice to serve yourself. You cannot stay focused all day and theres nothing wrong with "just show me random dumb stuff" i mean i wont lie, im old and RTL2 was a thing you know even in educated circles. These no tv people read bad fiction instead. And i actually kinda like the youtube algo to wind down - but i can also just switch to a timeshift archive. I think the problem is more that these platforms are reeeally comfortable to use and its not that easy to have the same - doable though. Oh and btw if you wanna go back to blog circles and xmpp and vpn lan games im all in but times are changing.
- soyboy_supreme 12d ago[flagged]
- Night_Thastus 12d agoThis is also why I view advertisement, in general, to be an invasive and incredibly dangerous. TV, online or elsewhere. It's all bad. It doesn't just let people know a product exists. It shapes the way we think about ourselves and the world around us. I view it like a cancer that we need to cut out. Everyone likes to think they're 'immune' to advertisement. That they're somehow mentally above the whole concept and it doesn't affect them at all. Total nonsense. I know the post mostly already says this, but related, this is also part of the danger of modern social media algorithms. Negative, outrage-inducing topics tend to get more attention in the short run. However, if someone watches it even once, that causes the algorithm to send a more of that kind of content. Before you know it, some people are getting fed and endless stream of the worst possible news, the most depressing takes, over and over. It can aggravate existing mental health issues or give them to people who did not have them before. The algorithm isn't 'designed' to do this, it's just trying to maximize metrics like time spent. It doesn't 'care' if the topics are negative or positive. But they have to be reshaped to consider the damage they can do when you start multiplying that by hundreds of millions of people.
- westoque 12d ago> But like every habit, the only thing you have to do is to keep at it. And if you pay enough attention to it, something will click in your brain. a bad habit i noticed after taking hiatus of focused work was me jumping into different windows and tabs and just mindlessly consuming content without finishing them. after noticing this pattern, i decided i will focus on one task at a time and finishing it, like reading the OPs article. it did wonders for me and i noticed i'm now even more focused that ever. the lesson is indeed being aware first, and then practicing the habit.
- baxtr 12d agoI see the same pattern with people on their phones. Opening up an app, immediately switching to the next app, repeat.
- nemo 12d agoReally, I encourage you to log off and go outside a while, take a look a what the local flora and fauna are about and learn about that, maybe use online tools or communities to learn, but remember to log off and go outside and experience the natural world where you can.
- cesarsk 12d agoHN feels a bit like the good old Internet. I hope it keeps being like this, as I feel interactions are nicer and calm here
- dwb 12d agoI remember reading an article in I think “net” magazine (Brits, remember that?) in the, I guess, early 2000s, that was predicting the “push” (as opposed to “pull”) internet. At the time I thought that sounded awful, and yep, it’s not good.
- Muhammad523 12d agoI noticed that i often spend lots of hours on my computer without doing anything useful but instead doomscrolling on hacker news and watching countless Youtube videos i hardly understand and give little attention. I think i could solve this by making a list of things i want to do on my computer before turning it on, but i haven't tried yet
- 0xdeadbeefbabe 12d agoOh funny I thought you were talking about LLMs.
- Dependance 12d agowait till we finally discover that attention is also all we are...!
- heyjonny 12d agoI hope that soon before we can rename "social media" to "AI media," there will be a renaissance of the old internet, where there's a mainstream desire for human-generated content, life events, and real exchanges with each other.