9 ms·
I’m in love with the theme switcher. This is how a personal blog should be. Great content. Fun site to be on. My issue is that crawlers aren’t respecting robot
by reactordev 1y ago
I’m in love with the theme switcher. This is how a personal blog should be. Great content. Fun site to be on.
My issue is that crawlers aren’t respecting robots.txt, they are capable of operating captchas, human verification check boxes, and can extract all your content and information as a tree in a matter of minutes.
Throttling doesn’t help when you have to load a bunch of assets with your page. IP range blocking doesn’t work because they’re lambdas essentially. Their user-agent info looks like someone on Chrome trying to browse your site.
We can’t even render everything to a canvas to stop it.
The only remaining tactic is verification through authorization. Sad.
- amelius 1y agoThe theme switcher uses local storage as a kind of cookie (19 bytes for something that could fit in 1 byte). Kind of surprised they don't show the cookie banner. Just a remark, nothing more. PS, I'm also curious why the downvotes for something that appears to be quite a conversation starter ...
- reactordev 1y agoBecause she’s using local storage…? If you don’t use cookies, you don’t need a banner. 5D chess move.
- amelius 1y agoSounds to me like a loophole in the law then. Which would be surprising too since not easy to overlook.
- reactordev 1y agoIt’s not a loophole. localStorage is just that, local. Nothing is shared. No thing is “tracked” beyond your site preferences for reading on that machine. I say it’s a perfect application of how to keep session data without keeping session data on the server, which is where GDPR fails. It assumes cookies. It assumes a server. It assumes that you give a crap about the contents of said cookie data. In this case, no. Blast it away, the site still works fine (albeit with the default theme). This. Is. Perfect.
- 0x073 1y agoGdpr don't assumes cookies, if you misuse local storage you also need confirmation.
- reactordev 1y agoonly if you are storing personal information. Email, Name, unique ID. Something as simple as "blue" doesn't qualify.
- dkersten 1y agoCorrect. But you can also use cookies for that, without violating GDPR or the ePrivacy directive.
- reactordev 1y agoThen you have the problem of some users blocking cookies at the browser level. LocalStorage is perfect application for this use case.
- account42 1y agoOr maybe you could respect those user's preferences of not having shit stored for your website.
- dkersten 1y ago> which is where GDPR fails. It assumes cookies. It does not assume anything. GDPR is technology agnostic. GDPR only talks about consent for data being processed, where 'processing' is defined as: ‘processing’ means any operation or set of operations which is performed on personal data or on sets of personal data, whether or not by automated means, such as collection, recording, organisation, structuring, storage, adaptation or alteration, retrieval, consultation, use, disclosure by transmission, dissemination or otherwise making available, alignment or combination, restriction, erasure or destruction; (From Article 4.2) The only place cookies are mentioned is as one example, in recital 30: Natural persons may be associated with online identifiers provided by their devices, applications, tools and protocols, such as internet protocol addresses, cookie identifiers or other identifiers such as radio frequency identification tags. This may leave traces which, in particular when combined with unique identifiers and other information received by the servers, may be used to create profiles of the natural persons and identify them.
- alternatex 1y agoLocalStorage is per host though. You can't track people using LocalStorage, right?
- reactordev 1y agoLocalStorage is per client, per host. You generally can't track people using LocalStorage without some server or database on the other side to synchronize the different client hosts. GDPR rules are around personal preference tracking, tracking, not site settings (though it's grey whether a theme preference is a personal one or a site one).
- root_axis 1y ago> though it's grey whether a theme preference is a personal one or a site one In this case it's not grey since the information stored can't possibly be used to identify particular users or sessions.
- dkersten 1y agoThe law is very clear, if you actually read it. It doesn't care what technology you use: cookies, localstorage, machine fingerprints, something else. It doesn't care. It cares about collecting, storing, tracking, and sharing user data. You can use cookies, or local storage, or anything you like when its not being used to track the user (eg for settings), without asking for consent.
- roywashere 1y agoThat is not how it works. The ‘cookie law’ is not about the cookies, it is about tracking. You can store data in cookies or in local storage just fine, for instance for a language switcher or a theme setting like here without the need for a cookie banner. But if you do it for ads and tracking, then this does require consent and thus a ‘cookie banner’. The storage medium is not a factor.
- root_axis 1y agoThere's no distinction between localstorage and cookies with respect to the law, what matters is how it is used. For something like user preferences (like the case with this blog) localstorage and cookies are both fine. If something in localstorage were used to track a user, then it would require consent.
- athenot 1y agoYou don't need the cookie banner for cookies that are just preferences and don't track users.
- dotancohen 1y agoWhich is why calling it the cookie banner is a diversion tactic by those who are against the privacy assurances of the GPDR. There is absolutely no problem with cookies. The problem is with the tracking.
- reactordev 1y agoOur problem is with tracking. Their problem is that other companies are tracking. So let’s stop the other companies from tracking since we can track directly from our browser. GDPR requires cookie banner to scare people into blocking cookies There, now only our browser can track you and only our ads know your history… We’ll get the other two to also play along, throw money at them if they refuse, I know our partner Fruit also has a solution in place that we could back-office deal to share data.
- bigstrat2003 1y agoYou're assuming bad intent where there are multiple other explanations. I call it the cookie banner and I don't run a web site at all (so, I'm not trying to track users as you claim).
- dotancohen 1y agoYou call it the cookie banner because you've been hearing it regularly referred to as the cookie banner. It was the regularization of calling it the cookie banner that confuses people into thinking the issue is about cookies, and not about tracking.
- bigstrat2003 1y agoSo, by your own admission, calling it the cookie banner is not only "a diversion tactic by those who are against the privacy assurances of the GPDR". My only point is that you were painting with an overly broad brush and saying someone is a bad actor if they call it the cookie banner, which is demonstrably not the case.
- hju22_-3 1y agoI'd guess it's due to it not being a cookie, by technicality, and is not required then.
- account42 1y agoInstead of guessing you could inform yourself.
- ProZsolt 1y agoYou don't have to show the cookie banner if you don't use third party cookies. The problem with third party cookies that it can track you across multiple websites.
- account42 1y agoWrong, you also need to ask permission before using first-party tracking cookies.
- lucideer 1y agoYou don't need a banner if you use cookies. You only need a banner if you store data about a user's activity on your server. This is usually done using cookies, but the banners are neither specific to cookies nor inherently required for all cookies. --- Also: in general the banners are generally not required at all at an EU level (though some individual countries have implemented more narrow local rules related to banners). The EU regs only state that you need to facilitate informed consent in some form - how you do that in your UI is not specified. Most have chosen to do it via annoying banners, mostly due to misinformation about how narrow the regs are.
- the_duke 1y agoYou only need cookie banners for third parties, not for your own functionality.
- root_axis 1y agoGDPR requires informed consent for tracking of any kind, whether that's 3rd party or restricted to your own site.
- input_sh 1y agoIncorrect, GDPR requires informed consent to collect personally identifiable information, but you can absolutely run your own analytics that only saves the first three octets of an IP address without needing to ask for constent. Enough to know the general region of the user, not enough to tie any action to an individual within that region. Therefore, not personally identifiable. Of course, you also cannot have user authentication of any kind without storing PII (like email addresses).
- root_axis 1y agoYou've stretched the definition of tracking for your hypothetical. If you can't identify the user/device then you're not tracking them.
- input_sh 1y agoI literally worked with digital rights lawyers to build a tool to exercise your GDPR rights, but sure, call it a hypothetical.
- root_axis 1y agoIt's literally a hypothetical situation you introduced for the sake of discussion. "Hypothetical" doesn't mean it doesn't happen in real life, the whole purpose of a hypothetical is to model reality for the sake of analysis.
- rafram 1y ago19 whole bytes!
- martin-t 1y agoThis shouldn't be enforced through technology but the law. LLM and other "genAI" (really "generative machine statistics") algorithms just take other people's work, mix it so that any individual training input is unrecognizable and resell it back to them. If there is any benefit to society from LLM and other A"I" algorithms, then most of the work _by orders of magnitude_ was done by the people whose data is being stolen and trained on. If you train on copyrighted data, the model and its output should be copyrighted under the same license. It's plagiarism and it should be copyright infringement.
- thewebguyd 1y ago> and resell it back to them. This is the part I take issue with the most with this tech. Outside of open weight models (and even then, it's not fully open source - the training data is not available, we cannot reproduce the model ourselves), all the LLM companies are doing is stealing and selling our (humans, collectively) knowledge back to us. It's yet another large scale, massive transfer of wealth. These aren't being made for the good of humanity, to be given freely, they are being made for profit, treating human knowledge and some raw material to be mined and resold at massive scale.
- martin-t 1y agoAnd that's just one part of it. Part 2 is all the copyleft code powering the world. Now it can be effortlessly laundered. The freedom to inspect and modify? Gone. Part 3 is what happens if actual AI is created. Rich people (who usually perform zero- or negative- sum work, if any) need the masses (who perform positive-sum work) for a technological civilization to actually function. So we have a log of bargaining power. Then an ultra rich narcissistic billionaire comes along and wants to replace everyone with robots. We're still far off from that even if actual AI is achieved but the result is not that everyone can live a happy post-scarcity life with equality, blackjack and hookers. The result is that we all become beggars dependent on what those benevolent owners of AI and robots hand out to us because we will no longer have anything valuable to provide (besides our bodies I guess).
- 1y ago
- pas 1y agoPoW might not work for long, but Anubis is very nice: https://anubis.techaro.lol/ https://anubis.techaro.lol/ That said ... putting part of your soul into machine format so you can put it on on the big shared machine using your personal machine and expecting that only other really truly quintessentially proper personal machines receive it and those soulless other machines don't ... is strange. ... If people want a walled garden (and yeah, sure, I sometimes want one too) then let's do that! Since it must allow authors to set certain conditions, and require users to pay into the maintenance costs (to understand that they are not the product) it should be called OpenFreeBook just to match the current post-truth vibe.
- pyrale 1y agoI’m not sure that the issue is just a technical distinction between humans and bots. Rather it’s about promoting a web serving human-human interactions, rather than one that exists only to be harvested, and where humans mostly speak to bots. It is also about not wanting a future where the bot owners get extreme influence and power. Especially the ones with mid-century middle-europe political opinions.
- pas 1y agoI appreciate the intents, needs, concerns of my fellow humans, but it seems they are fundamentally unaware of the sociotechnological limitations of Web 1.0. I also would like to see many of these bot owners to radically reevaluate their priorities (toward more giving back - and not just to shareholders).
- reactordev 1y agoSecurity through obscurity is no security at all…
- pas 1y agoit's not security, it's an economic disincentive, it's basically throttling.
- workethics 1y ago
- lrivers 1y agoPoints off for lack of blink tag. Do better
- mclau157 1y agoHomeStarRunner had a theme switcher
- jasonvorhe 1y agoThese themes are really nice. Even work well on quirky displays. Stuff like this is what makes me enjoy the internet regardless of the way to the gutter.
- Karawebnetwork 1y agoReminds me of CSS Zen Garden and its 221 themes: https://csszengarden.com/ https://csszengarden.com/ e.g. https://csszengarden.com/221/ https://csszengarden.com/221/ https://csszengarden.com/214/ https://csszengarden.com/214/ https://csszengarden.com/123/ https://csszengarden.com/123/ See all: https://csszengarden.com/pages/alldesigns/ https://csszengarden.com/pages/alldesigns/
- cxr 1y agoOnly somewhat related and unfortunately misses the point. CSS Zen Garden was powered by style sheets as they were designed to be used. Want to offer a different look? Write an alternative style sheet. This site doesn't do that. It compiles everything to a big CSS blob and then uses JS (which for some reason is also compiled to a blob, despite consisting of a grand total of 325 SLOC before being fed into bundler) to insert/remove stuff from the page and fiddle with a "data-theme" attribute on the html element. Kind of a bummer since clicking through to the author's Mastodon profile shows a bunch of love for stuff like a talk about "Un-Sass'ing my CSS" and people advocating others "remove JS by pointing them to a modern CSS solution". (For comparison: Firefox's page style switcher and the DOM APIs it depends on[1] are older than Firefox itself. The spec[1] was made a recommendation in November 2000.) 1. <https://www.w3.org/TR/DOM-Level-2-HTML/html.html#ID-87355129 https://www.w3.org/TR/DOM-Level-2-HTML/html.html#ID-87355129>)
- reactordev 1y agoI fault her static site builder and not the author for that. It’s just how her bundler bundles.
- cxr 1y agoThis makes as much sense as choosing not to fault the person carrying a dagger who buried it into your shoulder—because you just fault their dagger.
- reactordev 1y ago
- deleted 1y ago[deleted]
- heikkilevanto 1y agoI have been speculating on adding a tar pit on my personal web site. A script that produces a page of random nonsense and random looking links to the same script. The thing not linked to anywhere, but explicitly forbidden on robots.txt. If the crawlers start on it let them get lost. Bit of rate limiting should keep my server safe, and slow down the crawlers. Maybe I should add some confusing prompts on the page as well... Probably I never get around to it, but the idea sounds tempting.
- reactordev 1y agoI did something similar. On a normal browser it just displays the matrix rain effect. For a bot, it's a page of links on links to pages that link to each other using a clever php script and .htaccess fun. The fun part is watching the logs to see how long they get stuck for. As each link is unique and can build a tree structure several GB deep on my server. I did this once before with an ssh honey pot on my Mesos cluster in 2017.
- shakna 1y agoI have a single <a> element in my website's head, to a route banned by robots and the page is also marked by noindex meta tags and http headers. When something grabs it, which AI crawlers regularly do, it feeds them the text of 1984, about a sentence per minute. Most crawlers stay on the line for about four hours.
- Halian 1y agoAnubis or, like Xkeeper of The Cutting Room Floor has done, block the major Chinese cloud providers.
- oooyay 1y agohttps://localghost.dev/about/ https://localghost.dev/about/ The theme also changes the background of her profile picture. The attention to detail is commendable.
- jacobyoder 1y agoHovering over the netscape link renders it slowly, line by line, like images used to come down...
- oooyay 1y agohah, that's amazing
- clbn 1y agoNot just the background, the Netscape one is a different photo!
- Scrounger 1y ago> My issue is that crawlers aren’t respecting robots.txt Cloudflare has a toggle switch to automatically block LLM's + scrapers etc: https://blog.cloudflare.com/declaring-your-aindependence-block-ai-bots-scrapers-and-crawlers-with-a-single-click/ https://blog.cloudflare.com/declaring-your-aindependence-blo...
- ryao 1y agoIf you want a good example of a site with a theme switcher: https://www.csszengarden.com/pages/alldesigns/ https://www.csszengarden.com/pages/alldesigns/
- aledalgrande 1y agoThe Netscape theme is my favorite. Love the pixel-y cursor animation