5 ms·

Daniel Aleksandersen of ctrl.blog has absolutely no foot to stand on here. He is a proponent of this kind of algorithmic blocking for weird browsers and even im

Daniel Aleksandersen of ctrl.blog has absolutely no foot to stand on here. He is a proponent of this kind of algorithmic blocking for weird browsers and even implemented it on his own site and argued for it. https://www.ctrl.blog/entry/detect-non-browser-form-submission.html https://www.ctrl.blog/entry/detect-non-browser-form-submissi... It's only after it happened to him that now he's suddenly against it. Until he removes the same type of blocks from his own website I have absolutely no sympathy for him.

phreack 4y ago
Even if that were the case (which we can debate), him being wrong before does not prevent him from being right now. Being de facto banned from the common internet due to centralization is absolutely scary.
- ranger_danger 4y ago
- superkuh 4y ago
  I completely agree. I am against Cloudflare and the centralization it implies 100%. I never use it for sites I develop. I just have no sympathy for Daniel since up until just now he was trying to get everyone to do this.
  - scarface74 4y ago
    CloudFlare allows website host to have much finer grain control that would have solved many of these problems - if they pay for it. I see no problem with this.
    - dmix 4y ago
      The hosts aren't blocking him though, it's Cloudflare. > Just about every website I visited from my home internet connection would result in a challenge page.
      - scarface74 4y ago
        Cloudflare is blocking him because the hosts didn’t configure Cloudflare to not use captcha for sub domains that host non browser traffic like podcast RSS feeds. That was his theory. That capability is only available for paid CloudFlare plans.
- bashinator 4y ago
  It's almost as though sufficiently large communications providers should be regulated as utilities.
ceejayoz 4y ago
The techniques described in that article are pretty reasonable and shouldn't significantly impact users - swapping name/email fields' names won't do a thing to you. There's also a difference between "this one website doesn't work for me" and "I've been blocked from half the Internet".
IshKebab 4y ago
To be fair there's a difference between doing it for one site, and doing it for a significant portion of the internet.
jamespo 4y ago
None of those techniques affect normal browsing
ldoughty 4y ago
I don't know the author or his reputation, but his suggestions that you linked are (in my opinion) standard actions for any dev/server admin getting spammed by their forms... And the suggestions really only impact malicious actors accessing your website from a script... Virtually none of those would be an issue for any browser made in the last 15-20 years, or headless browsers, but would break rudimentary scripts like entry level hackers/spammers might use. He also specifically called out CAPTCHA as user-hostile.
- superkuh 4y ago
  I guess like ctrl.blog you can't grasp the significance of the issue until it happens to you. My firefox fork is definitely blocked by his algorithmic "bot" detector. Just because your browser isn't doesn't mean it only blocks bots. False positives happen. They happen a lot more than you think. And they are a serious problem. Even more serious when it's cloudflare, but arguing for everyone to implement these algorithmic blocks "that won’t inconvenience users" individually, taken to it's logical end, does the same.
  - ldoughty 4y ago
    I don't see the reason for the personal attack. The blog post also calls out that you should not block based on user agent. If a form post didn't respect the action property having a #, that name/email HTML names might be reversed (whole the type is correct, and the user displayed values are correct), or include hidden HTML form fields that have been standard since ~97? Back when I made my first few websites, I certainly would agree that they are likely bots. Again, apparently this person has some hateful following, but I don't appreciate you limping me into this hatred for agreeing with his statements on this one particular issue.
    - superkuh 4y ago
      You said, "And the suggestions really only impact malicious actors accessing your website from a script." and that was false. Since you didn't have experience being blocked you couldn't know. Not till it happens to you. I don't think pointing this out is a personal attack. It's just the way people work. People don't believe things are a problem until they become a problem for them. You and others can keep quoting the legit and clever ways to mitigate bot spam but if you ignore the false positives the other checks create it kind of defeats the point.
scarface74 4y ago
> I strongly discourage you from blocking or discriminating against unknown or uncommon browser User-Agent request headers. The web is weird and we as developers shouldn’t discourage it.
deleted 4y ago
[deleted]
bergwerf 4y ago
From the link you mentioned: > Bots often mimic the User-Agent of a common browser, but the version numbers used in the bots rarely change. Over time they drift farther and farther behind until a point (maybe two-year-old versions) where you can safely block them without inconveniencing legitimate users. This supports the idea that browsers are subject to constant change and everyone should be forced to come along (rather than respecting and supporting standards). I have a Chromebook that stopped receiving updates some years ago (thank you for your very safe and sustainable product Google!), his heuristic would litteraly block me.
- ReptileMan 4y ago
  Doesn't your chrome app updates? Never used chromebook. Just asking.
daenney 4y ago
Burn the witch! Lets read through that page for a second though: Drop support for obsolete HTTP versions Doesn't seem like that's going to cause much issue for any legitimate client from the past 10-20 years. He only recommends blocking HTTP 0.9/1.0, which fair enough Append a #hash to the form’s action URL Hah. Clever man. I don't see how this is going to stop any legitimate user from loading your website or submitting the form, but I can see how it might frustrate bots. Include a hidden prefilled form field This is just standard practice to mitigate CSRF. Verify the Host and Origin request headers Yes. You should be doing that. Set a test cookie and verify it gets included in the submission Another CSRF trick. Swap the name attributes in the name and email fields This one's a little user hostile to folks who use assistive devices like screen readers. But still won't prevent you from accessing the site in the first place. Verify the POST/Redirect/GET (PRG) chain As noted by the author, might cause some issues but again, won't stop anyone from loading your website. Block ancient versions of common browsers Alright please just don't do this. UA blocking is gross and might prevent access through specialist software. But he also calls this out himself. I strongly discourage you from blocking or discriminating against unknown or uncommon browser User-Agent request headers All in all, with the exception of UA blocking I don't see how any of these mitigations would result in users not being able to access said website, or having their loading times drastically increased.
- nijave 4y ago
  All that stuff is easily defeated by automated browsers anyway (i.e. selenium)
  - mh- 4y ago
    Yes, but those automated browsers are much more expensive to operate than simple HTTP clients pretending to be browsers. It's an arms race/defense-in-depth situation. If someone truly wants to automate your site in a targeted fashion, and it's profitable for them to do so, you'll have to invest a lot more in stopping it (and decide how much of it is worth stopping).
    - 4y ago
phantom_of_cato 4y ago
Throwing an ad hominem is not cool.
deleted 4y ago
[deleted]
ufmace 4y ago
I just read it, and I don't see any contradiction here. IMO, he's recommending simple and direct anti-bot methods to web admins specifically because it's better than relying solely on Cloudflare etc for all bot blocking. He never recommends making un-appealable access control decisions based on third-party lists, and specifically recommends caution on methods that might potentially impact innocent users. Seems perfectly consistent to me.

phreack 4y ago

Even if that were the case (which we can debate), him being wrong before does not prevent him from being right now. Being de facto banned from the common internet due to centralization is absolutely scary.

ranger_danger 4y ago
superkuh 4y ago
I completely agree. I am against Cloudflare and the centralization it implies 100%. I never use it for sites I develop. I just have no sympathy for Daniel since up until just now he was trying to get everyone to do this.
- scarface74 4y ago
  CloudFlare allows website host to have much finer grain control that would have solved many of these problems - if they pay for it. I see no problem with this.
  - dmix 4y ago
    The hosts aren't blocking him though, it's Cloudflare. > Just about every website I visited from my home internet connection would result in a challenge page.
    - scarface74 4y ago
      Cloudflare is blocking him because the hosts didn’t configure Cloudflare to not use captcha for sub domains that host non browser traffic like podcast RSS feeds. That was his theory. That capability is only available for paid CloudFlare plans.
bashinator 4y ago
It's almost as though sufficiently large communications providers should be regulated as utilities.

ceejayoz 4y ago

The techniques described in that article are pretty reasonable and shouldn't significantly impact users - swapping name/email fields' names won't do a thing to you. There's also a difference between "this one website doesn't work for me" and "I've been blocked from half the Internet".

IshKebab 4y ago

To be fair there's a difference between doing it for one site, and doing it for a significant portion of the internet.

jamespo 4y ago

None of those techniques affect normal browsing

ldoughty 4y ago

I don't know the author or his reputation, but his suggestions that you linked are (in my opinion) standard actions for any dev/server admin getting spammed by their forms... And the suggestions really only impact malicious actors accessing your website from a script... Virtually none of those would be an issue for any browser made in the last 15-20 years, or headless browsers, but would break rudimentary scripts like entry level hackers/spammers might use. He also specifically called out CAPTCHA as user-hostile.

superkuh 4y ago
I guess like ctrl.blog you can't grasp the significance of the issue until it happens to you. My firefox fork is definitely blocked by his algorithmic "bot" detector. Just because your browser isn't doesn't mean it only blocks bots. False positives happen. They happen a lot more than you think. And they are a serious problem. Even more serious when it's cloudflare, but arguing for everyone to implement these algorithmic blocks "that won’t inconvenience users" individually, taken to it's logical end, does the same.
- ldoughty 4y ago
  I don't see the reason for the personal attack. The blog post also calls out that you should not block based on user agent. If a form post didn't respect the action property having a #, that name/email HTML names might be reversed (whole the type is correct, and the user displayed values are correct), or include hidden HTML form fields that have been standard since ~97? Back when I made my first few websites, I certainly would agree that they are likely bots. Again, apparently this person has some hateful following, but I don't appreciate you limping me into this hatred for agreeing with his statements on this one particular issue.
  - superkuh 4y ago
    You said, "And the suggestions really only impact malicious actors accessing your website from a script." and that was false. Since you didn't have experience being blocked you couldn't know. Not till it happens to you. I don't think pointing this out is a personal attack. It's just the way people work. People don't believe things are a problem until they become a problem for them. You and others can keep quoting the legit and clever ways to mitigate bot spam but if you ignore the false positives the other checks create it kind of defeats the point.

scarface74 4y ago

> I strongly discourage you from blocking or discriminating against unknown or uncommon browser User-Agent request headers. The web is weird and we as developers shouldn’t discourage it.

deleted 4y ago

[deleted]

bergwerf 4y ago

From the link you mentioned: > Bots often mimic the User-Agent of a common browser, but the version numbers used in the bots rarely change. Over time they drift farther and farther behind until a point (maybe two-year-old versions) where you can safely block them without inconveniencing legitimate users. This supports the idea that browsers are subject to constant change and everyone should be forced to come along (rather than respecting and supporting standards). I have a Chromebook that stopped receiving updates some years ago (thank you for your very safe and sustainable product Google!), his heuristic would litteraly block me.

ReptileMan 4y ago
Doesn't your chrome app updates? Never used chromebook. Just asking.

daenney 4y ago

Burn the witch! Lets read through that page for a second though: Drop support for obsolete HTTP versions Doesn't seem like that's going to cause much issue for any legitimate client from the past 10-20 years. He only recommends blocking HTTP 0.9/1.0, which fair enough Append a #hash to the form’s action URL Hah. Clever man. I don't see how this is going to stop any legitimate user from loading your website or submitting the form, but I can see how it might frustrate bots. Include a hidden prefilled form field This is just standard practice to mitigate CSRF. Verify the Host and Origin request headers Yes. You should be doing that. Set a test cookie and verify it gets included in the submission Another CSRF trick. Swap the name attributes in the name and email fields This one's a little user hostile to folks who use assistive devices like screen readers. But still won't prevent you from accessing the site in the first place. Verify the POST/Redirect/GET (PRG) chain As noted by the author, might cause some issues but again, won't stop anyone from loading your website. Block ancient versions of common browsers Alright please just don't do this. UA blocking is gross and might prevent access through specialist software. But he also calls this out himself. I strongly discourage you from blocking or discriminating against unknown or uncommon browser User-Agent request headers All in all, with the exception of UA blocking I don't see how any of these mitigations would result in users not being able to access said website, or having their loading times drastically increased.

nijave 4y ago
All that stuff is easily defeated by automated browsers anyway (i.e. selenium)
- mh- 4y ago
  Yes, but those automated browsers are much more expensive to operate than simple HTTP clients pretending to be browsers. It's an arms race/defense-in-depth situation. If someone truly wants to automate your site in a targeted fashion, and it's profitable for them to do so, you'll have to invest a lot more in stopping it (and decide how much of it is worth stopping).
  - 4y ago

phantom_of_cato 4y ago

Throwing an ad hominem is not cool.

deleted 4y ago

[deleted]

ufmace 4y ago

I just read it, and I don't see any contradiction here. IMO, he's recommending simple and direct anti-bot methods to web admins specifically because it's better than relying solely on Cloudflare etc for all bot blocking. He never recommends making un-appealable access control decisions based on third-party lists, and specifically recommends caution on methods that might potentially impact innocent users. Seems perfectly consistent to me.