9 ms·
When I was at Google I started both the login risk analysis project and the Javascript-based bot detection framework they're now enforcing, so it's a pity to se
by mike_hearn 8y ago
When I was at Google I started both the login risk analysis project and the Javascript-based bot detection framework they're now enforcing, so it's a pity to see so many angry comments. Maybe a bit of background will make it seem more reasonable.
Firstly, this isn't some weird ploy to boost ad revenue. This is the login page - users are typing in a long term stable identifier already! The Javascripts they are requiring here are designed to detect tools, not people. All mass account hijacking attacks rely on bots that either emulate or automate web browsers, and Google has a technology that has proven quite effective at detecting these tools. There's a little bit of info on how it works scattered around the internet, but none of the explanations are even remotely complete, and they're all years old now too. Suffice it to say: no JS = no bot signals.
Google had the ability to enforce a JS-required rule on login at least 6 years ago and never used it until now. Without a doubt, it's being enforced for the first time due to some large account hijacking attack that has proven impossible to stop any other way. After so many years of bending over backwards to keep support for JS blocking users alive, it's presumably now become the weakest link in the digital Maginot line surrounding their network.
For the people asking for the risk analysis to be disable-able: you used to be able to do that by enabling two-factor authentication. You can't roll back your account to just username and password security though: it's worth remembering that account hijacks are about more than just the immediate victim. When an account is taken over it's abused and that abuse creates more victims. Perhaps the account is used to send phishing mails to your contacts, or financial scams, or perhaps it's used to just do normal spamming which can - at enough volume - cause Gmail IPs to get blocked by third party spam filters, and user's emails to get bounced even if their own account is entirely secure. The risk analysis is mostly about securing people's accounts, but it's also about securing the products more generally too.
- mysterydip 8y agoI understand what you're saying and it makes sense. I think in my mind it's the fact that javascript has the potential to do so many things, not that it's being used that way today. To use a bad car analogy, the in-car entertainment used to be just a dumb radio. Now that it's a computer connected to the main car network, it has a lot more potential to do things, whether it's a feature, bug, or an exploit.
- somecallitblues 8y agoThat was indeed some shit analogy.
- stevew20 8y agoAgreed, guts in the haggis.
- mpeg 8y agoNot to detract from your work there, but there's actually some great research papers about how Botguard itself is easy to bypass and google cookies provide most of the heavy lifting when it comes to bot detection. I've snooped around a bit myself and it doesn't seem like botguard does anything much more advanced than other fingerprinting solutions. I just don't buy that this is all about detecting more bots; every sophisticated bot I've seen in the ad world runs javascript as it's better to pose as a normal user, and only a tiny fraction of users would have javascript disabled. To be honest, I also don't think requiring javascript is a bad thing either.
- mike_hearn 8y agoAh, you're assuming it's the same strength on all places it's used - and also that it actually has been bypassed. There didn't used to be any public bots that can beat the strongest version and from a quick Googling around I don't see that it's changed. Someone took apart a single program manually, years ago, but the programs are randomly generated and constantly evolve. So that's not sufficient to be able to bypass it automatically/repeatedly. every sophisticated bot I've seen in the ad world runs javascript as it's better to pose as a normal user, and only a tiny fraction of users would have javascript disabled It's a lot faster and more scalable to not automate a full web browser. Bot developers would rather not do it, they only do because they're forced to. Forced to ... by requiring Javascript, like this.
- userbinator 8y agoIt's a lot faster and more scalable to not automate a full web browser. Bot developers would rather not do it, they only do because they're forced to. Forced to ... by requiring Javascript, like this. In other words, the bot developers are still getting through, and meanwhile it's the actual humans who don't want JS which get screwed. Reminds me of DRM... honest customers are the most inconvenienced, while crackers still break it.
- avip 8y agoYou are putting words he didn’t say. JS provides vast reactive surface to identify automated tools with.
- indeyets 8y agoThanks Mike, so the way I understand it, roughly it works like this: JS tries to gather some set of information about browser's environment (capabilities, network access, reaction to edge-cases, …) sends it to google and google decides if they should allow user to continue authentication (by providing crypto signature of request+nonce or smth.like that… or just by flipping key in session)
- xg15 8y agoJavascript ist also required for the vast majority of web-based exploits. I find it somewhat strange that you ask me to make my system less secure so you can better secure my account.
- whack 8y agoLike the blog post mentioned, 99.9% of users already have JS enabled, and this number is only going to go up as websites rely more and more on JS. For them, this is a purely beneficial change, with no downsides. It's somewhat selfish for you to ask that your system be made more secure, even at the cost of security for 99.9% of other users.
- knieveltech 8y agoAn error only becomes a mistake when you refuse to correct it.
- z3t4 8y agoI'm hijacking this thread to say we need a better ID system for the web! That preferably work without JS. Something built into browsers, that also allow you to create as many identities you want. When a id-signup header is detected, the user see a signup button, and can chose what information is sent to the web site/app. The user can login to any site with the push of a button, or even automatically. With a built in public-private key ID solution your friends will have the same ID-public-key on both site A and site B. The contact list can even be inside the browser, and web site's can ask for it, allowing for example white-listing in messenger apps, or let the user pick who are allowed to see their family pictures, etc. And web sites/app no longer have to store, username/password/keys, they only have to make a "challenge" where the browser automatically proves the ID. The private key should be exportable and standardized, and it should be possible to also use smart-cards and second factor logins. Having ID built into the browser means every site/app no longer have to build and manage all this functionality independently.
- plankers 8y agoThis is brilliant! Has the IETF put out any proposals for a standard of this sort or is this just an idea you had?
- exikyut 8y agoThanks so much for chiming in here. Admittedly, bot detection is really interesting to me as a subject. It's not the kind of thing you can throw infinite ML at and expect it to break even in terms of scaling; you need careful tuning and optimization baked in from the ground up, which means a fundamentally "manual" approach involving lots of lateral creativity and iteration. That creativity and one-upmanship (along maybe with the gigantic piles of money, because manual ;)) is what makes the field so interesting to me - but alas, I cannot pepper you with questions/topics/discussion/anecdotes/stories for two hours for obvious reasons :) So, instead, a couple of questions, considering datapoints likeliest to benefit others, and perhaps (hopefully) provide some appropriately dissuading signals as well. - How does Botguard detect Chromium running in headless X sessions? By looking at the (surely very wonky) selection of pages visited, or...? (Obviously IP range provenance is a major factor, considering the unusable experience Tor users reportedly have) - Regarding the note about "bending over backwards", while playing with some old VMs very recently I observed with amusement that google.com works in IE 5.5 on Win2K but not IE 6 on Win2K3SP4 (https://imgur.com/a/zg7FoAW https://imgur.com/a/zg7FoAW), entirely possibly due to broken client-side config, I'm not sure. In any case, I've also observed that Gmail's Basic HTML uses very very sparing XHR so as not to stackoverflow JScript, so I know the bending-over-backwards thing has stuck around for a long time. Besides general curiousity and interest in this practice, my question is, I wonder if this'll change going forward? Obviously some large enterprises are still stuck on IE6, which is hopefully only able to reach google.com and nothing else [external] :) - I wonder if a straightforward login API could be released that, after going through some kind of process, releases Google-compatible login cookies. I would not at all be surprised if such ideas have been discussed internally and then jettisoned; what would be most interesting is the ideation behind _why_ such an implementation would be a bad idea. On the surface the check sequence could be designed to be complex enough that Google would always "win" the ensuing "outsmart game", or at least collect sufficient entropy in the process that they could rapidly detect and iterate. My (predictable) guess as to why this wasn't implemented is high probability to incur technical debt, and unfavorable cost-benefit analysis. I ultimately have no problem that JS is necessary now; if anything, it gives me more confidence in my security. Because what other realistic interpretation is there?
- mike_hearn 8y ago
- hpcjoe 8y ago"Google had the ability to enforce a JS-required rule on login at least 6 years ago and never used it until now." Um ... this should terrify everyone. 'We had the power to impose this, and we graciously chose not to. You should be thankful for what we have done.' No. Just ... no.
- izacus 8y agoWhat exactly? The fact that login owner can change the login system at any point?
- gambler 8y agoSorry, but at this point it is pretty obvious that big tech companies care about account security only as far as it impact their services. The late revelation about Facebook abusing 2FA phone numbers for marketing is a great demonstration of how that works. Google too does some really funny things to make it nearly impossible to create and maintain an anonymous accounts not tied to a phone number. Even when those accounts are just used for browsing and bookmarking, without sending any outgoing information. >When an account is taken over it's abused and that abuse creates more victims. Pushing JavaScript everywhere increases the attack surface for every single user on the web. Except it doesn't happen overnight, and big companies who do it (by affecting standards, or by doing stuff like this ^) aren't affected by client-side exploits and privacy loss. If someone hijacks my browser through some clever JS API exploit and steals my credentials, what is Google's response? "Just use our 2FA." What about smaller websites that don't have resources to maintain 2FA? "They should authenticate through us." All roads seem to conveniently lead to centralization. BTW, it is worth noting that the impact of a compromised account isn't nearly as significant if a single account doesn't hold keys to pretty much everything you do online. Somehow this is rarely factored in during such discussions.
- danso 8y ago> Google too does some really funny things to make it nearly impossible to create and maintain an anonymous accounts not tied to a phone number. Well yes, these kind of accounts are highly susceptible to being bot accounts. What obligation does Google have to be the place for people's free, anonymous accounts? In any case, I haven't had a problem with the number of secondary accounts I've created that are tied to me only by another email address (which can point to another provider, like Yahoo). > BTW, it is worth noting that the impact of a compromised account isn't nearly as significant if a single account doesn't hold keys to pretty much everything you do online. Somehow this is rarely factored in during such discussions. How should this be factored into the current discussion? The use of JS is to ostensibly make it more difficult for automated hijacking to prey on users.
- drb91 8y agoThis is true. Google has explicitly never put the user first. We should be grateful to give them our information in the first place in the way they deem is best.
- mqus 8y agoCouldn't the attacker just switch to imap/pop3 for those attacks? (And (why )aren't they doing that already?)
- jsnell 8y agopop3 / non-oauth imap login attempts are blocked unless the user has explicitly opted in to allow those protocols: https://support.google.com/accounts/answer/6010255 https://support.google.com/accounts/answer/6010255
- msla 8y ago> Firstly, this isn't some weird ploy to boost ad revenue. Everything Google does is a ploy to boost ad revenue. That's the whole business model. > Without a doubt, it's being enforced for the first time due to some large account hijacking attack that has proven impossible to stop any other way. After so many years of bending over backwards to keep support for JS blocking users alive, it's presumably now become the weakest link in the digital Maginot line surrounding their network. You're doing two things here: 1. Reasoning from no evidence, and ignoring a much simpler reason in the process. 2. Acting like the honest web users, the ones blocking malware-laden JS, are the ones who are wrong. It's much simpler to conclude that Google engineers simply got lazy and decided to punt the hard work of security to some JS library, instead of looking at it honestly.
- antpls 8y ago> All mass account hijacking attacks rely on bots that either emulate or automate web browsers, and Google has a technology that has proven quite effective at detecting these tools It sounds untrue and short-sighted. Even Google provides the tools for anyone to automatically navigate in a javascript-enabled website with Chrome Headless. All this will do is to provide a short-term security before the bots can again perfectly mimic humans with JavaScript enabled, this time. Most "mass bots" can be stopped by logging the access attempts on the server side, with plain old HTML on the client side.
- danmg 8y agoDo you glow in the dark?
- cbkeller 8y agoInteresting to hear from someone involved. > This is the login page - users are typing in a long term stable identifier already! There are so many other considerations at work here though, and I can't imagine that they're not obvious to you as well? For starters, we're creatures of convenience, and this makes it significantly inconvenient to block google scripts on other websites even when not signed in. It also guarantees that you have the chance to produce a (likely unique) JS-based fingerprint of every google user that can then be used for correlation and de-anonymization of other data. But really the most basic point that probably makes folks here suspicious: if this were really only about preventing malicious login attempts by bots, then why not give users a clear, explicitly stated choice: either JS or 2FA.
- taeric 8y agoI'd be curious if the tools are only effective because of the fringe requirement of them. That is to say, right now, many malicious users are easy to spot using these, precisely because they don't have to be hard to spot. That is to say, in the long run, it will be interesting if this actually reduces malicious use. Seems like it would be just as easy to avoid.
- pharrington 8y agoIt's unfortunate that your work is being used in such a massively economically wasteful manner. The core issue is - it's all computers. The reason you can't reliably detect the difference between an average user using a computer to log in to your computer versus a fairly sophisticated computer using a computer to log in to your computer is that the transition between user and tool is not smooth. It is always going to be easier to find the various boundary points between user and tool than it is to construct a passable simulation of the user using that tool. Yes, bot detection does make account hijacking attempts more expensive, but it makes all logins more expensive, and the rate of expense increases faster for you than it does the account hijackers.
- deleted 8y ago[deleted]