14 ms·

Oauth2 support for GMail

338 points by edward 4y ago

pagutierrezn 4y ago
misunderstanding? https://twitter.com/wilbowma/status/1526697254140071936?t=451WHDUawEk-DeC2AecIRQ&s=19 https://twitter.com/wilbowma/status/1526697254140071936?t=45...
- jaapz 4y ago
  I don't think the problem here is that users can still use app passwords instead of OAuth2 - it's that the developer went through the trouble of developing a OAuth2 implementation, went through the necessary laborious steps to submit the application and then was faced with this message: > The cost of the assessment typically varies between $10,000 -$75,000 (or more) depending on the size and complexity of the application; smaller applications may see costs at a lower threshold of $4,500 That's just bad, but pretty typically google.
  - tssva 4y ago
    The Google documentation regarding Oauth access to Google APIs including Gmail explicitly mentions this requirement. It should not have been a shock to the author. Also this requirement only applies if the app is intended to store the data on a server. An email client which directly accesses and locally stores the email would not require a security audit. Pegasus would not be the 1st email client to use OAuth2 with Gmail and others have not required an audit. Some of the newer email client services which implement advance features by downloading email directly from Gmail to their own servers on the backend to do processing of the email would require a security audit.
solar-ice 4y ago
> and has the ability to access data from or through a third party server So Pegasus Mail accesses your email from their servers? For reference, they have an entire flow designed so that the auth credentials never touch the app developer's server for desktop and mobile apps - https://developers.google.com/identity/protocols/oauth2/native-app https://developers.google.com/identity/protocols/oauth2/nati...
- easton 4y ago
  I’m super confused about this too, there’s tons of open source apps that use Sign in with Google. Shoot, I’ve used the SDK myself in little apps and never saw this.
  - rjzzleep 4y ago
    The SDK on android or ios you mean? Cause that would be vastly different from adding a web authentication flow such as oauth to a legacy application.
  - Alex3917 4y ago
    "Sign in with Google" is just a button that apps are required to use when they allow the user to interact with Google in some capacity, but don't have their own account management system. Most apps with that button don't have the ability to get any data from Google other than your name and email address though; for apps where you are granting OAuth access to your Gmail, it is very obvious that you are doing so, so it's not likely that you have done it accidentally in the past. (Although you can obviously check on myaccount.google.com)
- snthd 4y ago
  Most mail clients can connect to both a google email account and a "third party server" email account. It's easier to rule out undetectable-by-google data exfilteration if the app can only connect to Google. The obvious way around this is to make a Google-only edition. Yuck.
  - solar-ice 4y ago
    I don't think that's what Google are saying here. I think the sentence is referring to Google user data; as long as the Google user data or credentials to access it does not touch another server, the entire thing does not apply. In fact, in Google's guidance on this subject, they say: > Local client applications that only allow user-configured transmissions of Restricted Scope data from the device may be exempt from this requirement [to get a Letter of Assessment]. And in another FAQ: > Local Data Storage: Local client applications don't need to undergo a security assessment because data is run, stored, and processed only on the user's device. Local client applications that only allow user-configured transmissions of Restricted Scope data from the device may be exempt from this requirement. My feeling is that the author of Pegasus Mail has checked a checkbox incorrectly somewhere, or alternatively has not implemented the desktop oauth2 flow correctly.
tatoalo 4y ago
> [...] including even having to prepare a Youtube video showing my code in action. What?!
- uniqueuid 4y ago
  This is unfortunately not unprecedented. When you apply for API access to facebook's graph API, you also need to document your app including videos.
- Alex3917 4y ago
  The video isn't really that hard. You just make a screencast of yourself downloading email while saying "now I'm using the download email scope", and then sending email while saying "now I'm using the send email scope." They don't care if the videos are terrible quality, it's only for them to understand what they're approving, and to have some documentation if a developer later changes an app to do something nefarious.
acatton 4y ago
I'm confused… Why do they want to support the OAUTH flow when they can just the normal app password flow without any change in their code, just documentation for their users. https://support.google.com/accounts/answer/185833?hl=en https://support.google.com/accounts/answer/185833?hl=en
- rob74 4y ago
  Maybe because the first line on the page you linked reads: > Tip: App Passwords aren’t recommended and are unnecessary in most cases. To help keep your account secure, use "Sign in with Google" to connect apps to your Google Account. So, for Google, App Passwords are clearly "second class citizens", deemed insecure and not recommended. And, as an app developer, you probably don't want to be seen as recommending an insecure authentication method?
  - baisq 4y ago
    I assume that users of Pegasus Mail are advanced enough to know that that is bullshit.
  - NoGravitas 4y ago
    This also strongly implies to me that Google is going to drop support for App Passwords in at most a few years.
- kevincox 4y ago
  The OAuth flow is better. It is easier for users and supports many more methods of authentication. Really the biggest downside about the OAuth flow is that it requires a relationship with Google (a client key). If this could be a fully decentralized standard it would be fantastic.
- sneak 4y ago
  If you have advanced protection enabled on your Google Account (and you should) then app passwords are no longer available.
Tepix 4y ago
Once again, i'm glad not to be using GMail.
- enriquto 4y ago
  Yep. All these anti-google complaints sound so useless. What did they expect? These issues are always an entirely self-inflicted problem. Google's terms of service are public, clear, and completely unacceptable. Why people keep using their services is beyond me.
  - blagie 4y ago
    Legacy. For example, my email is in Google. There are things related to e.g. evidence for litigation from many years back. I have documents shared with me in Google Docs. Google used to be pretty good about "don't do evil." I've degooglified what I can, but I can't degooglify 100%.
    - Tepix 4y ago
      It's never too late to switch.
      - blagie 4y ago
        It kind of is. If I want to have legal proof of when I shared a document with someone, I need my Google Docs. I guess what I should do is stop putting new stuff in Google.
  - hackmiester 4y ago
    It seemed like a good idea in 2005, and switching costs. But yeah, I bit the bullet and left this year.
  - xigoi 4y ago
    It's not easy to convince everyone to send mail to your new address. Especially the countless random websites where you signed up with the old one.
mbwgh 4y ago
I had a similar brick wall moment after having written a tool to sync Google Calendar events with the `when` calendar tool (http://www.lightandmatter.com/when/when.html http://www.lightandmatter.com/when/when.html). Which is why I never published it (although I have been using it for almost two years), and setting it up requires a series of laborious steps of activating the Google Developer Console, creating API keys etc., which I found just to embarrassing to put in a README :)
- VTimofeenko 4y ago
  I don't think it would be embarassing. Other tools use the same approach, like rclone: https://rclone.org/drive/ https://rclone.org/drive/
janandonly 4y ago
I am wondering if it is really legal for Google to even ask any money from 3rd party developers to use GMAIL (what should be) open standards like SMTP and IMAP. I guess I am so naive...
- richsouth 4y ago
  It might be immoral, but if you're trying to make money using their service I can't see there being a legal reason to prevent them doing this. I meant, they (Google) need the money right? Companies like this seem to get to the point where they feel justified in charging for every little thing that wouldn't kill them to give for free.
  - derbOac 4y ago
    If you're in the EU: https://en.m.wikipedia.org/wiki/Antitrust_cases_against_Google_by_the_European_Union https://en.m.wikipedia.org/wiki/Antitrust_cases_against_Goog...
nickdothutton 4y ago
Any takers for a bet on when Google will axe App Passwords?
- CyanBird 4y ago
  I'd say, no more than 3 years
qwerty456127 4y ago
I have one question in this regard: will I still be able to access my mail through my own script I myself wrote? I understand I will probably have to make some changes and click some things in GMail settings but is this still going to be possible or will I too have to "publish app" even if I only mean it for my own private usage?
- vivekv 4y ago
  For now app passwords
- mikeklaas 4y ago
  Development apps are exempt from the requirement. (They are also limited to 100 accounts)
  - cute_boi 4y ago
    But i find the token problem frequently. It seems it expires after 2-3 days. I don't know why :(
sneak 4y ago
Users should not be subjecting all of their correspondents' communications to US warrantless surveillance anyway; Google is doing the world a service by making their service harder and harder to interoperate with. Gmail and other huge centralized points of censorship and surveillance must be destroyed. If you are a user, move away. If you are a developer, do not support these closed systems.
- morganvachon 4y ago
  I agree wholeheartedly in principle, but the reality is that it's nearly impossible to escape Google getting their hands on yours and your correspondents' messages at some point. So many businesses (and third party email providers!) use Google's services on the back end, and many relays out there are owned by Google/Alphabet even if it's not readily apparent, that the only way to even attempt to avoid them is using encrypted email. Unless everyone you communicate with (including companies, governments, and other organizations) is willing to only communicate via encrypted email as well, it's a lost cause.
  - sneak 4y ago
    No, the perfect is the enemy of the good here. "Email is insecure anyway and Google is big, therefore it's ok to give the adversary 100% of the plaintext" is not sound logic. This is not some principled stance, it is a clear and practical command: stop using these services. If your personal email address ends in @gmail.com, you are doing it wrong.
    - morganvachon 4y ago
      I'm not saying you shouldn't give up Google services; I did. I'm just saying the unfortunate reality is that you'll never fully escape them.
- tptacek 4y ago
  Moving your data out of US jurisdiction doesn't shield it from US warrantless surveillance, but rather maximizes its exposure. The USG ostensibly requires due process (a warrant, whatever) to obtain information from US servers. It does not require any due process to obtain data from foreign servers. Coercively obtaining data from foreign servers is literally the chartered job of the NSA (of all signals intelligence agencies around the world, really). This is a super common misapprehension people have about data locality and surveillance, and it's no surprise that marketing plays off it ("keep your mail in Switzerland, so it's subject to Switzerland's strict privacy laws!"). There are other reasons to house things in Switzerland (or the Lesser Antilles or wherever). I'm not saying you should use GMail (unless security is a concern of yours, in which case you should use GMail).
Fizzadar 4y ago
I’ve gone through this process for my email client Kanmail [1]. The third party audit is not required for email clients that run on end users computers and store credentials locally. By the looks of it Pegasus falls into this category and should not have any issues getting approved (still need the YT video and such but the Google team are surprisingly responsive and helpful in my experience). [1] https://kanmail.io https://kanmail.io
- undecisive 4y ago
  The wording seems to imply that if, on a yearly whim, someone at Google decides to "empanel" a security assessment team, you have no choice, you will not necessarily be asked permission and you will be paying that invoice whether or not you needed that assessment. Do you have evidence - in writing - to the contrary from a Google official? Abridged wording and my non-lawyer interpretation below in case I'm not clear: > Every app that [accesses Gmail and also accesses other servers] is required to go through a security assessment from Google empanelled security assessors. [...] > In order to maintain access to restricted scopes, the app will need to undergo this security assessment on an annual basis, [... costs usually] between $10,000 - $75,000 (or more) [...] > This fee may be required whether or not your app passes the assessment and will be payable by the developer."
  - dal 4y ago
    So basically when your application resolves gmail.com you will access other servers. :P
    - undecisive 4y ago
      Well, certainly they haven't ruled it out. DNS servers are third-party servers. VPNs are third-party servers. They can be as kind or as nasty about this as they like, For all my fear mongering, I should point out that the only reason Google are saying this is to cover their backs when they decide to levy the maximum fee on an unsuspecting competitor. If they don't consider you a direct competitor, you might be ok. They have no reason to use this policy to alienate the majority of desktop applications that connect to email. But they also have no repercussions if they do.
wiradikusuma 4y ago
Does anyone has a link for that ToC? I'm curious what kind of use cases that need such payment. E.g. if I want to make my app to use Google login, I think it's completely free?
- corentin88 4y ago
  Not ToC but that’s where you will find all the details of the security assessment: https://support.google.com/cloud/answer/9110914?hl=en https://support.google.com/cloud/answer/9110914?hl=en
corentin88 4y ago
I believe many comments here will criticize Google. But objectively, Google is at its best here: - in terms of privacy, applications that have access to your Gmail inbox now require a security audit. - the audit is not required for MVP (<100 users) or applications internal to a Google Workspace org. Of course, you have to pay for the audit. But: - it’s only required when you ask for restricted user data (i.e. reading my emails). - Google doesn’t take 30% of your revenue to use its API - which is free by the way. To me, Google has created the perfect world for developers here. And that says a lot when I read the developer of Pegasus Mail doesn’t want to record a 2min long YouTube screencast to get approved. Also, just wanted to kudos the Google OAuth team that has greatly improved their process in the past years. If you follow the guidelines, you can get approved within a day now.
- lozenge 4y ago
  Is it my inbox or Google's inbox?
  - dspillett 4y ago
    It is part of Google's tracking and advertising system. They let you use it as an inbox.
  - dangrigsby 4y ago
    Is it your car or the government’s?
  - charcircuit 4y ago
    Google will be blamed if someone's data is misused even if that person authorized giving permissions to that app. If you want evidence supporting that, take a look at how much flak Facebook received with the Cambridge Analytica scandal.
  - scrapheap 4y ago
    If it's Gmail then it's Google's inbox, they're just letting you use it.
- 4y ago
adrianpthomas 4y ago
Went through this process with Mail Designer 365. Luckily as we only need sending access, the requirement was less arduous, but the process was not great. After submitting it took ages to get reviewed and even when it had been processed the status wasn't entirely clear. At the time it was made clear that using app passwords was considered a risk and users were receiving emails about it being a security risk.
throwaway787544 4y ago
Pegasus Mail, wow!!!! There's a blast from the past! I remember using this over 20 years ago!
- gommm 4y ago
  Same here :) It was my client of choice back 20 years ago when I used windows... Very nice to see that it still lives on
- dugmartin 4y ago
  We used it on our LAN around 1993 hooked to a local mail server (which ran on a Windows box using Netware I think - it was sitting on a filing cabinet next to my desk) that in turned fetched mail from the campus mail server. It worked really well and the local mail server let us easily share/email large attachments within the department as they never left the LAN.
- djbusby 4y ago
  In a DOS terminal-window on my NT4 box.
- pantulis 4y ago
  The Good Pegasus.
- tpmx 4y ago
  Yeah! It was my first ever email client back in 1994/1995. More details on the developer: https://en.wikipedia.org/wiki/David_Harris_(software_developer) https://en.wikipedia.org/wiki/David_Harris_(software_develop...
dewey 4y ago
There's a bunch of third party "indie" email apps, how are they doing it? Is this potentially just some boilerplate text and they evaluate the rate based on the size of the project in the end? I'm currently using https://mimestream.com/ https://mimestream.com/ and I somehow doubt all of them had to pay that kind of upfront money.
NickRandom 4y ago
I haven't seen a single comment (at time of posting this) touching on the cost aspect The cost of the assessment typically varies between $10,000 -$75,000 (or more) depending on the size and complexity of the application As he says, even if Google classed his as a small app (unlikely) "Regrettably, these kinds of fees are far beyond what I can afford, given that I rely on donations from my users to make ends meet: even the $4,500 fee is well beyond what I could find on an annual basis. " I get that doing proper auditing is expensive and the costs are within industry rates but to me it reeks of slowing turning the screws by boiling the frog on the slow path to monetization. (many mixed metaphors I know)
- jijji 4y ago
  It looks like this pay-for-play method is going to have a chilling effect on any developer creating an alternative Gmail interface. Same with removing IMAP.
xbar 4y ago
And so it is time to relegate my Gmail account to an auto-forwarder.
- tut-urut-utut 4y ago
  If you own your mail domain, it's the time to move off from Gmail. If not, make a clear cut, purchase domain, and never ever be at the mercy of mail provider.
  - mawise 4y ago
    This is so hard. I bought a domain and even started paying for (non-google) email on it over a year ago. But I can't bring my self to start the painful process of migrating everything.
    - r00fus 4y ago
      What it'll take is likely a cost/benefit for keeping your non-gmail-website workflow (and taking the leap to jump off gmail) vs. complying and using their (mostly usable) webmail site. They likely figure most will choose option#2 since it's so painful/expensive to do #1.
    - hackmiester 4y ago
      At least you will only have to do it once. I migrated from G Suite to Fastmail, but because I had a custom domain, it was basically entirely painless.
Saris 4y ago
Is gmail removing IMAP too? Why support OAUTH at all when you can use the normal method of logging in with IMAP and an app password?
- deleted 4y ago
  [deleted]
- nemothekid 4y ago
  They are likely removing IMAP. IMAP support is already disabled by default in Gmail
  - Mailtemi 4y ago
    Can you share а link for this statement. As far as I know IMAP works ok, you just need to use XOAUTH IMAP authentication.
  - Saris 4y ago
    If that happens, it sounds like a good chance to move away from gmail to something more robust.
- planede 4y ago
  App passwords are only available for 2fa enabled accounts, while OAUTH is available for everyone. So far I didn't get a clear answer if "the end of less secure authentication methods" means the end of app passwords as well, or only the end of signing into IMAP with your main google credentials.
sigzero 4y ago
Why the hell would they need to submit to that?! That's crazy.
sir 4y ago
If you don't have the time or resources to go through this process you can also use this simple proxy of mine, which intercepts the non-OAuth IMAP/SMTP commands and adds OAuth in the background: https://github.com/simonrob/email-oauth2-proxy/ https://github.com/simonrob/email-oauth2-proxy/
- dividuum 4y ago
  Ah yes. The "Just use FTP + curlftpfs + SVN" approach for solving this problem ;-) That really doesn't help OP as their customers most likely won't be capable to run that.
  - sir 4y ago
    Oh, it's certainly not an excuse for Google requiring this process. However, if you do want to use Pegasus Mail (or any other non-OAuth app/script/email tool, etc) after the end of May, this is a way to do so. I think it's quite a safe assumption that many of the HN audience are capable of using it, too ;-)
londons_explore 4y ago
I was aware of the security-audit-at-your-own-expense requirement, and I am not a gmail app developer, I just did a drive by of the documentation 1+ year ago... As a gmail user, this actually seems like a sensible thing. Gone are the days when users are trusted to just click 'allow' to let some random third party access all your data. It turns out, users don't fully understand the ramifications of that, and complain loudly when the 'free personality quiz' they allowed goes and downloads every private email to use for marketing. So, if I can't give access to my mail to a random untrusted third party, the next best thing seems to be to give access to someone who has at least had their systems audited and who at least has enough money to pay for an audit, meaning they probably have some business plan and can be tracked down by the courts if they mishandle the data.
superkuh 4y ago
Note that this "OAuth2" support is only for gmail. OAuth 2 is not really an open protocol like OAuth (1) was. It's just something the megacorps shoved down the IETF's throat and every implementation is different enough to need different software. OAuth2 is not good for the health of the internet and will lead to even more lock in. When Google disables imap at the end of the month for gmail (only allowing OAuth2+imap, which is not imap) that's the end of gmail for me.
- jeroenhd 4y ago
  OAuth is hardly Google only, it's part of RFC7628. There are very good reasons why a simple password login isn't necessarily what you want for your email protocol. IMAP and 2FA are often at odds, for example, usually leading to vulnerable application passwords that bypass security requirements out of necessity. I don't think many email clients support OAuth2 (Thunderbird does, but that could be Gmail specific?) but the concept isn't inherently bad, in my opinion. I don't really see the problem with OAuth2 itself, the email space is just very very slow at accepting new standards. Dovecot and UTF8 email addresses still don't play well together, for example. The lack of proper on-the-fly application registration for mail servers is annoying, but supporting them should be easy enough if the mail service isn't run by complete doofuses.
  - superkuh 4y ago
    >OAuth is hardly Google only, To be clear, I never said OAuth2 was google only. I said the standard OAuth2 is so much not a standard that an implementation written for Google's imap+OAuth2 will not work for any other email provider's imap+OAuth2.
  - yxhuvud 4y ago
    > Thunderbird does, but that could be Gmail specific No, it definitely works on Office 365 too.
- fivre 4y ago
  OAuth2 is a suite of protocols and standards you can use to build an auth system from people who have been working on auth systems for a while. it isn't strictly designed for federated auth, and arguably is targeted at internal systems because there are so many parts are "you need a system that does this, but how is up to you" OIDC builds on OAuth2 to fill in those gaps with prescribed details and add additional systems necessary for a workable federated auth system. it _should_ be possible to use it entirely provider-agnostic fashion, but in practice it's like any complex protocol with optional parts in that implementations have incorrect bits or idiosyncrasies (why does Azure's provider sometimes serve public keys it doesn't actually use to sign tokens in some modes? god only knows) and you'll probably only provide integrations for a subset of providers, not just let anyone with a discovery link show up and use it.
nullcipher 4y ago
Wow, is this the software that ran on Novell Netware? I had no idea this was still around.
malinens 4y ago
There is only one thing why this is done: to make impossible to move from gmail (also with custom domains) to another provider. We have the same problem. We spent hundreds of hours implementing their legacy auth (less secure apps), gmail oauth2 and domain-wide delegation to be later stuck in approval process. Until our app is not approved users can not use e-mail import/migration tool to our inbox.eu service. And there is lot of demand for that because people do not want to pay so much for e-mail. Shame on google... they also do not provide reas-only access to IMAP which is more than enough to export your mailboxes to other e-mail providers
- tptacek 4y ago
  This isn't true at all. The reason Google does this is that there is a huge ecosystem of random apps that request full access to people's mail accounts, which are the most sensitive accounts on the entire Internet, and many of those apps were hot garbage that generated a huge account takeover problem. It's perfectly fair to not like the policy (I don't like some things about it), but it's not reasonable to caricature it.
  - malinens 4y ago
    You seem to had never gone trough this approval process. There are many dark patterns there: * Asking for huge amount of money in the of the process (after you wasted hundreds of hours of expensive developer time) * forcing to use restricted scope (read,White,delete) for standard IMAP access even if you need only read-only access or use their proprietary API with read-only scope with unknown rate limits and which is difficult and time-consuming to use * cryptic messages why app is not approved and ignoring arguments made for their questions * often changing APIs (access token formats) They do everything to avoid competition not making better product but by locking existing customers to their ecosystem. And this is not only google. We had similar issues Apple and Meta.
    - tptacek 4y ago
      I am, because reasons, quite familiar with the security review portion of the process. Not only is it mostly/probably not run this way for anticompetitive reasons, but Google doesn't even run the audits; they picked specific auditors (last I checked, Bishop Fox and Leviathan), both of which have gold-plated reputations. Like I said, there are things I don't love about the program. But it's not an elaborate hazing ritual.
paxys 4y ago
Why can't I just enter my own Google client ID/secret on the app and auth against that? There's no approval from Google needed in that case, since I'm not publishing it anywhere.
userbinator 4y ago
Moves like this from Big Tech deeply disturb me. I wonder how long it will be until everything you do with them will require their approval. We're already seeing this in the form of "supported browsers" (i.e. Chrome) and their efforts at painting everything else as "less secure" or "insecure". It's time we started pushing more for freedom over "security", before things become even more dystopian...
masukomi 4y ago
this implies you won't be able to use Firefox, or Safari, to access Gmail unelss they're willing to fork over their source code and pay the $$$ because all of those are >...[an] app that requests access to restricted scope Google user´s data and has the ability to access data from or through a third party server
awinter-py 4y ago
> 'Publishing the app' in Google terms requires a quite astonishing number of steps this is my main gripe with oauth for login -- the amount of overhead for getting it working is absurd. it's not private (goog knows which users use which sites) and there's too much preapproval (company consuming oauth has to have a dedicated google project for each login system) better, simpler system would be 1) client site issues random token, 2) user forwards token to google, 3) google signs bundle with user's email address + client token, 4) client site verifies bundle with goog's public key all communication should be through the client, the three way handshake is absurd and some legit sites don't implement the standard correctly oauth for access federation maybe should require more setup but idk -- I feel like we haven't thought through access federation as a society
relaxatorium 4y ago
I'm sympathetic to the extremely legitimate security concerns here, but it's pretty rich for the "open, we're so open" company to be running a classic 90s Microsoft Embrace Extend Extinguish playbook on SMTP/IMAP which, for all its faults, is one of the more important open protocols undergirding the internet.
- pas 4y ago
  you can use SMTP to access GMail and auth via a web browser based flow (OAuth2/OIDC), this added extra security is needed if there's an additional server in the mix that manages the OIDC tokens for the user > https://news.ycombinator.com/item?id=31421066 https://news.ycombinator.com/item?id=31421066
gunapologist99 4y ago
This is really anti-competitive and designed to keep people from easily migrating their email to another provider. (Google Takeout is a huge pain and not at all live. I guess they see it as their data, their rules..)
_headcrash_ 4y ago
Google sucks

338 points by edward 4y ago

misunderstanding? https://twitter.com/wilbowma/status/1526697254140071936?t=451WHDUawEk-DeC2AecIRQ&s=19 https://twitter.com/wilbowma/status/1526697254140071936?t=45...

jaapz 4y ago
I don't think the problem here is that users can still use app passwords instead of OAuth2 - it's that the developer went through the trouble of developing a OAuth2 implementation, went through the necessary laborious steps to submit the application and then was faced with this message: > The cost of the assessment typically varies between $10,000 -$75,000 (or more) depending on the size and complexity of the application; smaller applications may see costs at a lower threshold of $4,500 That's just bad, but pretty typically google.
- tssva 4y ago
  The Google documentation regarding Oauth access to Google APIs including Gmail explicitly mentions this requirement. It should not have been a shock to the author. Also this requirement only applies if the app is intended to store the data on a server. An email client which directly accesses and locally stores the email would not require a security audit. Pegasus would not be the 1st email client to use OAuth2 with Gmail and others have not required an audit. Some of the newer email client services which implement advance features by downloading email directly from Gmail to their own servers on the backend to do processing of the email would require a security audit.

solar-ice 4y ago

> and has the ability to access data from or through a third party server So Pegasus Mail accesses your email from their servers? For reference, they have an entire flow designed so that the auth credentials never touch the app developer's server for desktop and mobile apps - https://developers.google.com/identity/protocols/oauth2/native-app https://developers.google.com/identity/protocols/oauth2/nati...

easton 4y ago
I’m super confused about this too, there’s tons of open source apps that use Sign in with Google. Shoot, I’ve used the SDK myself in little apps and never saw this.
- rjzzleep 4y ago
  The SDK on android or ios you mean? Cause that would be vastly different from adding a web authentication flow such as oauth to a legacy application.
- Alex3917 4y ago
  "Sign in with Google" is just a button that apps are required to use when they allow the user to interact with Google in some capacity, but don't have their own account management system. Most apps with that button don't have the ability to get any data from Google other than your name and email address though; for apps where you are granting OAuth access to your Gmail, it is very obvious that you are doing so, so it's not likely that you have done it accidentally in the past. (Although you can obviously check on myaccount.google.com)
snthd 4y ago
Most mail clients can connect to both a google email account and a "third party server" email account. It's easier to rule out undetectable-by-google data exfilteration if the app can only connect to Google. The obvious way around this is to make a Google-only edition. Yuck.
- solar-ice 4y ago
  I don't think that's what Google are saying here. I think the sentence is referring to Google user data; as long as the Google user data or credentials to access it does not touch another server, the entire thing does not apply. In fact, in Google's guidance on this subject, they say: > Local client applications that only allow user-configured transmissions of Restricted Scope data from the device may be exempt from this requirement [to get a Letter of Assessment]. And in another FAQ: > Local Data Storage: Local client applications don't need to undergo a security assessment because data is run, stored, and processed only on the user's device. Local client applications that only allow user-configured transmissions of Restricted Scope data from the device may be exempt from this requirement. My feeling is that the author of Pegasus Mail has checked a checkbox incorrectly somewhere, or alternatively has not implemented the desktop oauth2 flow correctly.

tatoalo 4y ago

> [...] including even having to prepare a Youtube video showing my code in action. What?!

uniqueuid 4y ago
This is unfortunately not unprecedented. When you apply for API access to facebook's graph API, you also need to document your app including videos.
Alex3917 4y ago
The video isn't really that hard. You just make a screencast of yourself downloading email while saying "now I'm using the download email scope", and then sending email while saying "now I'm using the send email scope." They don't care if the videos are terrible quality, it's only for them to understand what they're approving, and to have some documentation if a developer later changes an app to do something nefarious.

acatton 4y ago

I'm confused… Why do they want to support the OAUTH flow when they can just the normal app password flow without any change in their code, just documentation for their users. https://support.google.com/accounts/answer/185833?hl=en https://support.google.com/accounts/answer/185833?hl=en

rob74 4y ago
Maybe because the first line on the page you linked reads: > Tip: App Passwords aren’t recommended and are unnecessary in most cases. To help keep your account secure, use "Sign in with Google" to connect apps to your Google Account. So, for Google, App Passwords are clearly "second class citizens", deemed insecure and not recommended. And, as an app developer, you probably don't want to be seen as recommending an insecure authentication method?
- baisq 4y ago
  I assume that users of Pegasus Mail are advanced enough to know that that is bullshit.
- NoGravitas 4y ago
  This also strongly implies to me that Google is going to drop support for App Passwords in at most a few years.
kevincox 4y ago
The OAuth flow is better. It is easier for users and supports many more methods of authentication. Really the biggest downside about the OAuth flow is that it requires a relationship with Google (a client key). If this could be a fully decentralized standard it would be fantastic.
sneak 4y ago
If you have advanced protection enabled on your Google Account (and you should) then app passwords are no longer available.

Tepix 4y ago

Once again, i'm glad not to be using GMail.

enriquto 4y ago
Yep. All these anti-google complaints sound so useless. What did they expect? These issues are always an entirely self-inflicted problem. Google's terms of service are public, clear, and completely unacceptable. Why people keep using their services is beyond me.
- blagie 4y ago
  Legacy. For example, my email is in Google. There are things related to e.g. evidence for litigation from many years back. I have documents shared with me in Google Docs. Google used to be pretty good about "don't do evil." I've degooglified what I can, but I can't degooglify 100%.
  - Tepix 4y ago
    It's never too late to switch.
    - blagie 4y ago
      It kind of is. If I want to have legal proof of when I shared a document with someone, I need my Google Docs. I guess what I should do is stop putting new stuff in Google.
- hackmiester 4y ago
  It seemed like a good idea in 2005, and switching costs. But yeah, I bit the bullet and left this year.
- xigoi 4y ago
  It's not easy to convince everyone to send mail to your new address. Especially the countless random websites where you signed up with the old one.

mbwgh 4y ago

I had a similar brick wall moment after having written a tool to sync Google Calendar events with the `when` calendar tool (http://www.lightandmatter.com/when/when.html http://www.lightandmatter.com/when/when.html). Which is why I never published it (although I have been using it for almost two years), and setting it up requires a series of laborious steps of activating the Google Developer Console, creating API keys etc., which I found just to embarrassing to put in a README :)

VTimofeenko 4y ago
I don't think it would be embarassing. Other tools use the same approach, like rclone: https://rclone.org/drive/ https://rclone.org/drive/

janandonly 4y ago

I am wondering if it is really legal for Google to even ask any money from 3rd party developers to use GMAIL (what should be) open standards like SMTP and IMAP. I guess I am so naive...

richsouth 4y ago
It might be immoral, but if you're trying to make money using their service I can't see there being a legal reason to prevent them doing this. I meant, they (Google) need the money right? Companies like this seem to get to the point where they feel justified in charging for every little thing that wouldn't kill them to give for free.
- derbOac 4y ago
  If you're in the EU: https://en.m.wikipedia.org/wiki/Antitrust_cases_against_Google_by_the_European_Union https://en.m.wikipedia.org/wiki/Antitrust_cases_against_Goog...

nickdothutton 4y ago

Any takers for a bet on when Google will axe App Passwords?

CyanBird 4y ago
I'd say, no more than 3 years

qwerty456127 4y ago

I have one question in this regard: will I still be able to access my mail through my own script I myself wrote? I understand I will probably have to make some changes and click some things in GMail settings but is this still going to be possible or will I too have to "publish app" even if I only mean it for my own private usage?

vivekv 4y ago
For now app passwords
mikeklaas 4y ago
Development apps are exempt from the requirement. (They are also limited to 100 accounts)
- cute_boi 4y ago
  But i find the token problem frequently. It seems it expires after 2-3 days. I don't know why :(

sneak 4y ago

Users should not be subjecting all of their correspondents' communications to US warrantless surveillance anyway; Google is doing the world a service by making their service harder and harder to interoperate with. Gmail and other huge centralized points of censorship and surveillance must be destroyed. If you are a user, move away. If you are a developer, do not support these closed systems.

morganvachon 4y ago
I agree wholeheartedly in principle, but the reality is that it's nearly impossible to escape Google getting their hands on yours and your correspondents' messages at some point. So many businesses (and third party email providers!) use Google's services on the back end, and many relays out there are owned by Google/Alphabet even if it's not readily apparent, that the only way to even attempt to avoid them is using encrypted email. Unless everyone you communicate with (including companies, governments, and other organizations) is willing to only communicate via encrypted email as well, it's a lost cause.
- sneak 4y ago
  No, the perfect is the enemy of the good here. "Email is insecure anyway and Google is big, therefore it's ok to give the adversary 100% of the plaintext" is not sound logic. This is not some principled stance, it is a clear and practical command: stop using these services. If your personal email address ends in @gmail.com, you are doing it wrong.
  - morganvachon 4y ago
    I'm not saying you shouldn't give up Google services; I did. I'm just saying the unfortunate reality is that you'll never fully escape them.
tptacek 4y ago
Moving your data out of US jurisdiction doesn't shield it from US warrantless surveillance, but rather maximizes its exposure. The USG ostensibly requires due process (a warrant, whatever) to obtain information from US servers. It does not require any due process to obtain data from foreign servers. Coercively obtaining data from foreign servers is literally the chartered job of the NSA (of all signals intelligence agencies around the world, really). This is a super common misapprehension people have about data locality and surveillance, and it's no surprise that marketing plays off it ("keep your mail in Switzerland, so it's subject to Switzerland's strict privacy laws!"). There are other reasons to house things in Switzerland (or the Lesser Antilles or wherever). I'm not saying you should use GMail (unless security is a concern of yours, in which case you should use GMail).

Fizzadar 4y ago

I’ve gone through this process for my email client Kanmail [1]. The third party audit is not required for email clients that run on end users computers and store credentials locally. By the looks of it Pegasus falls into this category and should not have any issues getting approved (still need the YT video and such but the Google team are surprisingly responsive and helpful in my experience). [1] https://kanmail.io https://kanmail.io

undecisive 4y ago
The wording seems to imply that if, on a yearly whim, someone at Google decides to "empanel" a security assessment team, you have no choice, you will not necessarily be asked permission and you will be paying that invoice whether or not you needed that assessment. Do you have evidence - in writing - to the contrary from a Google official? Abridged wording and my non-lawyer interpretation below in case I'm not clear: > Every app that [accesses Gmail and also accesses other servers] is required to go through a security assessment from Google empanelled security assessors. [...] > In order to maintain access to restricted scopes, the app will need to undergo this security assessment on an annual basis, [... costs usually] between $10,000 - $75,000 (or more) [...] > This fee may be required whether or not your app passes the assessment and will be payable by the developer."
- dal 4y ago
  So basically when your application resolves gmail.com you will access other servers. :P
  - undecisive 4y ago
    Well, certainly they haven't ruled it out. DNS servers are third-party servers. VPNs are third-party servers. They can be as kind or as nasty about this as they like, For all my fear mongering, I should point out that the only reason Google are saying this is to cover their backs when they decide to levy the maximum fee on an unsuspecting competitor. If they don't consider you a direct competitor, you might be ok. They have no reason to use this policy to alienate the majority of desktop applications that connect to email. But they also have no repercussions if they do.

wiradikusuma 4y ago

Does anyone has a link for that ToC? I'm curious what kind of use cases that need such payment. E.g. if I want to make my app to use Google login, I think it's completely free?

corentin88 4y ago
Not ToC but that’s where you will find all the details of the security assessment: https://support.google.com/cloud/answer/9110914?hl=en https://support.google.com/cloud/answer/9110914?hl=en

corentin88 4y ago

I believe many comments here will criticize Google. But objectively, Google is at its best here: - in terms of privacy, applications that have access to your Gmail inbox now require a security audit. - the audit is not required for MVP (<100 users) or applications internal to a Google Workspace org. Of course, you have to pay for the audit. But: - it’s only required when you ask for restricted user data (i.e. reading my emails). - Google doesn’t take 30% of your revenue to use its API - which is free by the way. To me, Google has created the perfect world for developers here. And that says a lot when I read the developer of Pegasus Mail doesn’t want to record a 2min long YouTube screencast to get approved. Also, just wanted to kudos the Google OAuth team that has greatly improved their process in the past years. If you follow the guidelines, you can get approved within a day now.

lozenge 4y ago
Is it my inbox or Google's inbox?
- dspillett 4y ago
  It is part of Google's tracking and advertising system. They let you use it as an inbox.
- dangrigsby 4y ago
  Is it your car or the government’s?
- charcircuit 4y ago
  Google will be blamed if someone's data is misused even if that person authorized giving permissions to that app. If you want evidence supporting that, take a look at how much flak Facebook received with the Cambridge Analytica scandal.
- scrapheap 4y ago
  If it's Gmail then it's Google's inbox, they're just letting you use it.
4y ago

adrianpthomas 4y ago

Went through this process with Mail Designer 365. Luckily as we only need sending access, the requirement was less arduous, but the process was not great. After submitting it took ages to get reviewed and even when it had been processed the status wasn't entirely clear. At the time it was made clear that using app passwords was considered a risk and users were receiving emails about it being a security risk.

throwaway787544 4y ago

Pegasus Mail, wow!!!! There's a blast from the past! I remember using this over 20 years ago!

gommm 4y ago
Same here :) It was my client of choice back 20 years ago when I used windows... Very nice to see that it still lives on
dugmartin 4y ago
We used it on our LAN around 1993 hooked to a local mail server (which ran on a Windows box using Netware I think - it was sitting on a filing cabinet next to my desk) that in turned fetched mail from the campus mail server. It worked really well and the local mail server let us easily share/email large attachments within the department as they never left the LAN.
djbusby 4y ago
In a DOS terminal-window on my NT4 box.
pantulis 4y ago
The Good Pegasus.
tpmx 4y ago
Yeah! It was my first ever email client back in 1994/1995. More details on the developer: https://en.wikipedia.org/wiki/David_Harris_(software_developer) https://en.wikipedia.org/wiki/David_Harris_(software_develop...

dewey 4y ago

There's a bunch of third party "indie" email apps, how are they doing it? Is this potentially just some boilerplate text and they evaluate the rate based on the size of the project in the end? I'm currently using https://mimestream.com/ https://mimestream.com/ and I somehow doubt all of them had to pay that kind of upfront money.

NickRandom 4y ago

I haven't seen a single comment (at time of posting this) touching on the cost aspect The cost of the assessment typically varies between $10,000 -$75,000 (or more) depending on the size and complexity of the application As he says, even if Google classed his as a small app (unlikely) "Regrettably, these kinds of fees are far beyond what I can afford, given that I rely on donations from my users to make ends meet: even the $4,500 fee is well beyond what I could find on an annual basis. " I get that doing proper auditing is expensive and the costs are within industry rates but to me it reeks of slowing turning the screws by boiling the frog on the slow path to monetization. (many mixed metaphors I know)

jijji 4y ago
It looks like this pay-for-play method is going to have a chilling effect on any developer creating an alternative Gmail interface. Same with removing IMAP.

xbar 4y ago

And so it is time to relegate my Gmail account to an auto-forwarder.

tut-urut-utut 4y ago
If you own your mail domain, it's the time to move off from Gmail. If not, make a clear cut, purchase domain, and never ever be at the mercy of mail provider.
- mawise 4y ago
  This is so hard. I bought a domain and even started paying for (non-google) email on it over a year ago. But I can't bring my self to start the painful process of migrating everything.
  - r00fus 4y ago
    What it'll take is likely a cost/benefit for keeping your non-gmail-website workflow (and taking the leap to jump off gmail) vs. complying and using their (mostly usable) webmail site. They likely figure most will choose option#2 since it's so painful/expensive to do #1.
  - hackmiester 4y ago
    At least you will only have to do it once. I migrated from G Suite to Fastmail, but because I had a custom domain, it was basically entirely painless.

Saris 4y ago

Is gmail removing IMAP too? Why support OAUTH at all when you can use the normal method of logging in with IMAP and an app password?

deleted 4y ago
[deleted]
nemothekid 4y ago
They are likely removing IMAP. IMAP support is already disabled by default in Gmail
- Mailtemi 4y ago
  Can you share а link for this statement. As far as I know IMAP works ok, you just need to use XOAUTH IMAP authentication.
- Saris 4y ago
  If that happens, it sounds like a good chance to move away from gmail to something more robust.
planede 4y ago
App passwords are only available for 2fa enabled accounts, while OAUTH is available for everyone. So far I didn't get a clear answer if "the end of less secure authentication methods" means the end of app passwords as well, or only the end of signing into IMAP with your main google credentials.

sigzero 4y ago

Why the hell would they need to submit to that?! That's crazy.

sir 4y ago

If you don't have the time or resources to go through this process you can also use this simple proxy of mine, which intercepts the non-OAuth IMAP/SMTP commands and adds OAuth in the background: https://github.com/simonrob/email-oauth2-proxy/ https://github.com/simonrob/email-oauth2-proxy/

dividuum 4y ago
Ah yes. The "Just use FTP + curlftpfs + SVN" approach for solving this problem ;-) That really doesn't help OP as their customers most likely won't be capable to run that.
- sir 4y ago
  Oh, it's certainly not an excuse for Google requiring this process. However, if you do want to use Pegasus Mail (or any other non-OAuth app/script/email tool, etc) after the end of May, this is a way to do so. I think it's quite a safe assumption that many of the HN audience are capable of using it, too ;-)

londons_explore 4y ago

I was aware of the security-audit-at-your-own-expense requirement, and I am not a gmail app developer, I just did a drive by of the documentation 1+ year ago... As a gmail user, this actually seems like a sensible thing. Gone are the days when users are trusted to just click 'allow' to let some random third party access all your data. It turns out, users don't fully understand the ramifications of that, and complain loudly when the 'free personality quiz' they allowed goes and downloads every private email to use for marketing. So, if I can't give access to my mail to a random untrusted third party, the next best thing seems to be to give access to someone who has at least had their systems audited and who at least has enough money to pay for an audit, meaning they probably have some business plan and can be tracked down by the courts if they mishandle the data.

superkuh 4y ago

Note that this "OAuth2" support is only for gmail. OAuth 2 is not really an open protocol like OAuth (1) was. It's just something the megacorps shoved down the IETF's throat and every implementation is different enough to need different software. OAuth2 is not good for the health of the internet and will lead to even more lock in. When Google disables imap at the end of the month for gmail (only allowing OAuth2+imap, which is not imap) that's the end of gmail for me.

jeroenhd 4y ago
OAuth is hardly Google only, it's part of RFC7628. There are very good reasons why a simple password login isn't necessarily what you want for your email protocol. IMAP and 2FA are often at odds, for example, usually leading to vulnerable application passwords that bypass security requirements out of necessity. I don't think many email clients support OAuth2 (Thunderbird does, but that could be Gmail specific?) but the concept isn't inherently bad, in my opinion. I don't really see the problem with OAuth2 itself, the email space is just very very slow at accepting new standards. Dovecot and UTF8 email addresses still don't play well together, for example. The lack of proper on-the-fly application registration for mail servers is annoying, but supporting them should be easy enough if the mail service isn't run by complete doofuses.
- superkuh 4y ago
  >OAuth is hardly Google only, To be clear, I never said OAuth2 was google only. I said the standard OAuth2 is so much not a standard that an implementation written for Google's imap+OAuth2 will not work for any other email provider's imap+OAuth2.
- yxhuvud 4y ago
  > Thunderbird does, but that could be Gmail specific No, it definitely works on Office 365 too.
fivre 4y ago
OAuth2 is a suite of protocols and standards you can use to build an auth system from people who have been working on auth systems for a while. it isn't strictly designed for federated auth, and arguably is targeted at internal systems because there are so many parts are "you need a system that does this, but how is up to you" OIDC builds on OAuth2 to fill in those gaps with prescribed details and add additional systems necessary for a workable federated auth system. it _should_ be possible to use it entirely provider-agnostic fashion, but in practice it's like any complex protocol with optional parts in that implementations have incorrect bits or idiosyncrasies (why does Azure's provider sometimes serve public keys it doesn't actually use to sign tokens in some modes? god only knows) and you'll probably only provide integrations for a subset of providers, not just let anyone with a discovery link show up and use it.

nullcipher 4y ago

Wow, is this the software that ran on Novell Netware? I had no idea this was still around.

malinens 4y ago

There is only one thing why this is done: to make impossible to move from gmail (also with custom domains) to another provider. We have the same problem. We spent hundreds of hours implementing their legacy auth (less secure apps), gmail oauth2 and domain-wide delegation to be later stuck in approval process. Until our app is not approved users can not use e-mail import/migration tool to our inbox.eu service. And there is lot of demand for that because people do not want to pay so much for e-mail. Shame on google... they also do not provide reas-only access to IMAP which is more than enough to export your mailboxes to other e-mail providers

tptacek 4y ago
This isn't true at all. The reason Google does this is that there is a huge ecosystem of random apps that request full access to people's mail accounts, which are the most sensitive accounts on the entire Internet, and many of those apps were hot garbage that generated a huge account takeover problem. It's perfectly fair to not like the policy (I don't like some things about it), but it's not reasonable to caricature it.
- malinens 4y ago
  You seem to had never gone trough this approval process. There are many dark patterns there: * Asking for huge amount of money in the of the process (after you wasted hundreds of hours of expensive developer time) * forcing to use restricted scope (read,White,delete) for standard IMAP access even if you need only read-only access or use their proprietary API with read-only scope with unknown rate limits and which is difficult and time-consuming to use * cryptic messages why app is not approved and ignoring arguments made for their questions * often changing APIs (access token formats) They do everything to avoid competition not making better product but by locking existing customers to their ecosystem. And this is not only google. We had similar issues Apple and Meta.
  - tptacek 4y ago
    I am, because reasons, quite familiar with the security review portion of the process. Not only is it mostly/probably not run this way for anticompetitive reasons, but Google doesn't even run the audits; they picked specific auditors (last I checked, Bishop Fox and Leviathan), both of which have gold-plated reputations. Like I said, there are things I don't love about the program. But it's not an elaborate hazing ritual.

paxys 4y ago

Why can't I just enter my own Google client ID/secret on the app and auth against that? There's no approval from Google needed in that case, since I'm not publishing it anywhere.

userbinator 4y ago

Moves like this from Big Tech deeply disturb me. I wonder how long it will be until everything you do with them will require their approval. We're already seeing this in the form of "supported browsers" (i.e. Chrome) and their efforts at painting everything else as "less secure" or "insecure". It's time we started pushing more for freedom over "security", before things become even more dystopian...

masukomi 4y ago

this implies you won't be able to use Firefox, or Safari, to access Gmail unelss they're willing to fork over their source code and pay the $$$ because all of those are >...[an] app that requests access to restricted scope Google user´s data and has the ability to access data from or through a third party server

awinter-py 4y ago

> 'Publishing the app' in Google terms requires a quite astonishing number of steps this is my main gripe with oauth for login -- the amount of overhead for getting it working is absurd. it's not private (goog knows which users use which sites) and there's too much preapproval (company consuming oauth has to have a dedicated google project for each login system) better, simpler system would be 1) client site issues random token, 2) user forwards token to google, 3) google signs bundle with user's email address + client token, 4) client site verifies bundle with goog's public key all communication should be through the client, the three way handshake is absurd and some legit sites don't implement the standard correctly oauth for access federation maybe should require more setup but idk -- I feel like we haven't thought through access federation as a society

relaxatorium 4y ago

I'm sympathetic to the extremely legitimate security concerns here, but it's pretty rich for the "open, we're so open" company to be running a classic 90s Microsoft Embrace Extend Extinguish playbook on SMTP/IMAP which, for all its faults, is one of the more important open protocols undergirding the internet.

pas 4y ago
you can use SMTP to access GMail and auth via a web browser based flow (OAuth2/OIDC), this added extra security is needed if there's an additional server in the mix that manages the OIDC tokens for the user > https://news.ycombinator.com/item?id=31421066 https://news.ycombinator.com/item?id=31421066

gunapologist99 4y ago

This is really anti-competitive and designed to keep people from easily migrating their email to another provider. (Google Takeout is a huge pain and not at all live. I guess they see it as their data, their rules..)

_headcrash_ 4y ago

Google sucks