<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=1822615684631785&amp;ev=PageView&amp;noscript=1"/>

Five annoying issues with Google’s OAuth Scope Verification

It’s 2020, when Google promised to shut off many third-party apps that weren’t verified by December 31, 2019. While I haven’t heard any reports of anyone being shut off yet, having been through Google’s OAuth verification process, and having run some Unverified Apps, I’ve learned the ins and outs of the whole process. All of these insights I’ve concluded on my own, thanks to Google’s insufficient documentation.

1. There’s no hierarchy to the API Scopes, and that’s just ridiculous

Let’s say you’ve been approved for the almighty https://mail.google.com Gmail API scope, which gives you full access to read, write, and delete data in a user’s Gmail account. Then later, you realize you don’t need this almighty scope — you determine you just need modify-only access to Gmail. You also want the benefit of a less scary warning for your user. You might think that you can just switch your OAuth code to use the less permissive scope, https://www.googleapis.com/auth/gmail.modify, and you’ll be good to go. That makes logical sense, after all.

But you’ll be shocked to learn that you’ve just re-entered “Unverified App” territory.

 

That’s right. Internally, Google doesn’t know that if you’ve been approved for the mail.google.com scope, which covers everything, you should also be approved for the “modify” scope, which grants a subset of permissions of the mail.google.com scope. There’s no hierarchy to the scopes as far as Google is concerned, which is absolutely ridiculous given that Google’s main business is to organize the world’s information. Funny that Google is so good at organizing trillions of web pages but is terrible at organizing a few hundred of its own API scopes.

2. Where exactly do you designate the scopes you want to use?

The whole OAuth process might be confusing to you because you’ve likely noticed that there are three places to specify the API libraries you want access:

  1. In the Cloud Console, under the API Library section
  2. In the Cloud Console, under your specific OAuth consent screen
  3. In your code where you hit the Google OAuth2 endpoint

So which list is most important? Do you have to specify the scopes in the Cloud Console consent screen settings for your code to request access to that scope for a user? No, you don’t.

Here’s an example. In the Cloud Console for my GMass app, here’s the list of Enabled APIs:

 

My OAuth client consent screen lists these scopes:

In code, I can request access to a scope that’s not listed in either the Dashboard or my OAuth client page, and it might still work, depending on the scope that I add. For example, I can add the “https://www.googleapis.com/auth/genomics” scope, which has nothing to do with my project, and the user can still authenticate but IS presented with the Unverified App screen.

        private static readonly string[] GMass_Scopes = new[] {
        "https://mail.google.com",
        "https://www.googleapis.com/auth/userinfo.email",
        "https://spreadsheets.google.com/feeds",
        "https://www.googleapis.com/auth/genomics"
        };

        public ActionResult Login(string redir = null, bool connect = false, string emailaddress = "", string source = "extension")
        {
            OAuthState state = new OAuthState
            {
                Connect = connect,
                Redir = redir,
                Source = source
            };

            UriBuilder uri = new UriBuilder();
            uri.Scheme = "https:";
            uri.Host = "accounts.google.com";
            uri.Path = "/o/oauth2/auth";

            var qs = HttpUtility.ParseQueryString("");
            if (emailaddress != "")
            {
                qs["login_hint"] = emailaddress;
            }
            qs["access_type"] = "offline";
            qs["response_type"] = "code";
            qs["approval_prompt"] = "force";
            qs["client_id"] = OAuthHelper.ClientSecrets.ClientId;
            qs["redirect_uri"] = "https://extension.gmass.co/OAuth/AuthCallback";
            qs["scope"] = string.Join(" ", GMass_Scopes);
            qs["state"] = JsonConvert.SerializeObject(state);
            uri.Query = qs.ToString();

            return Redirect(uri.ToString());
        }

But, if I add the “https://www.googleapis.com/auth/drive.file” scope, also which is not enabled in Cloud Console’s API Library and not listed as part of my OAuth consent screen, then the user sails right through when logging in, and does not see the Unverified App screen. The “genomics” API isn’t documented as sensitive or restricted, so this difference doesn’t really make sense.

The rules are as follows:

  • An API scope only needs to be listed as part of your OAuth consent screen if it requires verification to be used. And to even get it to appear on this settings page, the API Library for that scope needs to be enabled.
  • If an OAuth scope does not require verification, or you choose to go Unverified, then the scope does not need to be listed on the Consent Screen settings, and your code can still request use of the scope, as I requested the use of the genomics scope in my code sample above.
  • If your code doesn’t request access to a scope, and then attempts to use the API for that scope, you’ll get this error from Google:
    Google.Apis.Requests.RequestError\r\nInsufficient Permissions: Request had insufficient authentication scopes.
  • If your code does request access to a scope, is granted access by the user, and then attempts to call an API for that scope, you will get an error if the API Library isn’t enabled in the Cloud Console.

The conclusion is that the scope need not be listed on your consent screen settings in order for you to request it, but the API Library does need to be enabled in order for your code to use the API once consent is granted by the user.

Bonus: What happens if your app has been verified and you need to add a scope later?

I contacted the Google OAuth team and asked them this very question. Here is the answer:

The steps to add a scope later are:

  1. Add the scope to your OAuth consent screen, and hit either “Save” or “Submit for Verification” if it’s a sensitive or restricted scope.
  2. The scope will now appear with the yellow warning sign.
  3. Do not modify your production code to use the scope. As long as your production code only requests the scopes that have been approved, your users won’t see the Unverified app screen. So, just the act of adding a scope to your consent screen doesn’t alter what your user sees when going through the OAuth flow.
  4. You’ll likely be contacted by the OAuth team requesting a YouTube video demonstrating the necessity for the new scope.
  5. After you answer their questions and are approved for the new scope, then it will show up as green in the consent screen settings, and only then should you add the scope to your production code OAuth flow.

3. Sensitive Scope? Restricted Scope? Google’s documentation is god-awful.

In the OAuth Verification FAQ, Google discusses sensitive scopes vs. restricted scopes. Yet while the genius who wrote this listed the restricted scopes, he did not list the sensitive scopes. In response to the question “What are sensitive API scopes?”, the page says:

Sensitive scopes allow access to Google User Data. If an app uses sensitive scopes, it must comply with the Google API User Data Policy and have its OAuth consent screen configuration verified by Google.

The app verification process can take anywhere from 3 to 5 business days.

To add to the confusion, in the Cloud Console, you might see this for your project under the “OAuth consent screen” settings:

 

Notice Google’s mistake here? They’ve flagged the gmail.insert and gmail.readonly scopes as “sensitive scopes,” except that they’re not actually sensitive scopes. They are “restricted scopes,” according to the FAQ. If that’s not enough, in Google’s own master scope documentation, where all the Gmail API scopes are listed, there’s not a single mention of some of them being sensitive or restricted.

You might think the main scope list would be the perfect place to specify what’s sensitive, restricted, or neither, but you’d be wrong.

There is one place where you can definitely tell whether a scope is sensitive or restricted, and it’s here. All other Google pages fail to tell you, or they give you incorrect information.

4. Want to mark your app as “Internal Only?” There’s a bug in the Cloud Console.

The docs say that an owner of a Cloud Console project can mark it “Internal Only,” but I haven’t found that to be the case. In one of my currently Unverified Apps, SearchMyEmail.com, the ability to set the project INTERNAL is disabled, telling me that I can’t mark it INTERNAL because I’m not a G Suite user.

Except that I am. I’m logged in with my [email protected] G Suite account, which is an OWNER of this Cloud Console project. I suspect there’s a bug here that only allows the CREATOR of the Cloud project to mark the project “INTERNAL.” In my case, I created the project with an @gmail.com account. For now, that means I’m stuck in circular logic. If I login to the creator account, which is a gmail.com account, I can’t mark the project INTERNAL because I’m not a G Suite account. If I log in to my G Suite account to mark the project INTERNAL, I can’t because I’m only the owner, not the creator.

5. Comfortable with being Unverified? It’s not as easy as it sounds.

You may choose to forego the whole verification process for a number of reasons. Perhaps you don’t want to pay for the security assessment. Perhaps your app is just for internal use. Perhaps you are under the 100-user threshold for requiring verification. You may choose to go Unverified thinking your users just won’t care whether you’re Verified or Unverified because they trust you. That’s all well and good, but if you choose to go Unverified, there are two things to know:

First, Google intentionally makes it difficult for a user to bypass the “Unverified App” screen. Here’s the standard Unverified App screen that users see:

It informs the user that if they trust the app, they can “proceed.” But just how does one proceed? The word “proceed” isn’t linked. There’s no button that says “proceed.” Instead, the user has to click “Advanced,” and only then is the user given the option to proceed, after noting in parentheses that doing so is unsafe. Holy moly. Talk about scaring a user unnecessarily. Or should I say, scaring a developer into paying $15,000 – $75,000.

Here’s my recommended approach. On my app SearchMyEmail.com, which I’ve decided to let remain “Unverified,” we’ve built an animation showing the user exactly how to bypass the “unverified” nonsense.

Feel free to copy this technique. Just go to SearchMyEmail.com, choose to sign up as a “boss,” and then view the source for the interstitial page that has this animation.

Secondly, everything with Google OAuth is inconsistent. I should say, the only consistency with their OAuth rules is their inconsistency.

Check this out:

Sometimes, and it’s unclear when, being Unverified results in a user being unable to connect, regardless of what they do. In my SearchMyEmail.com app, two supposedly equivalent Gmail accounts, [email protected] and [email protected] behave entirely differently when authenticating into SME.

Since SME is Unverified, and since these are both @gmail.com accounts, they’re both likely to be shown extra warnings. This is what happens when [email protected] authenticates:

I’m told the app is unverified but given the option to continue.

But this is what happens when [email protected] authenticates:

I’m told the app is unverified but NOT given the option to continue.

What’s the difference between the two @gmail.com accounts? The only one I can think of is that [email protected] was created 10 years ago, and [email protected] was created 10 hours ago, but nowhere in Google’s OAuth docs is it stated that account age makes a difference.

Even weirder, I have two apps that I’ve decided to leave Unverified, Wordzen and SearchMyEmail. Here are the Cloud Console config screens from each:

SearchMyEmail’s Consent Screen

SearchMyEmail’s Scopes

Wordzen’s Consent Screen

Both show the status as “pending security assessment.” BUT, when you go into the App Details:

It says “mail.google.com” behind the tooltip.

Well, look at that! The full https://mail.google.com scope is approved and granted. Yet the app is still labeled “pending security assessment.” I promise you Wordzen did not undergo the Security Assessment required by restricted scope apps. How did that happen? A Google miracle, perhaps. Update update update! It’s now January 25, 2020, and I spoke too soon. As I detailed on my live updates page, on January 13, I received notice from Google that Wordzen missed the deadline and use of the https://mail.google.com scope is now considered “unverified”.

Please, Google, fix this giant mess. But leave Wordzen’s awesome unfettered access alone (Never mind, they removed this access).

See why GMass has 300k+ users and 7,500+ 5-star reviews


Email marketing. Cold email. Mail merge. Avoid the spam folder. Easy to learn and use. All inside Gmail.


TRY GMASS FOR FREE

Download Chrome extension - 30 second install!
No credit card required
Love what you're reading? Get the latest email strategy and tips & stay in touch.
   


11 Comments
  1. Thank you for posting this. I thought I was going crazy for a second trying to understand these confusing guidelines and circular paths Google has created between the developer console, cloud console, and online help pages.

    I wish I would have understood all of this better before I submitted my app for verification. It’s been MONTHS since I submitted my details (possibly screwing this up since it was difficult to understand the process and the requirements) and I still haven’t heard anything from them. I also don’t know how to reset or restart the verification process. At this point, I’m not sure I even want to.

    Do you know if there’s any way to contact Google’s security assessment team to correct issues or reset my submission?

  2. Also ran into the issue saying I’m not a G Suite user. Turns out some projects are tied to your G Suite organization and some are not. The easy fix is to create a new project and when you do, assign it to your G Suite org.

  3. And you can’t get rid of it when you’re sick of trying to get it to work…at least not that I’ve been able to figure out. Between this, and Amazon SES, I spent ALL DAY trying to get incoming and outgoing email to work….and finally bailed back to the little cloud VPS it was already on. Home, sweet little home.

    Thanks for posting all this…I too thought I was going crazy trying to set things up.

  4. Google is like the U.S Government – Difficult and unorganized, yet they expect us to follow “their rules”, which they themselves DON’T follow…. What a bunch of crap!

    This is what happens when a company gets “too” big, which is something I NEVER thought I would say as a business owner and entrepreneur….

  5. This was pretty much my experience as well. Unorganized, unclear and even when you go through the verification process their communication was terrible. I had to submit a YouTube video 3 times and answer the same questions multiple times, it took us 6+ months to get through, only to have the app still be listed as unverified because we chose not to go through the very expensive security assessment.

    Additionally, their documentation states that if you don’t go through the security assessment, you can still get your app listed on the GSuite Workspace Marketplace, however each time I submit the app to get listed, it is immediately denied due to the app not being verified.

    Google – this is a disaster.

  6. Hi, I wrote a python app about 8 months ago that updates a google sheet in my google account. I created the project in the google cloud console, enabled the sheets api library, filled in the oauth consent screen and created credentials.
    And it all worked for months. The suddenly it no longer works.
    So I created a new project, redid all the steps … the one thing that seems to have changed is that first time round I am sure i was allowed to tick internal app now I can’t as it says I am not a gsuite user.
    Anyway, the behaviour I am getting is strange. When I run the app now I get the google sign in screen. After signing in I get taken to the “google hasn’t verified this app:” screen. When I hit continue on that screen I get taken back to the sign in screen – thus placing me in an infinite loop. I am very frustrated. Can anyone help?

  7. Hi Ajay , You have explained wonderfully I have submitted for verification of sensitive scopes and the problem is that the Oauth is working on localhost but not when I publish the application on server … there is no change in the code only the URL changes … I have verified the domains and added callback URL for both text and publish … I do not receive the access token although I received code and state , when the access token API is called form the same code it doesn’t return anything when published… Although working through localhost … Can you tell what may be the reason?

  8. Sir I’m stuck into very critical condition, I crrate gmail api credentials using my gsuite account and now when I send email and follow up them my email get disabled by Google so is there any condition or I have to remake this using Gmail account

Leave a Reply

Your email address will not be published. Required fields are marked *

Transform your Gmail account into an email marketing powerhouse

GMass is easy to learn and easy to use — but brings unbelievable email power into Gmail

Try GMass for free Then check out the quickstart guide to send your first mail merge email in minutes!

GMass

Share This