Google Search Console API - Java Example

Background

Google Search Console is an online tool provided by Google to manage how your site relates specifically to people searching with the Google Search Engine. The information it offers is somewhat similar to a subset of what Google Analytics provides but allows you to submit sitemaps and perform other indexing-specific operations not available in Analytics.

In my workflow I mostly use the Search Console as a quick overall snapshot to see how the site as a whole is performing on the search rankings and what search terms people are using to get there. As I like to automate operations I find myself doing manually on a regular basis, I thought it would be useful to put together a small checking program that would notify me when the new stats had been made available instead of checking the page periodically.

Google provides a REST API for Search Console along with several language libraries that simplify interacting with it. There are quickstart guides for python and Java, however I found the Java examples somewhat lacking and as a result decided to put together this post to help augment the documentation Google provides.

Below are instructions on how to set up and access the Search Console API using a service account authenticated Java program.

Note: This example depends on your site already added in the Search Console, official instructions on how to do that are available here

Setting Up The Service Account

Your application will need to authenticate its requests in order to perform operations against sites you have set up in Google Search Console. A comparison of the different Google authentication schemes is beyond the scope of this post, but suffice it to say a service account is a pseudo-user account meant for applications that can be given access to services similar to the process for a real user account.

You can create a service account for your application by going to the API Credentials page and clicking on Create Credentials then selecting service account.

You'll then be able to set the name of your service account along with some other optional settings.

At the end of the process you will be able to download the service account JSON credential file that will be needed when running the example below. Note: This file will allow access to any services that have been granted to the service account and should be secured and protected appropriately.

Granting The Service Account Access to Search Console

Log into Search Console with an account that has owner level access. Click the Settings menu item at the bottom left of the page, and then click on Users and Permissions row in the loaded page. At the top right you should now see an Add User button, click that and paste the generated email address for the service account you created earlier. The Permission field allows you to select Full or Restricted, the difference between them is documented here, but if you are only querying data you'll likely want to choose Restricted.

Adding Search Console API Maven Dependency

With the service account configuration out of the way we can now turn to the coding side. To use the Search Console Java API you'll need to add the following dependency to your Maven pom.xml:

        

    <dependency>
        <groupId>com.google.apis</groupId>
        <artifactId>google-api-services-webmasters</artifactId>
        <version>v3-rev35-1.25.0</version>
    </dependency>

Java Example: Retrieving Daily Statistics

The example below will print site-level daily statistics such as impressions, clicks, and CTR for any site the service account has been granted access to.

import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.services.webmasters.Webmasters;
import com.google.api.services.webmasters.WebmastersScopes;
import com.google.api.services.webmasters.model.ApiDataRow;
import com.google.api.services.webmasters.model.SearchAnalyticsQueryRequest;
import com.google.api.services.webmasters.model.SearchAnalyticsQueryResponse;
import com.google.api.services.webmasters.model.SitesListResponse;
import com.google.api.services.webmasters.model.WmxSite;
import java.io.FileInputStream;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;

// Source https://bhoey.com/blog/google-search-console-api-java-example/

public class SearchConsoleAPIExample {

    public static void main(String[] args) throws Exception {

        String keyFilePath = "/path/to/your/service_account.json";

        List<String> scopes = Collections.singletonList(WebmastersScopes.WEBMASTERS_READONLY);

        GoogleCredential credentials = GoogleCredential
                .fromStream(new FileInputStream(keyFilePath))
                .createScoped(scopes);

        SearchAnalyticsQueryRequest queryRequest = new SearchAnalyticsQueryRequest();
        queryRequest.setSearchType(null); // override 'web' filter default
        queryRequest.setStartDate("2020-05-01");
        queryRequest.setEndDate("2020-05-20");

        List<String> dimensions = Arrays.asList(
                //"page",             // An explanation of dimensions is available at:
                //"query",            //
                //"country",          // https://developers.google.com/webmaster-tools/search-console-api-original/v3/
                //"devide",           //    searchanalytics/query#dimensionFilterGroups.filters.dimension
                //"searchAppearance", //
                "date"                // Note 'date' isn't included in the list above, but is available nonetheless
        );

        queryRequest.setDimensions(dimensions);

        HttpTransport httpTransport = GoogleNetHttpTransport.newTrustedTransport();
        JsonFactory jsonFactory = new JacksonFactory();
        Webmasters service = new Webmasters.Builder(httpTransport, jsonFactory, credentials)
                .setApplicationName("MyAppName")
                .build();

        Webmasters.Sites.List siteListRequest = service.sites().list();

        SitesListResponse siteListResponse = siteListRequest.execute();
        for (WmxSite site : siteListResponse.getSiteEntry()) {

            String url = site.getSiteUrl();

            Webmasters.Searchanalytics.Query query = service.searchanalytics().query(url, queryRequest);

            SearchAnalyticsQueryResponse queryResponse = query.execute();

            // Print site header
            System.out.println("----[ " + url + " ]----");

            // Print full JSON response
            System.out.println(queryResponse.toPrettyString());

            if (queryResponse.getRows() == null){
                System.out.println("No rows returned in response");
            }
            else {
                for (ApiDataRow row : queryResponse.getRows()) {
                    // Print a single row
                    System.out.println(row);

                    // Print an individual data point
                    System.out.println(row.getClicks().intValue());
                }
            }
        }
    }
}

There's a lot going on in this short program, for more information on the individual classes and function calls you can look through the API's official Javadocs and/or source code.

Conclusion

Hopefully this example fills in some of the blanks of how to use the Search Console Java API with a service account. Good luck and happy coding!