Using the Google API for migration allows any of the Google Workspace products to be migrated even when IMAP is disabled on the Google tenant. This article covers setup, frequently asked questions, and the difference between API and IMAP endpoints.

This guide can be used if your tenant is G Suite, Gmail, Google Drive, Google Shared Drive, or Google Sites.

FAQ

How is this different from the IMAP endpoint?

Many corporate policies disable IMAP automatically due to security concerns. While this can usually be worked around, it adds another step to the complexity of the process. With the API endpoint, that step is skipped.

Changing from IMAP to API does not make your company or client less secure, however, with this feature, you can use your own G Suite account for the migration, rather than using a BitTitan service account. This also increases your throughput and circumvents a known issue on IMAP that causes problems with the end-user credentials.

The API endpoint is also faster than IMAP, allowing you to complete your migration and get back to work sooner. 

What items will migrate with this endpoint?

All items listed in our MigrationWiz: Migrated and Not Migrated Items article will still migrate. Note that these items will vary by source and destination, so carefully check the proper environment listings. This list is updated frequently, so we strongly suggest you check back regularly during your migration project.

How does this change my migration process?

If you are still using the IMAP endpoint, there are no changes to the existing process. If you choose one of the products of the new Google Workspace as an endpoint, you will be prompted to upload your credentials via JSON.

Can I switch between the IMAP and API endpoints during a single migration?

No. Each endpoint uses a different watermark. Switching the endpoints mid-stream will cause duplicates at the destination, so MigrationWiz will present an error if this is attempted.

Do I need separate service accounts if my source and destination are both the API endpoint?

You can either create separate service accounts for the source and destination or have a common service account. If you have a common service account, skip Steps 1-3 below and begin with Step 4. For separate service accounts, follow all the steps below. 

Migrating with Google API

Google Drive, Google Shared Drives, Google Groups, G Suite (Gmail API), and Google Sites

This section helps you to prepare your environment for migrations with Google Drive, Google Shared Drives, Google Groups, G Suite (Gmail API) endpoints (source or destination), and Google Sites.

The Google Drive (Own Service Account) connector launched in 2020 and requires the setup of a Google Service account. This significantly reduces the likelihood of the migration being throttled. This is the option we recommend for all Google Drive migrations. Follow the steps below under OAuth2 Requirements for Google Drive (Own Service Account) to set up your environment for this endpoint.

All accounts being migrated must be in Active status in the tenant. Users that are set to a status of Inactive will not be able to fully migrate and will fail in the project.

Prerequisites

Please review and meet the following requirements according to your endpoint.

Step 1: Create a Google Project

1. Go to the Google Cloud Platform (GCP) Console and sign in as a super administrator. Choose one of the options below:
   • If you haven't used the Google Cloud Platform Console before, agree to the Terms of Service and click Create Project.
   • If you have used Google Cloud Platform Console before, at the top of the screen next to your most recent project name, click Down to open your projects list. Then, click New Project.
2. Enter a project name and click Create.
3. When the new project creation completes, at the top of the screen next to the current project name, click the Down icon and select the newly created project name from the list.

If you are not able to create a project here, it may be that the ability to create projects has been disabled for your tenant. To check this, navigate to the Google Admin Center click on Apps > Additional Google Services, and select the Google Cloud Platform. Once there, you should see a setting that can be toggled to allow users to create projects.

Step 2: Enable APIs for the Service Account

1. From the Google Cloud Platform Console, click Menu > APIs & Services > Library.
2. Enable the following APIs by selecting the specific API and clicking Enable.
   Repeat for each API listed below for each endpoint:

Make sure that the Gmail, Calendar, and Contacts services are enabled within the Google tenant. You can control services for your users using the instructions on this page: Control who can access G Suite and Google Services.

Step 3: Create a Customer Tenant Service Account

1. Click Menu () > IAM & Admin > Service accounts from the Google Cloud Platform Console. 
2. Click + Create Service Account at the top middle of the screen and enter a name.
3. Click Create.
4. Assign the role of Owner to the new Service Account by selecting Owner from the Role Dropdown menu.
5. Click Continue to move to the next step, then click the Done.
6. You will now be returned to the "Service Accounts" page.
7. On the ‘Service accounts' page, click the vertical ellipsis “⋮” under the 'Actions’ column for the service account created above, and click on Manage Keys.
8. Click + Add Key.
9. Click Create New Key.
   
   • Make sure that JSON is selected as "Key Type".
10. Click Create.
11. Click Close. 
    • Make sure that you download the key as a JSON file and make a note of the name and location of the file. This JSON file will be used when setting up the migration endpoint in the Mailbox Migration project.
12. Click Close.

Step 4: Find your Unique ID

From the Google Cloud Platform Console:

1. Go to Menu > IAM & Admin > Service accounts.
2. Find the service account that was set up in Step 3: Create a Customer Tenant Account.
3. Find the Unique ID field for that service account by clicking the Column Display Options “|||” button in the upper right corner above Actions.
4. Select the Unique ID checkbox.
5. Click Ok.
6. Copy this unique ID number.
   
   • This is the Client ID number that will be used in a later step.
   • This Client ID should be considered as confidential as Administrator account passwords and handled securely.

Step 5: Setting the Scopes for the Migration

1. Go to https://admin.google.com and authenticate as a Super Administrator.
2. In the admin console, go to Menu > Security > Access and data control> API controls> Manage Domain Wide Delegation.
   

   Important

   If you do not see the security icon on your admin console home page, you do not have the necessary rights on your account to make these changes. Request Super Administrator access from the customer to implement these changes. Google limits settings access and configuration to only G Suite Super Administrator accounts.

3. Click Add New.
4. In the Client ID field, paste the Unique ID copied above.
5. In the OAuth scopes (comma-delimited) field, paste all scopes listed below:
6. For source/destination endpoint:
   
   At Source At destination

   • Google Suite (Gmail API)
     https://www.googleapis.com/auth/admin.directory.group.readonly,
https://www.googleapis.com/auth/admin.directory.user.readonly,
https://www.googleapis.com/auth/drive.readonly
   • Google Shared Drives
     https://www.googleapis.com/auth/drive,
https://www.googleapis.com/auth/admin.directory.group.readonly,
https://www.googleapis.com/auth/admin.directory.user.readonly,

https://www.googleapis.com/auth/admin.directory.domain.readonly

   • Google Groups
     https://www.googleapis.com/auth/apps.groups.settings,
https://www.googleapis.com/auth/admin.directory.group.readonly,
https://www.googleapis.com/auth/admin.directory.domain.readonly,

https://www.googleapis.com/auth/devstorage.read_only,
https://www.googleapis.com/auth/ediscovery,brhttps://www.googleapis.com/auth/admin.directory.user.readonly
   • G-Suite (Gmail API)
     https://mail.google.com/,
https://www.google.com/m8/feeds,
https://www.googleapis.com/auth/contacts.readonly,
https://www.googleapis.com/auth/calendar,
https://www.googleapis.com/auth/calendar.readonly,
https://www.googleapis.com/auth/admin.directory.group.readonly,
https://www.googleapis.com/auth/admin.directory.user.readonly,
https://www.googleapis.com/auth/drive,
https://sites.google.com/feeds/,
https://www.googleapis.com/auth/gmail.settings.sharing,
https://www.googleapis.com/auth/gmail.settings.basic,
https://www.googleapis.com/auth/contacts.other.readonly
   • Google Sites
     admin.directory.group.readonly,
admin.directory.user.readonly,
calendar.readonly, drive

   • Google Drive
     https://www.googleapis.com/auth/admin.directory.group,
https://www.googleapis.com/auth/admin.directory.user,
https://www.googleapis.com/auth/admin.directory.group.readonly,
https://www.googleapis.com/auth/admin.directory.user.readonly,
https://www.googleapis.com/auth/drive
   • G-Suite (Gmail API)
     https://mail.google.com/,
https://www.google.com/m8/feeds,
https://www.googleapis.com/auth/contacts.readonly,
https://www.googleapis.com/auth/calendar,
https://www.googleapis.com/auth/admin.directory.group,
https://www.googleapis.com/auth/admin.directory.user,
https://www.googleapis.com/auth/drive,
https://sites.google.com/feeds/,
https://www.googleapis.com/auth/gmail.settings.sharing,
https://www.googleapis.com/auth/gmail.settings.basic,
https://www.googleapis.com/auth/contacts.other.readonly
7. Click Authorize.

You should now see your specific Unique ID and the associate scopes listed.

Step 6: Additional Steps According to your Endpoint

Please review the following information for Google Drive (at source) and Google Groups. 

Export mailboxes to CSV file(s) (for Google Drive Source Enpoint Only)

From the Google Admin portal:

1. Click Users.
2. Click ⁝ (3 vertical dots).
3. Download Users.
4. Download All Users
5. Click OK.
6. Save.

Assigning Google Vault privileges (for Google Groups Only)

1. The user must be a Super Administrator user and signed in as one.
2. Sign in to Google Admin Console.
3. In the Overview page, click on Admin Roles.
4. Click on Create a New Role.
5. Enter the Name and Description for the role, and click Continue.
6. Find and expand the list on Google Vault, select the following:
   
   • Manage Matters
   • Manage Searches
   • Manage Exports
   • View All Matters
7. Click Continue.

G Suite IMAP and G Suite at destination

G Suite requires the use of OAuth 2.0 for migrations. The steps to set this up are explained below for Source preparation. OAuth 2.0 OR Google API may be used for the Destination.

Grant MigrationWiz OAuth 2.0 Access to G Suite

BitTitan products use OAuth 2.0 to authenticate to G Suite and utilize the G Suite (IMAP) endpoint in MigrationWiz. This is applicable to both mailbox and document migration projects. In order to obtain access to your G Suite data, it is necessary to add specifically allowed API scopes to the MigrationWiz project. Enabling access is required for both G Suite mailbox and Google Drive document migration projects.

Mailbox migration projects require that a G Suite administrator grant access to the BitTitan client ID and scopes listed in this article.

To configure the OAuth access within your G Suite environment, follow the directions here: OAuth Configuration.

Step 1: Create Users on G Suite

Google provides clear, step-by-step guidance for this project. Follow these steps and then move to the next step listed below.

Step 2: Set Authentication

BitTitan products use OAuth 2.0 to authenticate to G Suite and utilize the G Suite (IMAP) endpoint in MigrationWiz. This is applicable to both mailbox and document migration projects. In order to obtain access to your G Suite data, it is necessary to add specifically allowed API scopes to the MigrationWiz project.

• These steps must be followed whenever there is a migration project either to or from G Suite that will utilize the G Suite (IMAP) endpoint. This is currently required for all Google as Source projects. Enabling access is required for both G Suite mailbox and Google Drive document migration projects.
• Mailbox migration projects require that a G Suite administrator grant access to the BitTitan client ID and scopes listed in this article.
• Document migration projects require that a G Suite Super administrator grant access to the BitTitan client ID and scopes listed in this article and enable the API access. The steps to do this are included at the bottom of this article.

Step 3: Setting the Scopes for the Migration

Complete these steps to grant BitTitan client ID access to the appropriate scopes:

1. Go to Google Admin and authenticate as a Super Administrator.
2. In the admin console, go to Menu > Security > Access and data control> API controls> Manage Domain Wide Delegation.
   

   Important

   If you do not see the security icon on your admin console home page, you do not have the necessary rights on your account to make these changes. Request Super Administrator access from the customer to implement these changes. Google limits settings access and configuration to only G Suite Super Administrator accounts.

3. Click Add New.
   
4. Enter 113321175602709078332 into the Client ID field.
5. Enter the following groups of scopes into the OAuth Scopes (comma-delimited) field. Note that these scopes are for the source only. The destination scopes are found below.
   
   • G Suite as Source (read-only scopes):
     https://mail.google.com/,
https://www.google.com/m8/feeds,
https://www.googleapis.com/auth/contacts.readonly,
https://www.googleapis.com/auth/calendar.readonly,
https://www.googleapis.com/auth/calendar,
https://www.googleapis.com/auth/admin.directory.group.readonly,
https://www.googleapis.com/auth/admin.directory.user.readonly,
https://www.googleapis.com/auth/drive,
https://sites.google.com/feeds/,
https://www.googleapis.com/auth/gmail.settings.sharing,
https://www.googleapis.com/auth/gmail.settings.basic,
https://www.googleapis.com/auth/contacts.other.readonly

   • G Suite as Destination (read-only scopes):
     
https://www.google.com/m8/feeds,
https://www.googleapis.com/auth/contacts.readonly,
https://www.googleapis.com/auth/calendar,
https://www.googleapis.com/auth/admin.directory.group,
https://www.googleapis.com/auth/admin.directory.user,
https://www.googleapis.com/auth/drive,
https://sites.google.com/feeds/,
https://www.googleapis.com/auth/gmail.settings.sharing,
https://www.googleapis.com/auth/gmail.settings.basic,
https://www.googleapis.com/auth/contacts

6. Click Authorize.
7. The client name is 113321175602709078332 (make sure there are no leading or trailing spaces, as this may cause the error "URL ends with an invalid top-level domain name."). This will grant BitTitan products access to the appropriate scopes.

More than one domain?

If you are migrating from multiple domains, repeat these steps for each domain.

Step 4: Enable APIs and Whitelist MigrationWiz as a Trusted App

1. Go back to Security.
2. Click API Controls.
3. Click Manage Third-Party App Access.
4. Click Configure New App, then select OAuth App Name or Client ID.
5. Search for the MigrationWiz.com App name by entering the OAuth2 Client ID: 113321175602709078332.
6. Select the MigrationWiz.com App.
7. Select the OAuth2 Client ID (113321175602709078332).
8. Select the App Access as Trusted: Can access all Google Services and then click Configure.

Step 5: Enable IMAP

Ensure IMAP access is enabled for all users. For details on how to check this, refer to the Google support article here.

Enable Folder Size Limits

1. Ensure the folder size limits on IMAP folders have been removed for all users. For each user:
2. Click the gear icon.
3. Click Settings.
4. Select Forwarding and POP/IMAP tab.
5. Select Folder Size Limits.
6. Select the radio button for Do not limit the number of messages in an IMAP folder (default). This is an end-user setting, which can only be set on a per-user basis. Therefore, we recommend that you send instructions to your end users to check this setting.

Sample Communication to End Users

Action Required - Due Date XX/XX/XX We are preparing to migrate your environment. To ensure a seamless migration, certain end-user settings must be verified. Follow the directions below to remove the folder limits on your account migration. Failure to do so on time may result in lost items. G Suite has a setting that can limit the number of messages in an IMAP folder. If this is configured, this will restrict MigrationWiz so that it can only retrieve and migrate that number of messages from each folder.

1. Log in to your Gmail account.
2. Click on the gear icon in the upper right-hand side of the window, and choose Settings.
3. Click on the Forwarding and POP/IMAP tab.
4. Under the IMAP Access heading, look for Folder Size Limits.
5. Ensure that Do not limit the number of messages in an IMAP folder (default) is selected.

To further clarify the implications of this setting: If the limit has been set (e.g., to 1,000 as per the default suggestion), then if folders contain more than 1000 items, they will be truncated at 1000 items. This means that MigrationWiz will only be able to migrate 1000 emails from each folder. Thank you for your assistance. If you have any questions or concerns, please contact [Help Desk].

Step 6: Export mailboxes to CSV file(s)

This is an optional step, that will help you to migrate the users in MigrationWiz. From the Google Admin portal:

1. Click Users.
2. Click ⁝ (3 vertical dots).
3. Download Users.
4. Download All Users.
5. Click OK.
6. Save.