Google Vault Migration Best Practices

This article covers best practices, setup, and troubleshooting for Google Vault Migrations. It is intended for use with the Google Vault to Microsoft 365 Migration Guide.

The key to a successful Google Vault migration is to properly identify the data you want to extract from Google Vault. To get data out of Google Vault, you must search for it, then export the search results. The search is important for properly exporting the data required. To properly search the data, we need to use Google Search Terms to target the data required. BitTitan Google Vault Extractor will programmatically search, export, and download the results (automatically and at scale) 

Setting the Azure location for Google Vault migrations

This is a two-part process.

1. Set the Azure location during the initial upload to Azure.

• If you want to upload to a specific data center, and are using the UploaderWiz utility, be sure to include the command line parameter: azurelocation (available options are: gov (government), ger (Germany), china (China), as documented in Getting Started with UploaderWiz.
• If you want to upload to a specific data center, and are using AZCopy, refer to this TechNet article on how to specify the URL when using AZCopy. More information can be found in Set up and configure the Google Vault Extractor.
• Here is an example PowerShell script, which will provide you with the URl to enter:

$storageUri = (Get-AzureRmStorageAccount -Name $StorageAccountName -ResourceGroupName $ResourceGroupName).PrimaryEndpoints.Blob

It will return a URl like this one: “https://StorageAccountName.blob.core.cloudapi.de/”, which is what should be entered for the URl.

2. Set the MigrationWiz Advanced Option GoogleVaultCustomEndpointSuffix=Azure URI during migration, so that the migration servers know to look for PST files in that location.

• From your MigrationWiz project Advanced Options, under Support/Support options, add GoogleVaultCustomEndpointSuffix=Azure URI .
• Note: The Azure URI needs to be changed to one of the following values:
• core.chinacloudapi.cn (China)

• core.cloudapi.de (Germany)

• core.usgovcloudapi.net (government)

It is important to understand that if Germany is chosen, data will still securely leave the country during migration, because compute resources are outside of the country.

Extraction Process 

The way Google Vault extractions work is that BitTitan Google Vault Extractor creates and configures the matter, based on a list of users. When the matter is created, we then instruct Google to search the matter based on the terms that we specify. Google then searches Vault and prepares and exports. During the search, we are logging the number of anticipated items Google will return to us in the export. When Google has the export results ready for download, BitTitan Google Vault Extractor will automatically download the export from Google. The export is downloaded to a local Export Server and stored until it can be uploaded back into an Azure BLOB. BitTitan Google Vault Extractor can be setup to sequentially download from Google and then upload to Azure. The extractor will download two files from Google: the compressed zip folder with the data and an XML file. It will also create a .done file, which indicates a successful extraction. If the extractor fails, it will create a .fail file for the user.  

The export needs to be in Azure because that is where MigrationWiz can read the file and start the migration of data into the Microsoft 365 mailbox. Since Google Vault is a compliance system and not an archive, the typical destination for the data is in the Recoverable Items Folder in the user’s primary mailbox. It’s imperative to place the users mailbox on Litigation Hold, before migrating any contents.  

Google Vault Migration Guide 

Migrating from Google Vault is a three-step process: 

1. ​​Identify the data to be exported from Google Vault. 

a. Complete export or targeted data. 

2. Search, Export, and Upload Files (to Azure) using BitTitan Google Vault Extractor. 

a. Search Google Vault.

b. Export and download the contents from Google Vault.

c. Use UploaderWiz to upload the locally stored files to Azure Blob storage. 

3. Migrate the data using MigrationWiz  

What Mail to Extract?

Deciding what data to extract from Google Vault depends on if you are migrating the user’s primary mailbox or not. Google Vault is a mirror of a current mailbox. Migrating the entire Google Vault while performing a mail migration will result in many duplicates.  Because Google Vault contains everything the associated mailbox does, if you’re migrating the primary mailbox too, the only folder you need to migrate from Google Vault is the Permanently Deleted items.  

• Running the extraction with the search term label ^deleted will result in only extracting the permanently deleted items. 
• Running the extraction with the search term older_than:x or newer_than:x is commonly used when the Vault size is too big and you need to split the extractions into multiple extracts.  

Importance of Search Terms

It is recommended that you review the Google website and are familiar with how the search operators work. These are a function of Google and if they are not used correctly, it may result in missing email or not exporting the results that you were expecting. Using search operators (search terms) are the key to what is extracted from the Google Vault Mailbox.  See Google's Use Search Operators article for more information.

Common Search Terms 

Note: Anything with a dash (-) in front of it means that it will be excluded from the export.  

• label:^deleted is used when you want to extract the permanently deleted items. 
• in:trash is used when you want to extract the items in the trash.
• -in:trash is used when you don’t want to extract the items in the trash. 

• in:drafts is used when you only want to extract the items in the drafts folder. 
• -in:drafts is used when you don’t want to extract the items in the drafts folder. 
• older_than:1y is used when you only want to extract items older than one year. 
• newer_than:1y is used when you only want to extract items newer than one year. 

Google Suspended Users 

When migrating Google Suspended users, the primary mailbox is not migrated with MigrationWiz, so you will export all the mail items from Vault. Running the extraction with no search terms will result in the all the mail being extracted from Google Vault. These are usually handled in a separate project from the active users. Special considerations need to be taken into account when migrating into inactive accounts on the Microsoft 365 side. The account should only be made inactive after the migration has finished. Follow the Microsoft Guide for making the account inactive, post migration. Also take note that if you are migrating into the Recoverable Items of the inactive account, make sure the mailbox is on Litigation Hold. Failure to do so, will result in the mail being deleted from Office 365. 

Vault size

Vault size will usually require assistance from Google. Keep in mind what data is being extracted. Extracting only permanently deleted items is more manageable than exporting the complete mailbox. If you’re migrating the primary mailbox with MigrationWiz, most of the time you only need to extract the permanently deleted items. Accurately reporting the amount of data migrating out of Google Vault is important for assessing licensing needs. 

1. Log in to Google Vault and create a new matter.
2. Within the matter, run a search that represents the planned export. Typically, the only data migrated out of Google Vault is in the hidden "Permanently Deleted" label. This label cannot be accessed separately, and is included when searching for Deleted Items. This means all deleted items within Vault will be searched.
3. Run a search for all users (by keeping the Accounts field blank) with this search term: in:trash (all items with "deleted" and "permanently deleted" labels). Be sure to select the checkbox Labeled Exclude drafts. Once the results are returned, click the Export results button. Export times vary from minutes to hours depending on the size of the data set.

   What’s the hold up?

   If the export takes more than 24 hours to complete, Vault will cancel or stall the export. In these cases, you will need to split the data into multiple export operations. Skip to the section below titled "What if the Export is Canceled?”

4. Click on Export to view progress. Once complete, export size and item count can both be viewed here.

What if the Export is Canceled?

Exports can be canceled if they hit an unpublished maximum size, or take more than 24 hours to complete. In these cases, separate the exports by groups of user accounts. To do this,

1. Log in to the G Suite Admin Console, and select Users.
2. On the top right portion of the screen, select the triple dots
3. Click Download users.

The result is a CSV file containing a list of users. From here, follow the sizing instructions above, but when searching, split the user list across searches. Depending on the size of these exports, you may need to break the user lists down even further.

Timing is everything 

Google Vault represents a current version of a production mailbox, as opposed to a point-in-time capture.  Because of this, we recommend only migrating the Google Vault once all data is at rest (no changes are taking place) on the Source system. This ensures that no new items or changes to existing items are missed for migration. Because the Google Vault is at rest during migration, using a Full Migration strategy  will get everything from Azure to the Destination with the least amount of touch possible. 

Google Vault OAuth Client API 

BitTitan’s Google Vault Extractor allows for the use of your own OAuth Client API, rather than the BitTitan shared API. This option has many benefits and allows for greater control over throttling and security. It’s not a requirement of the migration, but BitTitan recommends this option. Below is the step by step information on how to set this up in your Google Tenant and will get the information required by the Google Vault Extractor to use your client API.  

1. Go to Google developer console: https://console.developers.google.com.
2. Sign in with an admin account.
3. Click Create Project.
4. Enter a Project Name.
5. Verify the Organization.
6. Verify the Location.
7. Click Create.
8. Click Credentials.
9. Click Create Credentials.
10. Click OAuth Client ID.
11. Click Configure Consent Screen.
12. Select Internal.
13. Click Create.
14. Enter the Application Name.
15. Enter the User Support Email.
16. Scroll down and enter the Developer Contact Information.
17. Click Save and Continue.
18. Click Credentials then click Create Credentials.
19. Click OAuth Client ID.
20. Change the Application Type to Desktop app.
21. Give the app a Name.
22. Click Create.
23. Your Client ID and Client Secret are displayed. This information is required when running the Google Vault Extractor. Make sure to note them down and keep them safe. Click OK.
24. Click Library.
25. Search for Google Cloud Storage JSON API.
26. Click the Google Cloud Storage JSON API..
27. Click Enable.
28. Search for G Suite Vault API in the top Google API Search Bar.
29. Select G Suite Vault API.
30. Click Enable.
31. The completed project should look like this:

Microsoft 365 Mailboxes 

Important: If you are migrating into the Recoverable Items Folder of the users mailbox, make sure the user is on Litigation Hold in Microsoft 365. If you do not, the data will be automatically deleted in Microsoft 365. Place the mailboxes on Litigation Hold: TechNet article. 

SQL Audit Logging 

It’s recommended you use the audit logging feature in MigrationWiz to properly count items that are migrated. This can be used in reporting for the migration and comparing the number of items that were exported from Google Vault and the number of items that were migrated into the Recoverable Items.  

BitTitan Google Vault Logging 

BitTitan Google Vault Extractor will log all activates to a log foil located here: %userprofile%\appdata\local\logs\GoogleVaultExport 

It is recommended to keep these logs in place for troubleshooting and verification of the number of items that were extracted from Google Vault.  

Troubleshooting Google Vault Extractor 

Item Counts and Verification 

The way that Google Vault extractions work, is that we create and configure the matter. Google then searches and exports the search results. During the search, we are logging the number of anticipated items Google will return to us in the export. Since this is an export, MigrationWiz cannot make an active connection to Google Vault. This means we need to rely on Google and Googles (Item Counts) in the export.  

It is recommended that before you migrate the data into the Microsoft 365 mailbox (Recoverable Items), that you get the size and item counts. When you run the migration with MigrationWiz, it is recommended that you turn on SQL Audit Logging to log all the items that have been migrated. After the migration is completed, we recommend that you get the size and item counts from the Microsoft 365 mailbox (Recoverable Items).  

With the information in the BitTitan Google Vault Extractor logs, the snapshot (before and after migration) of the Microsoft 365 mailbox (Recoverable Items) and the information in the SQL Audit Logging, you can properly troubleshoot discrepancies.  

Velocity 

Add greater velocity by adding more extractors. Adding more extractors means you can exponentially scale up your migrations, however, there are limitations. Each instance of the extractor will sequentially work its way down the list of users, so it will only run one extraction at a time. The bulk of the work is being performed by Google because Google is doing the search and extraction. The BitTitan Google Vault Extractor is orchestrating the creation of the matter and then downloading the exported results. Given that point, you can run multiple versions of the BitTitan Google Vault Extractor on the same Export Server. To run this successfully, you must keep everything linear and not have the extractions cross paths. 

See the diagram below, where all the swim lanes are separate this makes troubleshooting more efficient. Each extractor should work out of its worn working directory on the Export Server and then upload to its own container on the Azure BLOB. Each Azure BLOB should have its own MigrationWiz project so that it can only import the exported files of the intended users. If you uploaded to the same Azure container, for all the extractor instances,  MigrationWiz would import all the users into all the projects. Disk space on the Export Server is also critical so make sure there is enough to support the amount of data that needs to be exported.  

 

Working Directory 1 -> Extractor 1 -> Azure BLOB Container 1 -> MigrationWiz Project 1 

Working Directory 2 -> Extractor 2 -> Azure BLOB Container 2 -> MigrationWiz Project 2 

Working Directory 3 -> Extractor 3 -> Azure BLOB Container 3 -> MigrationWiz Project 3