The key to a successful Google Vault migration is to properly identify the data you want to extract from Google Vault. To get data out of Google Vault, you must search for it, then export the search results. The search is important for properly exporting the data required. To properly search the data, we need to use Google Search Terms to target the data required. BitTitan Google Vault Extractor will programmatically search, export, and download the results (automatically and at scale)
The way Google Vault extractions work, is that BitTitan Google Vault Extractor will create configure the matter, based on a list of users. When the matter is created, we then instruct Google to search the matter based on the terms that we specify. Google then searches Vault and prepares and exports. During the search, we are logging the number of anticipated items Google will return to us in the export. When Google has the export results ready for download, BitTitan Google Vault Extractor will automatically download the export from Google. The export is downloaded to a local Export Server and stored until it can be uploaded back into an Azure BLOB. BitTitan Google Vault Extractor can be setup to sequentially download from Google and then upload to Azure. The extractor will download two files from Google: the compressed zip folder with the data and an XML file. It will also create a .done file, which indicates a successful extraction. If the extractor fails, it will create a .fail file for the user.
The export needs to be in Azure because that is where MigrationWiz can read the file and start the migration of data into the Office 365 mailbox. Since Google Vault is a compliance system and not an archive, the typical destination for the data is in the Recoverable Items Folder in the user’s primary mailbox. It’s imperative to place the users mailbox on Litigation Hold, before migrating any contents.
Migrating from Google Vault is a three-step process:
- Identify the data to be exported from Google Vault.
a. Complete export or targeted data.
2. Search, Export, and Upload Files (to Azure) using BitTitan Google Vault Extractor.
a. Search Google Vault.
b. Export and download the contents from Google Vault.
c. Use UploaderWiz to upload the locally stored files to Azure Blob storage.
3. Migrate the data using MigrationWiz
What Mail to Extract?
Deciding what data to extract from Google Vault depends on if you are migrating the user’s primary mailbox or not. Google Vault is a mirror of a current mailbox. Migrating the entire Google Vault while performing a mail migration will result in many duplicates. Because Google Vault contains everything the associated mailbox does, if you’re migrating the primary mailbox too, the only folder you need to migrate from Google Vault is the Permanently Deleted items.
- Running the extraction with the search term label ^deleted will result in only extracting the permanently deleted items.
- Running the extraction with the search term older_than:x or newer_than:x is commonly used when the Vault size is too big and you need to split the extractions into multiple extracts.
Importance of Search Terms
It is recommended that you review the Google website and are familiar with how the search operators work. These are a function of Google and if they are not used correctly, it may result in missing email or not exporting the results that you were expecting. Using search operators (search terms) are the key to what is extracted from the Google Vault Mailbox.
Common Search Terms
Note: Anything with a dash (-) in front of it means that it will be excluded from the export.
- label:^deleted is used when you want to extract the permanently deleted items.
- in:trash is used when you want to extract the items in the trash.
- -in:trash is used when you don’t want to extract the items in the trash.
- in:drafts is used when you only want to extract the items in the drafts folder.
- -in:drafts is used when you don’t want to extract the items in the drafts folder.
- older_than:1y is used when you only want to extract items older than one year.
- newer_than:1y is used when you only want to extract items newer than one year.
Google Suspended Users
When migrating Google Suspended users, the primary mailbox is not migrated with MigrationWiz, so you will export all the mail items from Vault. Running the extraction with no search terms will result in the all the mail being extracted from Google Vault. These are usually handled in a separate project from the active users. Special considerations need to be taken into account when migrating into inactive accounts on the Office 365 side. The account should only be made inactive after the migration has finished. Follow the Microsoft Guide for making the account inactive, post migration. Also take note that if you are migrating into the Recoverable Items of the inactive account, make sure the mailbox is on Litigation Hold. Failure to do so, will result in the mail being deleted from Office 365.
Vault size will usually require assistance from Google. Keep in mind what data is being extracted. Extracting only permanently deleted items is more manageable than exporting the complete mailbox. If you’re migrating the primary mailbox with MigrationWiz, most of the time you only need to extract the permanently deleted items. Accurately reporting the amount of data migrating out of Google Vault is important for assessing licensing needs. See Sizing Google Vault for more details.
Timing is everything
Google Vault represents a current version of a production mailbox, as opposed to a point-in-time capture. Because of this, we recommend only migrating the Google Vault once all data is at rest (no changes are taking place) on the Source system. This ensures that no new items or changes to existing items are missed for migration. Because the Google Vault is at rest during migration, using a Full Migration strategy will get everything from Azure to the Destination with the least amount of touch possible.
Google Vault OAuth Client API
BitTitan’s Google Vault Extractor allows for the use of your own OAuth Client API, rather than the BitTitan shared API. This option has many benefits and allows for greater control over throttling and security. It’s not a requirement of the migration, but BitTitan recommends this option. Below is the step by step information on how to set this up in your Google Tenant and will get the information required by the Google Vault Extractor to use your client API.
Office 365 Mailboxes
Important: If you are migrating into the Recoverable Items Folder of the users mailbox, make sure the user is on Litigation Hold in Office 365. If you do not, the data will be automatically deleted in Office 365. Place the mailboxes on Litigation Hold: TechNet article.
SQL Audit Logging
It’s recommended you use the audit logging feature in MigrationWiz to properly count items that are migrated. This can be used in reporting for the migration and comparing the number of items that were exported from Google Vault and the number of items that were migrated into the Recoverable Items.
BitTitan Google Vault Logging
BitTitan Google Vault Extractor will log all activates to a log foil located here: %userprofile%\appdata\local\logs\GoogleVaultExport
It is recommended to keep these logs in place for troubleshooting and verification of the number of items that were extracted from Google Vault.
Item Counts and Verification
The way that Google Vault extractions work, is that we create and configure the matter. Google then searches and exports the search results. During the search, we are logging the number of anticipated items Google will return to us in the export. Since this is an export, MigrationWiz cannot make an active connection to Google Vault. This means we need to rely on Google and Googles (Item Counts) in the export.
It is recommended that before you migrate the data into the Office 365 mailbox (Recoverable Items), that you get the size and item counts. When you run the migration with MigrationWiz, it is recommended that you turn on SQL Audit Logging to log all the items that have been migrated. After the migration is completed, we recommend that you get the size and item counts from the Office 365 mailbox (Recoverable Items).
With the information in the BitTitan Google Vault Extractor logs, the snapshot (before and after migration) of the Office 365 mailbox (Recoverable Items) and the information in the SQL Audit Logging, you can properly troubleshoot discrepancies.
Add greater velocity by adding more extractors. Adding more extractors means you can exponentially scale up your migrations, however, there are limitations. Each instance of the extractor will sequentially work its way down the list of users, so it will only run one extraction at a time. The bulk of the work is being performed by Google because Google is doing the search and extraction. The BitTitan Google Vault Extractor is orchestrating the creation of the matter and then downloading the exported results. Given that point, you can run multiple versions of the BitTitan Google Vault Extractor on the same Export Server. To run this successfully, you must keep everything linear and not have the extractions cross paths.
See the diagram below, where all the swim lanes are separate this makes troubleshooting more efficient. Each extractor should work out of its worn working directory on the Export Server and then upload to its own container on the Azure BLOB. Each Azure BLOB should have its own MigrationWiz project so that it can only import the exported files of the intended users. If you uploaded to the same Azure container, for all the extractor instances, MigrationWiz would import all the users into all the projects. Disk space on the Export Server is also critical so make sure there is enough to support the amount of data that needs to be exported.
Working Directory 1 -> Extractor 1 -> Azure BLOB Container 1 -> MigrationWiz Project 1
Working Directory 2 -> Extractor 2 -> Azure BLOB Container 2 -> MigrationWiz Project 2
Working Directory 3 -> Extractor 3 -> Azure BLOB Container 3 -> MigrationWiz Project 3
Note: Do not run more than 30 instances of Google Vault Extractor. Google Vault can support a maximum of 30 parallel exports.