Folder Filtering & Mapping in MigrationWiz

November 28, 2023 14:55

The following guide explains folder filtering and mapping and how to apply them in MigrationWiz. These options can help simplify and expedite migrations.

Folder Filtering

MigrationWiz allows folder exclusions and inclusions using the Regular Expressions (RegEx) syntax.

Filtering folders can reduce the time needed to index the items to be migrated. MigrationWiz indexes all folder names on the source first, then applies any folder filters. Then items in folders are indexed, so adding folder filters shortens the indexing time needed to identify all folders to be included in the migration.

To configure a folder:

Sign in to your MigrationWiz account.
Click the Projects tab if you do not see your Project listed.
From the list, click on the name of your Project. Click on the Edit Project button.
Click on the Advanced Options button.
Under Filtering, specify a filter.
Click Save.

A useful utility for testing RegEx is: http://regexhero.net/

Folder Normalization

Here is how MigrationWiz normalizes folder names before applying a specified RegEx to decide if a folder should be skipped:

Case Sensitivity - Folder names are compared in a case-insensitive fashion.
Folder Separator - Regardless of the messaging system, all folder paths are separated with a forward slash (/) for exclusion comparison, for example, Inbox/SubFolder.
Preceding Separator - Do not specify any preceding separator when referencing the folder, i.e., do not specify /Inbox/SubFolder but rather Inbox/SubFolder.
- DropBox migrations are the exception to this. When DropBox is the source endpoint, the top folder is named "/"
Trailing Separator - Do not specify any trailing separator when referencing the folder, i.e., do not specify Inbox/SubFolder/ but rather Inbox/SubFolder.

If your folder name is localized (for example: "Postvak In" is "Inbox" in Dutch), make sure to use the localized name.

Folder filters are based on the Source or export side, not the Destination or import side. Therefore, if the Destination has a different format than the Source, the filter needs to reflect the Source side. This is common in document migration projects, e.g., Google Drive to OneDrive for Business. For more information, refer to the example filterSyntax: ^(?!My Drive/Some/Folder/Path) in the table below.

^ indicates "match the beginning of the string", meaning that you should filter everything that matches the beginning of the string.

^(?!abc) indicates a negative look ahead , meaning that you only process folders that match the string that follows "^(?!".

Make sure you end the folder path with ")"

Folder Filtering for Document Migrations

Google Drive

If the source is Google Drive, then the folder filter needs to include the root My Drive in the filter.

Example: ^My Drive/Folder A/Folder B

Google Shared Drives

If the source is Google Shared Drives, then the folder filter needs to include the root Drive Name of the Google Shared Drives.

Example: ^DriveName/Folder A/Folder B

SharePoint & OneDrive Folder Filters

If the source is SharePoint or OneDrive, then the folder filter needs to include the root Documents or Shared Documents in the filter. If the source SharePoint document library name is something other than Shared Documents, replace Shared Documents with the name of the intended source document library name.

Examples: (More syntax examples can be found further down in the table named Filter Syntax Examples)

To exclude OneDrive folders:

^Documents/Folder A/Folder B

To exclude SharePoint folders using the default document library name of Shared Documents:

^Shared Documents/Folder A/Folder B

To include OneDrive folders:

^(?!Documents/Folder A/Folder B)

To include SharePoint folders using the default document library name of Shared Documents:

^(?!Shared Documents/Folder A/Folder B)

Dropbox Filters (The destination does not make a difference)

The folder path will always start with / to represent the folder path starting at the root of Dropbox:

To exclude folders for Dropbox:

^(/FolderA/FolderB/FolderC)

To include folders for Dropbox:

^(?!/FolderA/FolderB/FolderC)

Filter Syntax Examples

FilterSyntax	Description
^Inbox/	Do not migrate any subfolders under the Inbox.
(^Inbox$\|^Inbox/)	Do not migrate the Inbox or any subfolder of Inbox.
(^Folder1$\|^Folder2$\|^Folder3/)	Do not migrate Folder1, Folder2, and any subfolder of Folder3. Note that subfolders of Folder1 and Folder2 are still migrated.
^(?!Inbox$)	Migrate only the Inbox. All other folders will be filtered.
^(?!Inbox)	Migrate only the Inbox and its corresponding subfolders. All other folders will be filtered.
^(?!Foo/ABC)	Only process folders starting with "Foo/ABC"
^Foo/ABC	Skip folders starting with "Foo/ABC"
^(?!.*Marketing)	Only process folders that have "Marketing" in their path
^Deleted Items/Frequent Contacts	This folder filter will filter out the folder named Frequent Contacts, under the Deleted Items folder so that it is not included in the migration
^(?!Inbox/Foo\|Inbox/Bar)	This inclusion filter means that only Inbox/Foo and Inbox/Bar folders will be migrated. Note: The parentheses are required!
^(?!INBOX/james1\|INBOX/james2\|INBOX/samantha\|INBOX/john)	Filter everything except INBOX/james1, INBOX/james2, INBOX/samantha, and INBOX/john. Therefore, only these four folders will be migrated.
^(?!My Drive/Some/Folder/Path)	Only export the subtree under Google Drive that matches: My Drive/Some/Folder/Path. Note: The filter is always based on the source. Therefore, if migrating to OneDrive for Business, then the default destination matching directory path would be "Documents/Some/Folder/Path" - but the filter is not based on this path because that is the Destination, rather than the Source.

Deleted content with SharePoint and OneDrive

If data has previously been deleted at the source after being migrated to the destination, applying Folder Filtering to the source to skip the deleted files in a subsequent pass will not be as effective. MigrationWiz API will restore the deleted contents in the destination from the second-stage recycle bin.

For the folder filter to work in this case, the data from the destination SharePoint second stage recycle must be deleted before beginning another migration pass.

Folder Mapping

Use the folder mapping support option to map a Source folder to a Destination folder. For example, when you want to map the "Inbox" folder on the Source to an "Old Inbox" folder on the Destination in a mailbox migration.

The folder mapping support option uses regular expression (regex) and is added to the Adding Support Options screen for individual items or for an entire migration project, which applies to all items in the migration project. This article provides detailed information about using the folder mapping Support Option for mailbox migrations and some examples of how the folder mapping support option can be modified to work for you.

Important

For Source folders to map properly, the escape character "\" is required after the use of the following characters:

[ \ ^ $ . | ? * + ( ) { }

Make sure to use the escape character after every Source Folder.

Add folder mapping for a migration project

Follow these steps to add folder mapping to a migration project, which applies to all items in the migration project:

From the MigrationWiz dashboard, click on the migration project that will have the folder mapping added.
From the migration project page, click Edit Project and then select Advanced Options.
In the Support Option section, add this support option:
FolderMapping="^SourceFolder->DestinationFolder" Replace "SourceFolder" and "DestinationFolder" with the names of the folders in the Source and Destination environments.
Click the "+" sign to add the folder mapping.
Click Save.

Add folder mapping for an individual mailbox

Important

If you are migrating an entire source mailbox folder hierarchy to exist under a new single subfolder of a Destination mailbox, then you must add additional advanced support options for calendar and contact items (See Migrate Mailbox to Single Subfolder outlined below). Otherwise, you'll experience problems accessing migrated items at the destination.

Follow these steps to add a folder mapping to an individual item in a mailbox migration project:

From the MigrationWiz dashboard, click on the migration project that contains the item that will have folder mapping added.
From the migration project page, find the item that will have folder mapping.
Click the Edit icon (pencil icon) on the right side for that individual item to open its Advanced Options.
In the Support Option section, add this support option:
FolderMapping="^SourceFolder->DestinationFolder"Replace "SourceFolder" and "DestinationFolder" with the names of the folders in the Source and Destination environments.
Click the "+" sign to add the folder mapping.
Click Save Item.

Folder mapping examples

Here are a few examples of folder mapping commands and processes, along with special notes for those specific mappings.

FolderMapping="^Sent Emails->My Sent Items" This maps the "Sent Emails" folder to a folder named "My Sent items".
FolderMapping="^Inbox->Managed Folder" This maps the "Inbox" folder to a folder named "Managed Folder".
FolderMapping="^.*$->DestinationFolder" This maps all files across all source folders into a single folder on the Destination. Replace "DestinationFolder" with the name of the folder in the Destination environment.
- ^.*$: Is the regex version of "Select All". (^) Match beginning of string; (.*) Any characters; ($) Match end of string.
FolderMapping="^Documents->Documents/DestinationFolder" This mapping will map everything being migrated to the defined Destination folder, which will appear as a top-level folder within OneDrive.
- Replace "DestinationFolder" with the name of the folder in the Destination environment.
- When migrating to OneDrive there is a hidden root-level folder titled Documents.
FolderMapping="^->DestinationFolder/" This maps everything from the root level down into a single folder on the Destination while maintaining the folder hierarchy within the Destination folder.
- Replace "DestinationFolder" with the name of the folder in the Destination environment.
- Be sure to add the / (forward slash symbol) at the end of the folder name to tell MigrationWiz to create a new subfolder under that Destination folder.
- Be careful with this, as it will map system folders as well as regular folders (such as Google Drive's "Permissions" folder).
  
  Warning
  
  If migrating a mailbox to a specific Destination folder, this command can cause folders to become invisible and inaccessible due to the order of migration. Unless you are ONLY migrating a single folder type, you will instead want to use the folder mapping format outlined under Migrate Mailbox to Single Subfolder below.
FolderMapping="^Floor 5-A5-8-T4-$AV\+TC$->Calendar" This maps the "Floor 5-A5-8-T4-(AV+TC)" folder to the "Calendar" folder. The part to the left of "->" is regex, so special characters need to be escaped.
- In the above example, the special characters escaped with a backslash before the special characters.

Folder Mapping For Document Migrations

Folder mapping only occurs at the destination and the root document library name or root drive name for the destination (not the source root drive name or source root document library name) must be used for both the left and right side (left and right side of ->) of the folder mapping expression.

The following format examples will create a new parent folder in the root of the destination document library (SharePoint or OneDrive) or the root of the destination drive (Google Drive or Google Shared Drive) and migrate the folder hierarchy and files to be under the new parent folder named in the folder mapping.

Migrating to SharePoint: (If the destination document library name is different or in a different language, replace Shared Documents with the correct name of the destination document library or with the correct language used for Shared Documents and replace DestinationFolder with the name of the new parent folder in the destination)

FolderMapping="^Shared Documents->Shared Documents/DestinationFolder"

Migrating to OneDrive: (If the destination document library name is in a different language, replace Documents with the correct language being used for Documents and replace DestinationFolder with the name of the new parent folder in the destination)

FolderMapping="^Documents->Documents/DestinationFolder"

Migrating to Google Drive: (Replace DestinationFolder with the name of the new parent folder in the destination)

FolderMapping="^My Drive->My Drive/DestinationFolder"

Migrating to Google Shared Drive: (Replace DestinationSharedDrive with the name of the Google Shared Drive in the destination and replace DestinationFolder with the name of the new parent folder in the destination)

FolderMapping="^DestinationSharedDrive->DestinationSharedDrive/DestinationFolder"

Migrate Mailbox to a Single Subfolder (Calendar, Contacts, and Mail)

It is possible to migrate a Source mailbox into a single subfolder of a Destination mailbox, but contact and calendar folders will need to reside under the root contacts and calendar folders.

These folder mapping support options ensure the folders are created at the correct location:

FolderMapping="^(?!Calendar$|Calendar/|Contacts$|Contacts/)->DestinationFolder/"
FolderMapping="^((Calendar)([/]?)|(Contacts)([/]?))->$2$4/DestinationFolder$3$5"

The support options above both need to be added in the Advanced Options for your project or individual mailboxes.

To add a single mailbox, follow the steps below:

Click on the pencil icon on the far right side of the email address that you wish to edit.
Replace "DestinationFolder" with the name of the folder you want to use at the Destination.

Important

The folder mapping support options listed above must be added in the order shown. If they are not added in this order, some contents of the migrating mailbox may become invisible and inaccessible on the destination

The spelling of both 'Calendar' and 'Contacts' must also reflect the language of the folder name at the Destination mailbox, e.g., 'Calendar' will be 'Calendrier' in French and 'Contacts' will 'Contactpersonen' in Dutch.

Additional information about folder mapping

The Source folders may be different (at the API level) from what is displayed in the user interface or Outlook. Therefore, when defining the folder mappings, pay special attention to the exact format of the folder name (e.g., [Gmail]/Sent for Sent).

We recommend that you use Regex Hero to test your regular expression code.

{%escape%}is a reserved word that must be avoided when creating folder mappings. If original folder names in the Source end with {%escape%}, they must be manually renamed before migration.

G Suite (GMail API) Limitations

For ‘GSuite (Gmail API) to GSuite (Gmail API)’ migrations only:

If you want to map multiple source labels into a single label on the destination, then the parent destination label must be created before starting the migration.

Example: If you wish to map three Source labels ('ABC', 'PQR', and 'XYZ') into one parent label ('DestinationLabel') at the destination then you must create the parent label ('DestinationLabel') at destination mailbox before running the migration.

Escaping Special Characters

Example: FolderMapping="^Floor 5-A5-8-T4-$AV\+TC$->Calendar" This maps the "Floor 5-A5-8-T4-(AV+TC)" folder to the "Calendar" folder. The part to the left of "->" is regex, so special characters need to be escaped. In the above example, the special characters escaped with a backslash before the special characters.

Here is a list of the common special characters that would need to be escaped and what each one does:

^ matches the beginning of a string.
$ matches the end of the string.
{} are used as range quantifiers.
() are used to group characters.
[] are used to create a character class that matches a single character.
. matches any character except a new line.
* is a match 0 or more quantifier.
? is the 0 or 1 quantifier.
| Separate a series of alternatives.