Public Folder Migration Best Practices

September 27, 2022 18:29

The tips below are a few best practice suggestions for general public folder migrations, and should be used in conjunction with the migration guide best suited to your migration scenario.

For Migrations with >1000 Folders

If Source Public Folders have more than 1000 folders below the top-level folder, it is important to work with BitTitan Support so that the migration can be split up, based on migration points after every 1000 folders.

Here are the steps to follow:

Create a Public Folder migration project.
Select the Source endpoint and enter admin credentials.
Select the Destination endpoint and enter admin credentials.
Submit a ticket to BitTitan Support, by following the directions in How do I get support for your products? In the ticket description notes, add the name of the Public Folder migration project, and specify that the Source Public Folders have more than 1000 folders. The title of ticket should include the following text: "Request to split Public Folder migrations based on child Public Folder count." In addition, run the following PowerShell command in your Source Exchange Management Shell (EMS) or Source Office 365 PowerShell environment and include the results of the command, by providing the folders.csv file, generated from running the script, to the ticket:

Get-PublicFolder -Recurse -Resultsize Unlimited | select Identity | Export-Csv -Encoding Unicode C:\folders.csv

BitTitan Support will then create a new MigrationWiz project on your behalf, and configure the new migration points as needed.

This split process will create a new MigrationWiz project called (project)_SPLIT and create additional "migration points" in addition to the top-level folders already configured. These are created in Office 365. The extra migration points will assist with processing the migration expeditiously.

It will also count the number of subfolders from the root, and then create a new migration point after every 1000 folders from the top-level folder.

Finally, it will configure folder filters. These are used to start and stop the migration points.

Important:

Any new folders created after the Public Folder split is initiated will not be in scope for the migration.

The original project will not be affected or changed, and will remain in your list of MigrationWiz projects. However, do not continue the Public Folder migration project from the original project.

Once the tool is complete, you should run the migration from the new project, and can then delete the original project.

Project Creation

You do not need to create separate projects for differing item types. Projects are based on folder size and number. The same project can be used when migrating folders containing different item types.

Item & Data Assessment

In order to determine the number of mail, calendar and contact items that are in your public folders, you'll need to run the following PowerShell script.

# Exchange Management Shell Script # This script can be run to count the number # of items that exist in all public folders starting # at the root. # Set the execution policy Set-ExecutionPolicy Unrestricted -Force # Retrieve the credentials $LiveCred = Get-Credential # Create and import the PowerShell session for ExchangeOnline Install-Module -Name ExchangeOnlineManagement Import-Module -Name ExchangeOnlineManagement Connect-ExchangeOnline -ConnectionUri https://ps.outlook.com/powershell/ -Credential $LiveCred # Recursively go through all public folders counting the items Get-PublicFolder -Recurse | Get-PublicFolderItemStatistics | Group-Object -Property ItemType -NoElement # End Script

This will generate output giving the count and item name (IPM.Note for mail, IPM.Contact for contacts, and IPM.Appointment for calendar items).

Folder Management

Prior to migrating your Public Folders we recommend the following actions:

When migrating to Exchange Online (Office 365), ensure that all of your Public Folders are within the stated limits for public folders.

Purge old or unused data. This will save on your migration costs, and also save some space in the target environment.

There is no need to create folders in your destination before performing a Public Folder migration. MigrationWiz will create all folders from the root to the folder being migrated. However, only the folder being migrated will have proper permissions set and will be mail-enabled (if appropriate).

One thing to note is that if you do create the folders before performing a migration, the type of the folder has to be correct. This means that if you are migrating a Contacts or Calendar items folder, then the folder has to be created as a contact or calendar folder. This cannot be done from the current Exchange Online UI; it has to be done through PowerShell. If the folder is not of the correct type, then no items will be migrated into it (e.g., mail items cannot go into a Contacts folder, and vice versa).

Migration Strategies

The easiest and cleanest migration strategy for Public Folders is the Big Bang strategy. This refers to migrating all of the data in one pass. This is most suitable when migrating smaller Public Folders.

For larger Public Folders, follow a multi-pass strategy, which is defined below. This refers to migrating data older than a defined date for the first Full pass, and then performing a second Full (Delta) pass.

When following this strategy, you will not be selecting Pre-Stage from the Start drop-down list. For this scenario, you will first be migrating older data (using Date range filtering, which is set under the Advanced Options of the Project) via a Full Migration pass. Then you will perform another Full Migration (Delta) pass, having removed the date filtering.

It is very important to follow the exact steps, as outlined below. Connectivity problems and timeouts can cause Public Folder migrations to restart, which will increase the time for migration. The strategy, in conjunction with setting the proposed project Advanced Options, will provide optimal migration speeds.

Here are the MigrationWiz steps to perform a Public Folder migration. Note: The Source and Destination environments must be prepared before running through these steps. For details, refer to the migration guide that matches your scenario.

From the MigrationWiz Public Folder main dashboard, select the Public Folders that will be migrated. Click on the Start button then, from the drop-down list, choose Full Migration.

For smaller Public Folder migration projects (<20 GB), follow a Big Bang Migration strategy. This will migrate all data at once, rather than performing multiple migration passes. This will take time to complete, but it will avoid the issues indicated above. It is not necessary to add any Advanced Options flags for this type of migration strategy; all that is needed is to select Full Migration from the drop-down list, under the Start button.

For larger Public Folders, (>20 GB) follow a multi-pass strategy.

First pass:

Under project Advanced Options:

Set the MaintainWatermarkCompletionState flag. It should be added under the Advanced Options/Support/Support Only field in the project: MaintainWatermarkCompletionState=1

Set the date to migrate items older than six (6) months. This should be set under Advanced Options/Filtering/Only migrate items whose create date is before a specified date. Set the date to be six (6) months prior to the current date, and set the time to be 0/0.

Note: Make a note of this date, because during the Delta pass you will migrate items newer than this date.

Select the Public Folders to be migrated, click on the Start button, select Full migration, and click on the Start Migration button.

Second pass:

Under project Advanced Options:

Set the SkipImportFolderWhenPublicFolderExists flag, after we know there has been at least one item migrated. It should be added under the Advanced Options/Support/Support Only field in the project: SkipImportFolderWhenPublicFolderExists=1

Why it is important to add this flag? A separate call, in addition to the call for the initial retrieval of the folders, has to be made to retrieve permissions (i.e., cannot include permission in the initial list of properties to retrieve). This means that not only do we need to make a call to the Exchange server to retrieve all the folders at the current point in the hierarchy, but we have to make a separate call per folder to Exchange to retrieve that folder's current permissions. Due to the large number of folders, this can significantly increase the initial folder retrieval and migration. However, this can be skipped when running the Delta pass, since retrieval of folders, and permissions, were made during the first pass. It is skipped by setting the SkipImportFolderWhenPublicFolderExists=1Advanced Option. Note: If new folders or items are created after the first pass, this flag will not affect the retrieval of the folders and permissions for these, because it only skips existing folders. No new permissions should be added after the Pre-Stage pass, however, since it will skip migrating these if this flag is set.

Remove the MaintainWatermarkCompletionState=1 Advanced Option before running the Full (Delta) Migration pass.

Since the flag is removed, MigrationWiz will now run another index before the second pass begins. However, if the date filter is added in the next step, then this will vastly improve the speed of the second pass, since it won't have to re-index ALL Public Folders.

Set the date to migrate items newer than six (6) months. This should be set under Advanced Options/Filtering/Only migrate items whose create date is after a specified date.

An example of date filter: If, during the first pass, a date filter is set to migrate only items older than 2/1/2015, at 0 hours, 0 minutes, then before running the Full (Delta) Migration pass, it is necessary to enter a date filter under Advanced Options, to migrate items newer than 2/1/2015 at 0 hours, 0 minutes.

Be sure to remove the date filter set in the first pass, i.e., remove the checkmark from the box labeled Only migrate items whose create date is before a specified date.

By adding this date filter, MigrationWiz needs to index and thus watermark only recent items that were not included in the first migration pass.

This is important if the migrated users will still access the on-premises Public Folders during migration.

Select the Public Folders to be migrated, click on the Start button, select Full migration, and click on the Start Migration button.

Further information on why it is important to add the flag MaintainWatermarkCompletionState=1 during the first pass:

When there is a large amount of data in Public Folders, there can be a higher number of errors than in mailbox migrations. This could happen for a number of reasons, like corrupt items or incompatible MIME type items within the Public Folders. These errors slow the process down since MigrationWiz will be running many checks to try to migrate the items with errors. Without this flag, MigrationWiz will have to re-index all Source Public Folders again, before the migration can begin.

If a BitTitan server restarts due to Microsoft patching, it will have to begin scanning and indexing all over again before it can begin migrating. The flag will ensure that MigrationWiz can begin from where it left off, upon restart.

Note: We recommend that this should be set before the first migration pass. Remember to remove this prior to performing a second Full (Delta) migration pass, if there is an expectation of any new items to be added to the Public Folders after the first migration pass.

By default, MigrationWiz does not migrate extra email addresses (email aliases) that are assigned to a mail-enabled public folder. This means that alternate email addresses assigned to a public folder that is mail-enabled at the source will not be set up properly at the destination. Rather, only the primary email address is set, which is the name of the public folder (without spaces), plus the email domain (for example, myPublicFolder@myDomain.com).

If you wish to also migrate all of the alternate (aliased) email address, use the following PowerShell scripts to migrate both the primary as well as the extra (aliased) email addresses: https://help.bittitan.com/hc/en-us/articles/115008257228-Migrating-mail-enabled-public-folder-email-addresses

The following steps are recommended before migrating to help reduce the number of errors or delays you might experience in your Public Folder migration.

When your project is split by BitTitan Support

If there has been a SPLIT created for your project due to migrating more than 1000 folders, as per our article How do I split migrations based on child Public Folder count?, any new folders created on the source after the SPLIT has been created will NOT be included in the migration. It is recommended that you perform one of the following, as doing so will save you time, especially during longer migrations:

Block the ability to create new folders at the source.

Create a single top-level folder at the source and require that all new folders be created under that single folder. Once the migration is closer to completion, the name of that top-level folder can be added to the project as its own Root Path and migrated. Once the folder has finished migrating, the end users can then move their folders to their intended parent folders.

Exchange as the Source or Destination

If the source or destination is Exchange 2010+, disable throttling for the administrator account being used for the migration using the following steps:

Exchange Server 2010

To disable all throttling parameters for an admin account called "MigrationWiz":

On a computer that hosts the Microsoft Exchange Management Shell, open the Microsoft Exchange Management Shell.

Type the following command and press Enter:
New-ThrottlingPolicy MigrationWizPolicy

Type the following command and press Enter:
Set-ThrottlingPolicy MigrationWizPolicy -RCAMaxConcurrency $null -RCAPercentTimeInAD $null -RCAPercentTimeInCAS $null -RCAPercentTimeInMailboxRPC $null -EWSMaxConcurrency $null -EWSPercentTimeInAD $null -EWSPercentTimeInCAS $null -EWSPercentTimeInMailboxRPC $null -EWSMaxSubscriptions $null -EWSFastSearchTimeoutInSeconds $null -EWSFindCountLimit $null -CPAMaxConcurrency $null -CPAPercentTimeInCAS $null -CPAPercentTimeInMailboxRPC $null -CPUStartPercent $null

Type the following command and press Enter:
Set-Mailbox "MigrationWiz" -ThrottlingPolicy MigrationWizPolicy

Exchange Server 2013+

To disable all throttling parameters for an admin account called "MigrationWiz":

Open the Exchange Management Shell.

Type the following command and press Enter:
New-ThrottlingPolicy MigrationWizPolicy

Type the following command and press Enter:
Set-ThrottlingPolicy MigrationWizPolicy -RCAMaxConcurrency Unlimited -EWSMaxConcurrency Unlimited -EWSMaxSubscriptions Unlimited -CPAMaxConcurrency Unlimited -EwsCutoffBalance Unlimited -EwsMaxBurst Unlimited -EwsRechargeRate Unlimited

Type the following command and press Enter:
Set-Mailbox "MigrationWiz" -ThrottlingPolicy MigrationWizPolicy

Office 365 as the Destination

Increase the public folder quotas to Unlimited. The default Office 365 Public Folder quota setting is 2GB, with the issue warning set to 1.7GB. These steps are from our article Office 365 Public Folder Migration FAQ.

Run the following PowerShell commands against the destination tenant. (Note: After you apply these settings, wait at least 30 minutes before you start the migration.)

Open up a PowerShell session and enter the following. Then press Enter:

$LiveCred = Get-Credential
Install-Module -Name ExchangeOnlineManagement
Import-Module -Name ExchangeOnlineManagement
Connect-ExchangeOnline -ConnectionUri https://ps.outlook.com/powershell/ -Credential $LiveCred

Enter the following command and press Enter. This will show you the current public folder limits:
Get-OrganizationConfig | fl *DefaultPublicFolder*

Enter the following command and press Enter. This will set the new quota and issue warning quota to Unlimited:
Set-OrganizationConfig -DefaultPublicFolderProhibitPostQuota Unlimited -DefaultPublicFolderIssueWarningQuota Unlimited

Note: The public folder mailboxes must exist before this can be set. These steps can be found in our article Migrating large items/attachments.