SharePoint to SharePoint Online Document Library Migration Guide (including Microsoft 365 Groups)

This guide may be used for the following scenarios:

  • SharePoint On-Premises to SharePoint Online
  • SharePoint Online to SharePoint Online

This guide does not support GCC High or China migrations. For a GCC or China migration, see SharePoint to SharePoint GCC/China migration guide.

This article outlines the workflow for migrating document libraries from a SharePoint source tenant to SharePoint Online destination tenant. This migration requires licenses.  This guide may also be used to migrate Microsoft 365 Groups (Documents) and File Share instances. 

If this is your first time performing a migration, we have created a Migration Planning & Strategy Guide to walk you through planning, set-up, and general migration best practices. If you have never performed a migration before, we suggest reading that before beginning the steps outlined in this scenario. 

MigrationWiz is a migration tool, not a syncing tool. We do not have “live” monitoring of changes (as with a sync agent) and we cannot handle scenarios such as conflict resolution without user interaction. 

Due to limitations on connections allowed by GoDaddy, we do not support migrating to or from GoDaddy using this migration type.

GCC High and MFA

We are not able to support migrations with 2-factor or multifactor authentication. This endpoint does not support GCC High migrations at this time. 

Information about the types of item migrated with this scenario, including metadata and versions, can be found in the following dropdown. More information about metadata and versions can be found at Versions & Metadata Migration for SharePoint & OneDrive.

Document Properties Limitation

Microsoft Word document properties in Content Controls are not retained at the destination after migration. References to custom metadata values within a document appear broken after being migrated to the destination document library. This is a Microsoft limitation and can be replicated by copying a basic document to a new document library within the same tenant without the use of MigrationWiz. 

We have provided a workaround for this issue.   

Microsoft 365 Groups (Document) Migrations

Effective June 28, 2021, Microsoft 365 Group (Documents) source and destination endpoints will be deprecated. Microsoft 365 Groups will be migrated as part of SharePoint to SharePoint migrations. 

Which Items Are and Are Not Migrated

MigrationWiz supports migration of documents, folders, permissions, versions, and metadata set at document library levels.

Migrated

Document libraries

  • Documents

  • Folders / Folder Structure

  • Permissions

  • Versions (Max total 25 versions)

  • Custom Metadata

    • Single line of text

    • Choice

    • Currency

    • Date and time

    • Number

    • Person or Group (User/Group should be available at destination)

    • Yes/No

    • URL/hyperlink

    • Picture

    • Location

    • Calculated (Supported for calculation based on other supported columns)

  • Migrations of over 15GB are now supported, but this must be set up in Advanced Options. These settings will prevent timeout errors when a Speedster import takes more than 10 minutes to complete.
    • Set LargeFileMigrations=1
    • Set LargeFileMigrationsTimeout=7200000
      • 720000 value is an example; time is measured in milliseconds

Custom metadata limitations

  • Multiple lines of text: Migrated as single lines of text; migrated values cannot be updated at the destination

  • Task Outcomes: Migrated as single line of text to destination

  • External data: Not supported

  • Managed metadata: Not supported

  • Lookup: Not supported

Not Migrated

  • Site/Site Collection

  • Site logos and customization

  • Document library settings

  • Custom permissions

  • Lists

  • Custom task

  • Newsfeed

  • Any metadata referencing information from outside the document library (such as lists or other site-level data) is not migrated. For example: external data, managed metadata, lookup, retention policy tags/retention labels.
  • Shared permissions for files/folders with symbols in name 
  • Links giving access/Link sharing

Preparing the Source

Optional: Save library templates. This step is only required if you want to apply templates from your source libraries onto your destination libraries. 

Application Permissions for SharePoint

Follow the steps to enable permission level at the source. This authentication process gives you control over who is entitled to use the source.

  1. Sign in as a Global Admin.
  2. Go to MigrationWiz-SharePoint-FullControl and consent to the app access when prompted.
  3. Navigate to the Microsoft 365 Admin Portal and create a new Security Group named “MigrationWiz”. 
  4. Create new user.
  5. Add new user to previously created MigrationWiz security group as a member. These user credentials are also required during setup of the SharePoint source endpoint.
  6. Add the following advanced option to the project during the advanced option phase: UseApplicationPermission=1 

Steps to remove these permissions are provided below in the Post-Migration section.  

Full-Control vs Read-Only

Why does the source require consent to a Full-Control application permission?

All AMR migrations require full-control permission. If you have a specific need to not allow full-control permissions, you can use MigrationWiz-SharePoint-ReadOnly (only for the source). However, please note that with read-only permissions, MigrationWiz will not export document permissions, versioning or metadata, and cannot use AMR.

Prepare Azure Environment

Microsoft-provided Azure storage

We recommend using Microsoft-provided Azure storage for this migration. Refer to Microsoft documentation for more details. If you choose this option, skip to Prepare Destination Environment.

Own Azure storage

If you plan to use your own Azure storage, refer the following steps to prepare your Azure environment. We recommend that you create your Azure storage account in the same Microsoft data center as the destination Microsoft 365 tenant. You do not need to create any Azure containers for this migration.

  1. Estimate Azure storage costs. This step is optional but is useful in order to provide the customer with up-front storage costs ahead of time. 
  2. Buy an Azure subscription (or use the free one-month trial, and be aware that this option is only viable if you are performing a very small migration). 
  3. Create an Azure storage account. You will need to set up a STORAGE (General Purpose v2) account rather than a storage blob.
  4. Take note of the storage account name and the primary access key. (In Azure, from the storage screen, click Manage Access Keys at the bottom of the screen.) These need to be entered into the MigrationWiz migration project when specifying the destination settings. 

To create the Azure storage account:

    1. Visit the Azure portal.
    2. Click New.
    3. Select Storage > Storage account.
    4. Enter a name for your storage account.
    5. Choose Resource manager for the Deployment model.
    6. Choose STORAGE (General Purpose v2)
    7. Record the storage account name (-accesskey, example: “accountname”) and primary access key (-secretkey, example: “W1RrDfkPNkfYfdVqizMNJjn5mXchwMP5uYBY8MsMqWTA7EubG911+4fZlki0Gag==”)
    8. In the Replication field, select Locally Redundant Storage (LRS).
    9. Select the subscription in which you want to create the new storage account.
    10. Specify a new resource group or select an existing resource group.
    11. Select the geographic location for your storage account.
    12. Click Create to create the storage account.

Now the storage account appears in the storage list.

Prepare Destination Environment

  1. Create a SharePoint Online administrator or Site Collection administrator account to be used for migration or use the global admin account for the tenant.
  2. Create a SharePoint site and make a note of the site URL where the document libraries are stored.
  3. Ensure all necessary users/groups are set up at the destination SharePoint site.

If you wish to maintain the same look, feel, and design of your source libraries on your destination, follow these additional steps before starting the migration project:

  1. Create the structure of the document libraries.
  2. Create the actual document libraries on the destination SharePoint Online site. To learn how, follow the steps from Microsoft: Create your document library.
  3. Save library templates from the source and apply them to your destination libraries.

Application Permissions for SharePoint

Follow the steps to enable permission levels at the destination. This authentication process gives you control over who is entitled to use the destination.

  1. Ensure you are signed in as a Global Admin.
  2. Go to MigrationWiz-SharePoint-FullControl and consent to the app access when prompted.
  3. Create a new Security Group named “MigrationWiz” on the Microsoft 365 Admin Portal.
  4. Create new user.
  5. Add the new user to the previously created MigrationWiz security group as a member.
  6. Record these credentials for use during destination endpoint setup.

Steps to remove these permissions are provided below in the Post-Migration section.  

MigrationWiz Steps

Licenses

This migration scenario uses the Shared Document License type. 

  • This license type is used only for SharePoint Online to SharePoint Online migrations.
  • One license per line item in project (i.e. per document library migration)
  • Data limit: 50 GB per license

To purchase licenses:

  1. Sign in to your BitTitan account. 
  2. In the top navigation bar, click Purchase.
  3. Click the Select button and choose the license type you need.
  4. Enter the number of licenses you want to purchase. Click Buy Now.
  5. Enter a Billing address if applicable.
  6. Click Next.
  7. Review the Order Summary and enter a payment method.
  8. Click Place Your Order.

​​Licenses are released once payment has been received:

  • If you purchase via credit card, licenses will be available immediately upon payment.
  • If you purchase via wire transfer (100+ licenses), licenses will be available once payment has been received and accepted.

Set up a Document Migration project

  1. Sign in to your MigrationWiz account.
  2. Click Go To My Projects.
  3. Click Create Project.
  4. Click on Document Project to create new project.
  5. Click Next Step.
  6. Enter a project name and select a Customer. 
  7. Click Next Step.
  8. Select a Source Endpoint from the endpoint dropdown menu.
  9. If you have not already added the source endpoint, you will need to click New to create the source endpoint. Fill in the fields below to set up a new source endpoint:
    1. Name: Type any name you want for the endpoint.
    2. Endpoint Type: Select SharePoint from the source endpoint type drop-down list.
    3. URL: Enter the URL for the source SharePoint tenant.
    4. Administrator Username/Password: Enter the email for the user authorized to run the migration for the source tenant. This is the user added to the MigrationWiz security group for source tenant.
    5. Click Add.
  10. Click Next Step.
  11. Select a destination endpoint from the endpoint dropdown menu.
  12. If you have not already added the source endpoint, you will need to click New to create the destination endpoint. Fill in the fields below to set up the destination endpoint:
    1. Name: Type any name you want for the endpoint.
    2. Endpoint Type: Select SharePoint Online from the destination endpoint type drop-down list.
    3. SharePoint Online Tenant URL: Enter the URL for the destination SharePoint tenant.
    4. Office 365 Username/Password: Enter email for the user authorized to run the migration for the destination tenant. 
    5. Use the credentials of the Microsoft 365 user which was added to the MigrationWiz security group for the destination tenant.
    6. Select Which Azure Storage to use
      • If using Custom Azure Storage enter the Storage Account Name and Access Key that was acquired in the `Prepare Azure Storage` section earlier in this guide.
    7. Click Add.
  13. Click Save and Go to Summary.
  14. Click Save Project.

Add Line Items

There are three options for adding items. Choose which one you will use, then follow the steps below. While Autodiscover is recommended, Bulk Add and Quick Add may also be used.

Discovered Sites

Autodiscovery discovers both Communication and Team sites but it filters out the Team sites which are linked to Teams channels. Currently a Microsoft Teams site can be created in one of two ways:

  1. Adding Team site from SharePoint - which can be later linked to Teams channels.

  2. Creating a Teams channel - which creates a SharePoint site by default.

Autodiscover will filter out out the second one automatically, and may filter Team sites connected to some Teams channels in the first case. 

Important: Currently autodiscovery will not be able to filter out Teams Private channels, so those need to be manually removed.

Autodiscover Prerequisites

  1. The source tenant must have one of the following permissions enabled:

    1. MigrationWiz-SharePoint-FullControl - Recommended. Use with the Advanced Option set as UseApplicationPermission=1 or UseApplicationPermissionAtSource=1

    2. MigrationWiz-SharePoint-Delegated

Important: If you are using Delegated permissions with Global Admin credentials, this process will only discover the SharePoint sites for which the Global Admin is also site admin. For example, the Global Admin should be present here under Site admin in Permissions for this site to be discovered.AD1.png

 

Steps to Enable Autodiscover

This section outlines the workflow to use the SharePoint Autodiscover option to automatically discover all document libraries for the sites and subsites available for migration under the SharePoint site URL provided in the source settings.

  1. Create a new Document Project and set the URL under the source endpoint to the required root site. Details on allowed source URL configuration can be found later in this article.

  2. If MigrationWiz-SharePoint-FullControl permissions are being used at the source, set the UseApplicationPermission=1 or UseApplicationPermissionAtSource=1 as Advanced Options.

  3. Click the Autodiscover-items button. AD2.png

  4. Click Start Autodiscover.

  5. Once discovery is complete, click on ‘Import Items’ to add the discovered sites and document libraries as line items to the project.

    AD3.png

     

  6. Once imported, the discovered items will show up as line items.

    AD4.png

Source URL Configuration

Source URL Settings

Discovered Items

Root_url
Example: https://mdbttest123.sharepoint.com

Will discover all the document libraries under root URL and its subsites (both Team and Communication sites).

Note: This will exclude MS Teams Site URLs (URLs linked to Teams), but will not exclude Teams Private Channel URLs.
Example: This below site and its corresponding Document Libraries will be filtered out:

5.png

Root_url/sites
Example: https://mdbttest123.sharepoint.com/sites

This is an incorrect format and will return an error.

Root_url/sites/site_name
Example: https://mdbttest123.sharepoint.com/sites/TestSite2

Will discover all the document libraries under this site and its subsites.

Limitations

  1. Currently the MigrationWiz-SharePoint-ReadOnly permission at the source is not supported and will show a permission error.
  2. For sites with a high number of items (Document libraries + subsites) it might take a bit longer. A higher number of subsites is more likely to increase this time than a high number of document libraries, although there are exceptions. 
    For example, our tests show that, for 3k+ items, it takes between 30 minutes and an hour to auto-discover.
  3. If the root URL is provided, it will show all the sites under it (including all sub sites).
    1. This will exclude Teams Site URLs, but will still include Teams Private Channel URLs due to a Microsoft limitation. 
    2. After importing, the specific document libraries can still be removed from the line items before starting migration
  4. On-Premises SharePoint is not supported
Autodiscover FAQ

The Document Library name has special characters but that item is not showing when importing items:
If the Document Library has special characters, the URL may not reflect that. Additionally, the imported item in the project is based on the URL, not the name.

AD5.png

The count of Import Items seems to be different if different permissions are used:
MigrationWiz-SharePoint-FullControl has more permission access than MigrationWiz-SharePoint-Delegated. It is possible when using Delegated permissions that  the global admin may not have permission to access some sites, meaning those will not get auto-discovered. Thus the count may differ.

Autodiscovery shows Teams URLs:
SharePoint Autodiscovery will discover both Teams and Communication Sites.
But it will automatically filter out Teams SharePoint sites which are not Private Channels. Teams Private Channel SharePoint URLs need to be manually removed by the customer after importing as line items.

How long will it take to autodiscover all items:
The following are time estimates to discover all document libraries under the provided site URL. These are estimations only, your results may vary.
~100 items - <1 minute
~900 - 1k items - <5 minutes
~2k - 3k+ items - <0.5 - 1 hour

Manually Add Items

Quick Add: This option allows you to add items o​ne at a time. You have to enter Source library and destination library path for each line item.

Bulk Add: This option allows you to add multiple items at once by copying and pasting from a spreadsheet or by importing a CSV file.

Add the source and destination Document Library path for each line item. We recommend that you use the document library path from the URL.

The following examples show the breakdown of each type of URL.

Example 1: Document library located under a site:

Library1.png

Example 2: Document library located under a sub-site:

Library2.png

Example 3: Document library located directly under SharePoint Online root site:

Library3.png

Example 4: Document library located directly under SharePoint Online root sub-site:

Library4.png

Example 5: Site and Document Library names may contain special characters, but the URL may not reflect the same. It’s important that the document library path is captured correctly from the URL.

Library5.png
Library6.png

Site Provisioning

SharePoint sites are not automatically provisioned at the destination. 

License

This is a pre-migration step and does not consume licenses. With this new feature, we are now able to provision sites at the destination, similar to how we provision document libraries that do not yet exist. This is an optional step. 

Prerequisites

The source tenant must have either one of the following permissions enabled:

  1. MigrationWiz-SharePoint-FullControl (Recommended)
    (with AO set as UseApplicationPermission=1 or UseApplicationPermissionAtSource=1)

  2. MigrationWiz-SharePoint-Delegated

If using Delegated permissions with Global Admin user credentials, we recommend that the Global Admin should also be added as an administrator of the site collection. If this is not done, the site provisioning may fail. For example, the Global Admin user should be present here under site admin in permissions.Provision1.png

Create SharePoint-SharePoint Online project 

Only SharePoint source and SharePoint Online destination endpoints are supported currently. ​Provision2.png

Create a line item

Use Quick Add/Bulk Add/ Auto Discovery to add the source and destination library paths for a line item in the project.Provision3.png

Add the ‘TestSharePointWithProjectConfigUrl’ Advanced Option

Go to the Advanced options page and add TestSharePointWithProjectConfigUrl=1 . This allows you to verify credentials even if the destination site does not exist.Provision4.png

Select ‘Setting Up SharePoint’

To provision sites and subsites before performing migration, click on the Full Migration and select only the Setting Up SharePoint --> Sites Creation step and proceed with ‘Start Migration’. This will create all the required sites and subsites at the destination.

Provisioning6.png

Select Migrate Data

Once the “Setting Up SharePoint” step is complete, proceed with “Migrate Data” to complete migration of the site content.

Supported authentication mode

Authentication mode*

Is supported

Remarks

Basic auth (User name + Password)

No

 

Application permission (BT app)

Yes

Credentials of user added to the “MigrationWiz” security group

Delegated permission (BT app)

Yes

GA/SP Admin credential

Delegated permission (BYO app)

Yes

GA/SP Admin credential

*Applies to both source and destination.

Set the Project Advanced Options

In the Support tab, add:

  • InitializationTimeout=8- This increases the initialization timeout window to eight hours. This value is in hours, up to a maximum of 100 hours. Values above 100 are in milliseconds. For example:​
    • InitializationTimeout=2 will increase the timeout to 2 hours.
    • InitializationTimeout=8 will increase the timeout to 8 hours.
    • InitializationTimeout=14400000 will increase the timeout to 4 hours.
    • InitializationTimeout=21600000 will increase the timeout to 6 hours.
  • Add UseApplicationPermission=1 if you did not do so during the app permissions phase. This is required for the application permission to function at the source.
  • Migrations of over 15GB are now supported. These settings will prevent timeout errors when a Speedster import takes more than 10 minutes to complete.
    • Set LargeFileMigrations=1
    • Set LargeFileMigrationsTimeout=7200000
      • 720000 value is an example; time is measured in milliseconds

If the UPN prefix changes

User mapping: By default, we match users from the source to the destination based on the prefix in their User Principle Name (UPN). For example, if the user UPN is “name@domain.com”, we match the “name” portion.

If there are multiple people with the same prefix at the destination (e.g. name@domain1.com and name@domain2.com, or the prefix of a particular user is changed in the destination (e.g. name@source.comname.full@destination.com), it is necessary to use the advanced option UserMapping="name@source.com->full.name@destination.com" to set the new or correct name for each user.

  1. The User Mapping command goes in the Support Options section and will require one line per user that needs mapping. Click the + to add additional lines. Replace the UPN addresses in the example with the actual UPN addresses. 

Under Source/Destination tab

Set number of Versions to be migrated as per project requirement and click Save.

  • The minimum number of versions is 1, the maximum number is 25. 
Advanced_Option.png

Verify Credentials

You may verify the credentials of items in MigrationWiz without migrating data or consuming any licenses.

  1. Open the project containing line items you wish to validate​.

  2. Select the items you wish to validate.

  3. Click on the Start button in your dashboard.

  4. Select Verify Credentials from the drop-down list.

  5. Once complete, the results of the verification will be shown in the Status section.

Perform a Full Migration

  • SPO Migration API migrates documents and permissions together. We recommend you select & migrate Documents and Document Permissions together.

  • To migrate metadata, the metadata checkbox must be selected during the first migration run.  

To initiate migration of document libraries:

  1. Open the Project containing the items you wish to migrate.

  2. Select the items to migrate.

  3. Click on the Start button in your dashboard.

  4. Select Full Migration from the drop-down list.

  5. Ensure that the Documents, Document Permissions, Metadata, and Versioning checkboxes are selected. By default, the latest three versions will be migrated. The number of versions to migrate can be updated in the Advanced Options section on the source/destination tab. See the Versions section below for more information.

    Items_to_Migrate.png
  6. Click Start Migration.

  7. Once complete, the results of the migration will be shown in the Status section.

Post-Migration Steps

  1. To prevent users from inadvertently using the Source SharePoint libraries, decommission the Source SharePoint server, libraries, or user accounts.

  2. Delete all the Azure containers used for this migration. This will prevent incurring post-migration costs. Be careful to only delete the containers created for this migration.

Remove App Permissions

  1. Remove the newly created user.

  2. Remove the MigrationWiz Security Group.  

  3. To remove the app from the source or destination, perform the following steps:

    1. Launch PowerShell.

    2. Connect PowerShell to Microsoft 365.

    3. Enter the command: Connect-AzureAD

    4. Enter the admin credentials in the prompt.

    5. Enter the command: Get-AzureADServicePrincipal -SearchString Migration

    6. Look for the ObjectId of the app you want to remove and enter the following command: Remove-AzureADServicePrincipal -objectId <the object id>

Group Permissions

MigrationWiz will migrate group permissions if the same group name exists at the destination. If the identically named group is not created at the destination prior to migration, group permissions will not migrate. MigrationWiz will not create groups at the destination, this must be done manually during destination setup.

Group permission migration does not support the RecipientMapping or UserMapping advanced options. 

Can I set up source & destination endpoints with SharePoint site URL?  

Yes, you can set up your source or destination endpoints with site-level URL.  

For example, if all of your document libraries at the source or destination are directly under a single site, you may prefer to set up the endpoint URL in this format: https://tenant.sharepoint.com/sites/site-name 

If your source/destination endpoint is set up with a SharePoint site URL, your line item should only include the library name and not the site path. 

Was this article helpful?
0 out of 0 found this helpful