MigrationWiz: Duplicate Prevention

MigrationWiz uses a system of watermarks to track items that have previously been migrated. There are a several advantages to using watermarks instead of detecting items by their names, titles, or other properties.

How are watermarks created and stored for later use?

As items are migrated, a small watermark is embedded in the header of each item. This allows us to track it once it resides in the Destination. Watermarks also convey information such as the endpoint type specified in MigrationWiz. 

Additionally, a matching watermark is stored in one of our databases. We do not store any content data from the item, only the watermark data. The database holds the items that have been migrated and prevents items from being remigrated if they are deleted by users at the Destination.

Because watermarks are tracked in two different places, there is a double layer of protection against the creation of duplicates.

How does MigrationWiz use watermarks in additional migration passes?

When another migration pass begins, the Destination is scanned for watermarks so we will know what data has been migrated. The watermark database is also scanned to prevent the re-migration of items that were deleted or moved at the Destination between passes. This scanning process occurs at the beginning of every migration pass. The larger the quantity of data migrated, the longer this process will take. For example, a migration that previously transferred 20GB of data will take longer to scan for watermarks than a migration with 20MB transferred.​

Note: Watermarked items are not migrated a second time. Only items found without a watermark will be migrated.

What happens to items at the Source?

Data at the Source will not be modified. this includes watermarks.

How do I delete watermarks?

  • If a user manually deletes a migrated item from the Destination, it will not be migrated because it will still be tracked on the back end.
  • To delete our end of the watermarks database, check the box next to the migration you want to do this for, then click the circular Reset Items button on the toolbar at the top of the user list. This will only clear our end of the database.
  • If you would like to delete watermarks stored at the Destination, you need to delete the items themselves, as the watermarks are embedded in the item headers. This is typically done only when restarting your migration from scratch. Deleting items at the Destination will only clear the Destination watermarks. Unless the watermarks are reset on our end, items deleted from the Destination will not be remigrated.

To learn more about resetting items and item statistics, see resetting item statistics.

Additional notes:

  • Because we use proprietary watermarks that are specific to our system, we will not be able to detect items migrated by other means. For example, items forwarded from the Source to the Destination (also items migrated by other solutions) will not contain our watermarks. Therefore, we have no knowledge of them existing at the Destination. As a result, we will migrate these items even if they are already at the Destination.
  • Since we store the endpoint type in the watermark, changing the Source or Destination endpoint type in between passes will void the watermark and all items will be migrated again. If this happens, a new watermark will be generated in each item based on the new Source or Destination endpoint type.
  • If an item at the Source is moved in between passes, we will not be able to match the Source item with its Destination counterpart. As a result, the item will be migrated a second time. It will then exist in both locations​.
Was this article helpful?
5 out of 5 found this helpful