Lotus Notes Extractor FAQ

Lotus Extractor Setup: Lotusextractorsetup.exe

Multiple Extractors can be set up to support your migration from Lotus Notes, or Lotus Domino, to Destinations, such as Office 365.

The Extractors need to be set up on physical or virtual machines that are part of the customer's Local Area Network.

The Lotus Extractors will communicate directly with the Source Lotus Notes or Domino servers.

The Lotus Extractor will then be used for the communication to BitTitan servers. Communication to BitTitan occurs as follows: 

  • To BitTitan MSPComplete and MigrationWiz platforms to retrieve the endpoints and submitted mailboxes.
  • To one or more migration servers in the cloud (to receive migration commands and push Lotus Notes data).

The diagram below provides an overview of this mechanism.

 

screenshot     

This method is best suited to large-scale migrations, intended for large businesses and distributed environments.

Read this Knowledge Base article for detailed configuration information on the Lotus Extractor: KB004286.

Notes:

  • Regarding connectivity of the Lotus Extractors, there is no need for additional connectivity, like external IP addresses, since the communication is initiated by the Extractor.
  • We recommend that customers do not work with their mail files during the migration. We also recommend that custom Domino agents, if any, working with mail files, are stopped during the migration to avoid document locking.
  • Install one Lotus Extractor per machine (or virtual machine). MigrationWiz can migrate up to 15 mailboxes per Extractor. ​Each running Lotus Extractor requires a separate endpoint in MSPComplete and a separate machine (or virtual machine).​​

Running the Lotus Extractor

To run Lotus Extractor, you need to have the following:

  • A MigrationWiz account
  • A Lotus Notes administrative account
  • Web access
  • Recommended hardware: 3.0 GHZ Quad Core processor or better. 
  • A Windows-based computer that is not the Lotus Notes server
    • An ideal configuration would be 8+ GB of RAM with Windows 7 or Server 2008+
  • The machine you run the Extractor on can be either 32-bit or 64-bit
  • .NET Framework version 4.6.1 or later installed on the computer on which you install and run the Extractor. To determine the .NET Framework version currently installed, follow instructions provided by Microsoft: How to: Determine Which .NET Framework Versions Are Installed​​.​
  • Lotus Notes v6.5+ client installed.
    We only support migrations from Domino Server v6.5 to v9.0.1.
  • Free disk space for saving Lotus Extractor logs
    • The size of the required free space may be estimated using the following formula: 100MB + 50MB * (number of mailboxes to be migrated). For very large mailboxes (10GB or larger), the log file size per mailbox may reach 100MB.

Important Details:

  • The Lotus Extractor cannot be run on a machine with Domino Administrator or any Lotus Server components installed.
  • Depending on system and server resources, Lotus Extractor can migrate up to 15 concurrent mailboxes per instance.
  • Only one instance of Lotus Extractor can run per physical or virtual machine.
  • Each instance of Lotus Extractor requires a separate mailbox migration project in MigrationWiz.
  • When running Domino on A/S 400, we recommend first running a proof of concept to validate that your A/S 400 version is supported. Our QA department has not tested all versions of A/S 400, so we cannot guarantee that all are supported.

Related Downloads: Lotus Extractor Setup: Lotusextractorsetup.exe

Troubleshooting

The Lotus Extractor just disappears

Historically when this happens we see there is nothing in the logs leading up to the crash. For this, we will have you run an extractor in a PowerShell session with transcribing enabled which should catch what is happening at least in the shell leading up to the crash. To do this, open a PowerShell session on the extractor machine and run the following:
Open the Lotus Extractor to make sure you have the latest version. Once the version is determined

Start-Transcript
cd $ENV:APPDATA
cd LotusExtractor
$name = Get-ChildItem | ?{$_.name match "App"} | sort -Descending | Select -First 1
cd $name.name
.\LotusExtractor.exe

This will start the transcript in the PowerShell session. If the extractor dies, run the following:

Stop-Transcript

This will let you know where the transcript is located. Please navigate to that location, zip the transcript .txt, and reply to this ticket with that .zip file attached.

Navigate to the %localappdata% directory on the extractor, add the entire BitTitan folder to a .zip file, include the name of the extractor in the zip file name, and add it as an attachment to your Support ticket. If the files are larger than 20MB, please contact Support and let them know you need an FTP location created to upload the logs.

The Shared Memory error

The "Shared Memory from a previous Notes/Domino run has been detected, this process will exit now" error thrown during migrations occur due to the Lotus Notes Client being built using a shared-memory architecture. This means the Lotus Notes Client creates a shared memory space when it runs.

The Lotus Extractor uses the Lotus Notes Client library to perform migrations, during which the Lotus Notes Client writes to the shared memory space created. Unfortunately, portions of the shared memory space being used by one thread can get overwritten by other threads before the original thread is done, or the shared memory block can become corrupted, but the Lotus Notes Client still tries to use the shared memory space.

In either case, the only way to clear the shared memory space is to follow the steps in our article [How do I troubleshoot the "Lotus Notes Shared Memory Error"?]https://help.bittitan.com/hc/en-us/articles/115008260388-How-do-I-troubleshoot-the-Lotus-Notes-Shared-Memory-Error-) when the "Shared Memory from a previous Notes/Domino run has been detected, this process will exit now" is thrown.

If the error is thrown and the OK button is clicked the error window will close, but the same shared memory space is still being used. This is why after clicking the OK button the error may not come back but migrations will eventually fail even though the extractor looks like it is processing migrations. If the steps are not followed, the error will eventually come back up.

Items causing issues

Having a lot of Delivery Failure Report Messages and Calendar/meeting update items (i.e. meeting cancellation, meeting acceptance, meeting date/time update, etc.) in a folder.
Delivery Failure Report Messages take a lot of time to process before they are found to be Delivery Failure Report items types, which are not migrated by MigrationWiz. If there are multiple Delivery Failure Report, this slows things down considerably. The more Delivery Failure Report that exists in a folder that needs to be processed, the slower things will move along and may cause the `GetFullItems` or `GetAlltems` command to not return anything within the timeout threshold for the command, causing this error and failing the migration.

If this is the case you can either delete all of the Delivery Failure Report items from the source mailbox, or create a new folder named Delivery Failure Report in the mailbox, move all Delivery Failure Report items (Note: Move the items, do not copy them) in their mailbox to the folder Delivery Failure Report, and filter out the folder Delivery Failure Report from the migration by following the steps in our [How do I filter a folder?](https://help.bittitan.com/hc/en-us/articles/115008101527-How-do-I-filter-a-folder-) knowledge base article. This is the same case for Calendar/Meeting acceptance items, such as meeting cancellations, meeting acceptance, meeting declined, meeting date/time updates, etc.. The same can be done for these item types. They can either be deleted, or a new folder created for them, the items moved to that folder, and that folder excluded from the migration by following the steps in our [How do I filter a folder?](https://help.bittitan.com/hc/en-us/articles/115008101527-How-do-I-filter-a-folder-) knowledge base article.

Having a lot of corrupt items in the Inbox folder.

MigrationWiz will attempt to repair corrupt items the best it can but is not always able to repair corrupted items. Corrupted items also take a lot of time to process before they are determined to be too corrupt to fix and moving to the next item, causing the `GetFullItems` or `GetAlltems` command to not return anything within the timeout threshold for the command, causing this error and failing the migration. Have your Domino Admin run an offline defrag of the NSF file, or a FixUp against the end-users NSF file on the source Domino server then re-submit the migration.

Having too many items in the Inbox folder. 

If there is a very large number of items in a folder, lets us Inbox as an example folder, the `GetFullItems` or `GetAlltems` command may fail to return anything within the timeout threshold for the command and cause this error, failing the migration. If there are more than 20,000 items in the Inbox folder you will need to create sub-folders under the Inbox folder and distribute the content of the Inbox folder equally across the number of sub-folders created. I would recommend not putting any more than 15,000 items in each sub-folder. An example would be if an Inbox folder has 60,000 items in it, you would:

  1. Create 4 subfolders under the Inbox folder named something like Inbox-1, Inbox-2, Inbox-3, and Inbox-4

  2. Move 15,000 items from the Inbox to Inbox-1, another 15,000 items from the Inbox to *Inbox-2*, etc., until all items are distributed across all 4 sub-folders equally. Note: Move the items, do not copy them.

  3. Follow the process in our knowledge base article [How do I reset statistics for my item(s)?](https://help.bittitan.com/hc/en-us/articles/115008259728-How-do-I-reset-statistics- Re-submit the migration.

  1. Having a lot of large items in a folder.
    Having a lot of large-sized items in a folder, lets use Inbox as an example folder, can cause GetFulltems failures as well. If this is the case:

    1. Create subfolders under the Inbox folder named something like Inbox-LargeItems-1, Inbox-LargeItems-2, etc.

    2. Sort items in the Inbox folder by size in descending order

    3. Move all items, say, larger than 10MB, equally distributing them, to the newly created subfolders

    4. Follow the process in our knowledge base article [How do I reset statistics for my item(s)?](https://help.bittitan.com/hc/en-us/articles/115008259728-How-do-I-reset-statistics- Re-submit the migration

Multiple extractors and large concurrent migrations

Multiple extractors can alleviate some issues and provide higher migration volue, however, there are some things to consider, the biggest being not to over-submit the number of migrations that can be handled per Domino servers themselves, and the other that there should be no more than 3 extractors pointed to a single Domino server, with no more than 5 migrations running on any of those extractors at any given time.

A common misconception is that the more extractors being used the more throughput can be gained during Lotus Notes migrations. The bulk of migrations are processed on the Domino server itself, such as copying items, performing item conversion, cleaning up copied and converted items, then process the next batch of items, consistently. This means if there are 8 migrations processing all at the same time, the Domino server resources are going to be heavily used and in most cases, the resource will be exhausted if too many migrations are run concurrently.

The number of concurrent migrations needs to be determined on a per-migration basis, meaning, if migrations are coming from a Domino Server DOMSRV01, then it will need to be determined how many migrations can be done on DOMSRV01 concurrently. This may be a bit painful because the process is to do a single migration and wait for it to complete, then determine how the performance was. If the performance is good, increase the Maximum number of concurrent migrations of a MigratoinWiz project from 1 to 2, submit 2 migrations and let them complete, then determine how the performance was. If the performance was good, increase to 3, let 3 migrations complete successfully and determine how that performance was, etc., increasing the number to find out what a good number of concurrent migrations is. If the number is 6, and you are using 3 extractors, then you should have no more than 2 migrations per extractor at any one time. If the number is 4, then only run 4 at a time across multiple extractors, or 4 on one extractor if it is easier to track that way.

The bad news here is that if running 1 migration works and running more than 1 crashes or causes problems, you will need to run 1 migration at a time. If it is one particular migration causing problems, such is the case you seem to be having, you will want to run the trouble migrations 1 at a time, with no other migrations running from any other extractors pointing to that Domino server until those migrations are completed.

Domino servers of version 10 or higher are not supported as a MigrationWiz migration source.

Why are there frequent updates being made to the Lotus Extractor?

The Lotus Notes Extractor is part of a larger system which is built using "Continuous integration". The larger system depends on other supporting libraries (logging, reporting etc.), which must be rebuilt continuously to remain synchronized.

The entire system is deployed at once, which results in the daily updates.

Is there any way to make the Lotus Notes Extractor more self healing? 

We have added significant code to try to react to certain errors that are thrown by the IBM Lotus Notes Client itself (unfortunately we do not have control over this library)

In some cases we are able to reset the connection and retry the request.

Have you ever set up the Lotus Notes Extractor to restart itself on a regular basis using PowerShell or batch? 

Currently we don't have any evidence that periodic restarts are going to give a benefit but this is also something that could be implemented at the customer-VM level.

The architecture of our software is such that we actually start a new Windows Process for each mailbox extraction so this gives us a level of protection.

Unfortunately the IBM client does use Windows shared memory so this can cause issues between those processes. Sometimes Windows has trouble killing these processes.

In such cases we recommend a full reboot in order to clear out any IBM Lotus Notes process, threads or blocks of shared memory as the best remediation strategy.

Is there any way to set it up as a service so that it doesn't need to be installed with the user profile of the admin? 

Not currently but we will add this as a feature request into our system.

How many mailboxes can be migrated at a time?

You can migrate up to 15 mailboxes simultaneously. Once you have created your migration project, go to MigrationWiz advanced options and set the number of concurrent migrations to 15. 

Where should the Lotus Extractor be deployed?

Deploy the Lotus Extractor to the same local area network where the target Domino server is located. However, do not deploy the extractor to a computer on which the Domino server is deployed.
Deploy no more than one extractor instance to a given computer.

Was this article helpful?
0 out of 1 found this helpful