Bits & Bytes

The BitTitan Blog for Service Providers

Stacey Farrar

Scaling Migrations Performance Considerations

In Sergio Leonne’s epic spaghetti western The Good the Bad and the Ugly, Tuco (the Ugly) says, “If you work for a living, why do you kill yourself working?”. This quote not only applies to many tasks in IT, but the phrase the good, the bad and the ugly can also aptly sum up many of the variables IT professionals encounter when planning or performing a migration.

When trying to calculate the timeframe of your next migration it is important to be aware of all the factors that might affect the performance of the migration. At BitTitan, we know your time is valuable, that is the main reason IT professionals use our software. However, while it is a worthwhile endeavor to calculate the amount of time your cut-over will take it is important to understand all the factors that are working with you and against you.

The Good

While having an amazing network with terabytes of throughput is certainly a checkbox in your favor comparing bandwidth to migration performance isn’t always 1:1. Network speed tests generate random bytes in memory and checks how quickly the data can be transferred. A speed test does not convert data, encrypt data, parse data, convert data, index data, read data from disk, write data to disk, or authenticate users. Time and speed will depend on your network speed, server load, throttling, and number of folders, there are some average speeds MigrationWiz users have reported:

  • Low end: 250MB per hr/per user
  • Mid end: 750MB per hr/per user
  • High end: 1.25GB per hr/per user

MigrationWiz will only open one connection per mailbox. This ensures we can scale to a large number of concurrent mailboxes. In this way MigrationWiz can migrate hundreds of thousands of mailboxes simultaneously. Migrating more mailboxes at the same time allows parallel processing, reducing the duration of your migration. If all your mailboxes are migrated at the same time, the migration duration is the time it takes to migrate the largest mailbox.

The Bad

There are certain practices that mailbox users do that make it more difficult to perform efficient migrations. The two most egregious examples are very large files sizes and mailboxes with many items. There is a per-item transaction overhead and time cost. For example, given two mailboxes of the same size but with a different number of items, the one with fewer items will migrate faster than the one with more items. The more data there is, the longer the migration. Four hundred users with 40MB of data will transfer far faster than one user with 4G of data. For these larger users, we recommend breaking the data into smaller files to speed up the migration. Essentially: more smaller users are faster than a few larger users. When migrating between two hosted providers, the size of the mailbox is unlikely to overload our bandwidth, but some Destinations might have capacity issues. When mailboxes exceed a certain size, speed reduction may occur because of those scalability issues.

The Ugly

Even if you can account for all the known variables, both good and bad there are still aspects of the migration that are unknown and out your control. Both Microsoft and Google have policies to throttle requests if they are hitting with too much frequency. Even if you are being very careful not to send too many requests there are other factors that come into play. It is important to keep in mind that during a migration you are using a shared network, in any given instance there could be another migration or equally heavy traffic that causes load balancing or other network protocols that affect traffic. These factors are moving targets and are almost impossible to prepare for.

The Solution

For all migrations if you have scoped your project and know there are some very large files or large number of users. The best thing you can do is perform a few test runs or a proof of concept (poc) with a broad data set. The type of data files should range from large too small. Then increase concurrence until failures appear. At this point back off the concurrence to a manageable threshold and give yourself a 50% buffer. This manner of proof of concept is the best way to hit the moving targets of migration performance. While migrations can have the good bad and ugly all intertwined as Clint Eastwood’s character (The Good) said in the movie, “I’ll sleep better knowing my good friend is by my side to protect me”, and with that you can be rest assured purchasing MigrationWiz licenses is your best choice for success.

Related Posts

Related Posts

Register for a FREE BitTitan Account

Create an account now and start planning your project.