This section of the Nexus pages contains information about how the migration from the previous Herald email system to Microsoft Exchange, locally called Nexus, was achieved.

1. The use of username@nexus

The following announcement was made regarding this issue on 30 June 2009.

OUCS has always heavily discouraged username@machine for use as an email address, principally because it causes confusion and email address changes when services are replaced or renamed. That position will not change with the introduction of the Oxford Nexus groupware solution.

We recognize however some units have 'Direct Deliver' status for their email, i.e. where email is sent direct to the unit Mail Transport Agent (MTA) regardless of what is in front of the "@" symbol. These units have to use username@machine-type addresses to route users' mail to the right place, including to Herald and, in future, Nexus. This may be direct routing from the unit MTA or it may be from another machine within the unit.

For the above reason, Nexus will be able receive email sent to username@nexus but we intend to refer to such addresses as "routing addresses" rather than "email addresses" and ask you always to inform your users that their address is of the form first.last@unit.ox.ac.uk. This should avoid any future confusion or email address changes as the Nexus service develops end evolves.

2. The use of username@herald

In the same email of 30 June 2009:

Please note that during the migration period, we will make our best endeavours to have herald forward emails to username@herald onto the user's Nexus account, once their migration has happened. After all Herald accounts have been migrated and that service approaches decommissioning (not before 2010) then these routings will cease to work. Adequate warning will be given of this but we strongly urge you to stop using username@herald routing addresses in scripts etc. that email people as soon as you feasibly can. You can make LDAP lookups for people's "real" email addresses in scripts and send to them. Please see http://www.oucs.ox.ac.uk/services/oak/sp/ldap/using_ldap_client_software_with_oak.xml for more information on how to do this in various different scripting languages.

Separate provision will be made for projects and student clubs and societies that currently have account@herald email addresses information about this will follow in the near future.

3. Migration

3.1. What was migrated?

The following items were migrated from Herald/Webmail:

  • Email data from Herald IMAP mailstore

  • Contacts from Webmail

  • Preferred Sender from Webmail (if this was an Oxford address associated with the user)

  • Signature from Webmail

  • Quota (if >2GB, otherwise was set to 2GB)

  • Vacation Message/Setting

  • Forwarding Addresses

  • Whitelist/Blacklist

  • Move to Junk Mail SPAM setting (see below)

The following items didn’t get migrated:

  • Discard SPAM threshold

  • Webmail Contact Groups

  • IMAP flags

  • IMAP folder subscription list

3.2. How did the SPAM setting map across?

Nexus has SPAM filter settings of Low, Medium, High and Off, which mapped to a Herald equivalent of >7, >5, >3, and 0 respectively. The value from Herald was mapped across as per below:

0 -> Off

1 -> High(>3)

2 -> High(>3)

3 -> High(>3)

4 -> Medium(>5)

5 -> Medium(>5)

6 -> Medium(>5)

7 -> Low(>7)

8 -> Low(>7)

9 -> Low(>7)

10 -> Low(>7)

>10 -> Off

The ‘automatically delete SPAM over a certain threshold’ setting was not migrated to Nexus.

3.3. How did the items get migrated?

 Various techniques were used to push items from Herald to Nexus, all data was encrypted in transit via HTTPS or SSL secured IMAP.

  • Email - IMAP to Exchange Webs Services conversion

  • Contacts - Exchange Webs Services

  • Preferred Sender - HTTPS to powershell

  • Signature - Screenscape OWA Light

  • Quota - HTTPS to powershell

  • Vacation Message/Setting - Exchange Webs Services

  • Forwarding Addresses - HTTPS to powershell

  • Whitelist/Blacklist - Screenscape OWA Light

  • SPAM settings - HTTPS to powershell

  • Corrupt messages found during the migration process

 Some messages couldn’t be migrated as some element of their MIME structure was corrupt, or they couldn’t be stored in Exchange for some other reason. When a corrupt message was detected its subject was logged (and only the subject was logged). At the end of the migration process, the system walked the Herald mailbox and located all messages with subject lines that matched known corrupt messages. All those messages were bundled in one or more zip files. The majority of the messages in the zip file would have actually been in the Exchange datastore, but because only the subject of the corrupt message was logged we were unable to determine which message on Herald with that subject was the corrupt one, and thus we had to place all messages into the zip file.

 This ensured no data was lost during the migration process and left it down to the end-user's software to try to figure out the content. Some clients were more tolerant of these corrupt messages than others, but all would display an approximation of the "true" intended message.

 For some, the corrupt messages email/attachment were confusing and generated support requests, but this was the best we could do when moving the email between these different systems.

 We found a particular issue with files which had a mime type of application/applefile. A number of corrupt messages we saw had files of other types marked as applefile and this caused issues. If a user receivedan email with their corrupt messages in it or an email saying their migration was stopped due to the number of corrupt messages, we recommended searching for attachments with that mime type first as they were usually the cause of the problem.

 We also found that the zip files we produced would not open on some Mac machines, although they were fine on Windows, Linux/UNIX and other Macs. If the zip file appeared corrupt when opening on a Mac, we advised opening it on another machine, preferably a PC.

3.4. Error Codes in the Migration Picker

If an account failed to migrate it was marked the next morning (circa 9am) in the migration picker. For each failed account, a failure code was available. Codes were added/removed as issues were found and solved. The last used codes are listed below with an explanation for each.

Code Explanation Response
Non-Herald Routing This user didn’t route their main email address to Herald. None
ForwardAddress=PreferredAddress The forwarding address on Herald was the same as the address used by the Herald account to send email  - not supported on Nexus Change forwarding on Webmail prior to re-migration
ForwardAddress=HeraldAddress The forwarding address on Herald was set to username@herald.ox.ac.uk - this is not supported on Nexus Change forwarding on Webmail prior to re-migration
ForwardAddress=NexusAddress The forwarding address on Herald was set to username@nexus.ox.ac.uk - this is not supported on Nexus Change forwarding on Webmail prior to re-migration
Early Failure - Unknown An error occurred early in the migration process. This usually occurred when the migration system was under too much load. OUCS will investigate and schedule re-migration
Case Two folders with the same name but different case were found (for example, Sent and sent) Change folder names
Quotes One or more folders with double quotes in their name were found Change folder names
Slash One or more folders with trailing slashes in their name were found Change folder names
Clash One or more folders with reserved names that could not be remapped to foldername_Herald, as foldername_Herald already exists, were found Change folder names
Folder Select StartIndex A problem was found opening a folder on Herald - a folder rename is required Change folder names on OUCS advice
Mig-Failed Folder Read(1) A problem was found opening a folder on Herald/creating a folder on Nexus OUCS will investigate and schedule re-migration
Mig-Failed Folder Read(2) A problem was found opening a folder on Herald/creating a folder on Nexus OUCS will investigate and schedule re-migration
Mig-Failed Exch Folder Create A problem was found creating a folder on Nexus OUCS will investigate and schedule re-migration
Mig-Slash One or more folders with trailing slashes in their name were found Change folder names
EWS-Error A system error occurred during migration. A cached copy of the old user object prior to the mailbox being added stopped the migration tool accessing the true mailbox. This was more likely to occur at the start of the migration window. OUCS will investigate and schedule re-migration
Mig-Unknown A system error occurred during migration OUCS will investigate and schedule re-migration
Mig-Transporter A system error occurred during migration OUCS will investigate and schedule re-migration
Mig-Transporter PS A system error occurred during migration OUCS will investigate and schedule re-migration
Mig-Transporter(Died) A system error occurred during migration OUCS will investigate and schedule re-migration
Forward Corrupt A number of corrupt messages were found that could not be migrated. There were either too many messages or they were too large to bundle into a zip file and forward on Examine messages with subject lines as per personal failure email. Remove messages where possible and contact OUCS to re-schedule migration
Large Msg in Mailbox A number of corrupt messages were found that could not be migrated. One or more of these were too large to bundle into a zip file and forward on Examine messages with subject lines as per personal failure email. Remove messages where possible and contact OUCS to re-schedule migration
(no code) Ran out of time to migrate account or the account was disabled OUCS will re-schedule migration, most likely to a Sunday in the next couple of weeks. OUCS will contact all users informing them of the new migration date.

NB: The owner of the account being migrated should have got a message detailing what they needed to do or a generic "Unknown Error" message.

4. Pre-Migration

4.1. Quotes

Quotes (") in folder names couldn’t be migrated. They can be stored on Nexus, but the migration tool in use wouldn’t migrate them. We emailed users with quotes in folder names.

[HELPDESK] If a user said they had changed their folder names, we advised them to double check this in WebReg and say ‘thanks’ if they had, or assist them if the rename had not worked.

4.2.  Slashes

Slashes on the end of folder names sometimes caused email to be misfiled after migration. A slash at the end implied another folder existed inside that one. As with quotes we emailed users.

[HELPDESK] If a user said they had changed their folder names, we asked them to double check this in WebReg and say ‘thanks’ if they had or assist if the rename had not worked.

4.3. Case

 Exchange is case-insensitive and dovecot (the IMAP server on Herald) was case-sensitive. Users were being contacted on a three-weekly cycle to rename these folders. On migration these folders were merged, however the migration process checked for this condition, emailed the user and failed the migration.

4.4.  Reserved Names

 Some folder names such as "Calendar" are reserved in Exchange. If these folder names were in use, they were re-mapped on migration as below:

Herald folder Oxford Nexus folder
Calendar   Calendar_Herald
Contacts Contacts_Herald
Notes Notes_Herald
Tasks Tasks_Herald
Outbox   Outbox_Herald
Junk   Junk_Herald

5. During Migration

5.1.  Checking what is happening

We arranged that, on logging on to https://webmail.ox.ac.uk/ a message stated that a migration was in progress and when it started was also displayed. The migration process was not dependant on the size of the mailbox, but rather the number of messages in the mailbox. Each mailbox could be migrated at a rate of approximately 20,000 messages per hour, and 36 migrations could be run in parallel across the six migration servers in the migration cluster.

6. After Migration

 If it works
IMAP connections continued as before, but it was sometimes necessary to re-subscribe to folders. Webmail users found a message pointing to the new Outlook Web Access (OWA) service when they attempted to connect to Herald Webmail.

The username.herald.ox.ac.uk IMAP connection address was remapped to the Nexus IMAP server. Connections were presented with an appropriate Herald certificate and could connect using the same credentials. We hoped this made the migration as seamless as possible to IMAP clients.

 What Username/Password do I use?
 The system uses the Oxford (WebAuth) username and password.
 If it fails
 Full service was restored on Herald and we investigated the reason for the failure. If the reason was fairly simple, such as quotes in a folder name, the user received an email informing them of the nature of the problem and pointing them at a web link to fix it.
 How did I change my forwarding?
 These settings were changed on the self-registration web-site: https://register.it.ox.ac.uk/self/index/
 How did I change my preferred email address?
 These settings were changed on the self-registration web-site: https://register.it.ox.ac.uk/self/index/
 How did I change my SPAM Settings?
 These settings were changed on the self-registration web-site: https://register.it.ox.ac.uk/self/index/

7. Connectivity

 OWA reported my account is disabled
 First we advised you to check on Webmail to see if the account had been migrated. If it was still being migrated then all access was disabled. If migration had finished, then it could have been that some information was cached either in the browser, or on the server. We advised logging off Nexus and then Logging on again to see if the account was still disabled. If so, the next step was to try restarting the browser. If neither of these worked, then information might have been cached on the web server. The caches were set to clear after 30 minutes. If access still appeared to be disabled one hour after the migration had finished, we advised you to contact: help@oucs.ox.ac.uk

[HELPDESK] This can be checked on WebReg.

 I couldn’t log into my new account after migration
 Occasionally, for users who did not use WebAuth-protected sites on a regular basis, passwords may not have been synchronized to Nexus. We advised visiting the old Webmail, and logging in. A few minutes later your password should have synchronized with Nexus.

8. Outlook Problems

Outlook couldn’t autodiscover my account
Autodiscover relies on a DNS entry of the form "autodiscover.emailsuffix.ox.ac.uk" which needs to be an alias for autodiscoverredirect.nexus.ox.ac.uk. We contacted ITSS in batches with text similar to the below:
                        As I hope you're well aware, we are proceeding well with early 
                        adopters to the (Exchange) Nexus groupware service right now. This message
                        concerns units that have Outlook users. Please read on, even  if you only
                        have a few Outlook users, as this may make their and your  lives a little
                        easier.  
                        
                        Outlook 2007 can be configured so that it discovers all of its 
                        'technical' server settings automatically. This is called "autodiscover"  and
                        can make life a lot easier for users and their support staff. 
                        See  Outlook 2007 for a little background. 
                        
                        In order for Autodiscover to work, every domain that is also an
                        email domain needs to register an alias (not an A record) pointing to 
                        autodiscoverredirect.nexus.ox.ac.uk. 
                        
                        For example, the @oucs.ox.ac.uk domain
                        needs an alias record of  autodiscover.oucs.ox.ac.uk in the DNS. 
                        

[modification to original note] Note that the situation below no longer exists - IT staff can use the regular DNS interface to enter their autodiscover alias in the alias section without any OUCS involvement

                        We feel that we should not create these uninvited, so please could you email
                        (hostmaster email link) with a 
                        request to create an alias for the domain that you administer? For your Outlook 2007 
                        users, it will make an enormous difference in the easy configuration of the client.
                        
                        If you could do this soon, we will have a good chance of creating the aliases before 
                        your users are scheduled to be migrated. 
                        
                        As a final note, we expect that Outlook users - who are currently  connected to Herald 
                        via IMAP - will not need to change any settings at the time of the migration (as 
                        username.herald.ox.ac.uk will route initially). However, in order to gain all of the 
                        benefits of Exchange (the calendar, tasks, out of office, etc. etc.), the settings should  
                        be changed to connect via 'Outlook Anywhere'.
                        
                        With Outlook 2007, autodiscover is the most  painless way of achieving this.

Autodiscovery can also fail if the account is not found in the Global Address List. See My account does not appear in the Global Address List above.

I can't see free/busy information in Outlook 2007
 This was often caused by the lack of an autodiscover alias in the DNS. Please see the above item 'Outlook cannot autodiscover my account' as this could have resolved the issue. Note that free/busy information provision works differently in Outlook 2003 and you may not see the problem with that client (although the information can be less up to date).
My off-line address book is not being downloaded in Outlook 2007
 One cause of this problem was also the lack of an autodiscover alias in the DNS. Please see the above item 'Outlook cannot autodiscover my account' as this could resolve the issue.
Out of office problem: "the server is currently unavailable"
With Outlook 2007 (but not 2003), you could have seen the following message when trying to amend your ‘Out of Office’ settings via Tools - Out of Office Assistant:

Your Out of Office settings cannot be displayed, because the server is currently unavailable. Try again later.

This may also have been caused by the lack of an autodiscover alias in the DNS. Please see the above item 'Outlook cannot autodiscover my account' as this could resolve the issue. Note that the easy work-around was to set up the Out of Office information in Outlook Web Access.

 Repeated Password Prompt
 With an Outlook Anywhere connection via Outlook 2010 it offered a method to cache your password and avoid a prompt each time Outlook was started.
 Emailing a recently migrated collegue failed
If the failure information included diagnostic text similar to:
                        From: Microsoft Exchange
Sent: 21 July 2009 15:07
To: XXXXXXXXXXXXXXXXX
Subject: Undeliverable: XXXXXXXXXXXXXXXXX Delivery has failed to these recipients or distribution lists: Test User<mailto:IMCEAEX-_O%3DNEXUS_OU%3DEXCHANGE%2B20ADMINISTRATIVE% 2B20GROUP%2B20%2B28FYDIBOHF23SPDLT%2B29_CN%3DRECIPIENTS_CN% 3DTestuser1@ad.oak.ox.ac.uk> The recipient's e-mail address was not found in the recipient's e-mail system. Microsoft Exchange will not try to redeliver this message for you. Please check the e-mail address and try resending this message, or provide the following diagnostic text to your system administrator.
Sent by Microsoft Exchange Server 2007

The solution to this issue can be found on the non-delivery page.

Outlook does not reconnect via IMAP after migration
 On some machines using Outlook, a seamless migration for IMAP client users was not possible. It appeared that domain-joined machines did not correctly remember the IMAP credentials after the account has been migrated to Nexus: they needed to be re-entered. After re-entering the credentials IMAP worked as before, but folders may have needed to be resubscribed.

9. IMAP Problems

Odd items appearing in IMAP mailboxes
Odd messages appear in e.g. Thunderbird after the user has setup Outlook 2007. These have message subjects similar to:
               Subject: Outlook Message Manager (Nexus) (KEY:EF858013EDF13B4796FCC546AB439DFD)
               
The 'message' itself was empty but the headers stated:
 Thread-Topic: Outlook Message Manager (Nexus) (KEY:
                  EF858013EDF13B4796FCC546AB439DFD)
               Message-ID:<7105CC05C1D8264BB17497808993B2394190FE0E01@EXMBX01.ad.oak.ox.ac.uk> 
               Accept-Language: en-GB, en-US Content-Language: en-US
                  Content-Type: text/plain;
               charset="iso-8859-1" Content-Transfer-Encoding:
                  quoted-printable MIME-Version: 1.0
            

 These messages were produced by Outlook and were not meant to be viewable to the user. After a short time Thunderbird rendered them hidden from view again.

IMAP Message not retrievable
 A message is dispalyed similar to below:
                  Subject: Retrieval failed using IMAP4 protocol for
                     message: 14222
                  From: Microsoft Exchange 2007
                  To: XXXXXXXXXXXXXXXXXXXX
                  Exchange 2007 IMAP4 server failed to retrieve the
                     following message:
                 
                  Subject: XXXXXXXXXXXXXXXXXXXXXXXXX
                 From: XXXXXXXXXXXXXXXXXXXXXXXXXXXx
                  Sent Date: 23/03/2009 13:40:31
                 
                  The message could not be retrieved using the IMAP4
                     protocol. The message has not been deleted and may be accessible using
                     either Microsoft Outlook or Microsoft Office Outlook Web Access. You
                     can also try contacting the original sender of the message to
                     find out about the contents of the message.
                 
                  Retrieval of this message will be retried when the
                     server is updated with a fix that addresses the problem.
                  

 We did not identify a fix for this issue.

IMAP Connection Problem Post-migration
 The herald IMAP address (username.herald.ox.ac.uk) was remapped to nexus as part of the migration process. Although the Time-To-Live setting was relatively low at 5 minutes, some caching DNS servers may have caused problems. If connectivity was an issue, we advised you to first try restarting the email client software then investigating the DNS entry for your IMAP connection. At a shell prompt on a Unix/Linux machine, in a terminal windows on OS X, or a cmd prompt on Windows, we advised you to type: nslookup username.herald.ox.ac.uk

 The responses should have included herald.nexus.ox.ac.uk rather than an imapXXX.herald.ox.ac.uk address.

Server:         163.1.2.1

Address:        163.1.2.1#53

Non-authoritative answer:

username.herald.ox.ac.uk  canonical name = herald.nexus.ox.ac.uk.

Name:   herald.nexus.ox.ac.uk

Address: 163.1.154.193

 If all else fails, we advised reconfiguring the IMAP client using the appropriate Nexus instructions and connecting via imap.nexus.ox.ac.uk.

IMAP Connection Problem Post-migration (Apple Mail)
 We have had a couple of reports of issues where Apple Mail did not realise the server had changed, or started repeatedly prompting for credentials. We only saw this for a very small number of people, all other Apple Mail migrations worked well and only required re-subscribing to folders. If you suffered this problem, we advised reconfiguring to use the imap.nexus.ox.ac.uk IMAP server.
X Headers are Missing after Migration
From what figured out, this was the product of a change in the behaviour of Exchange 2007 SP1 that appeared to have been introduced with rollup 8 on 19 May 2009, to fix another issue.

The Exchange development blogged on this issue and their blog post implied that the Exchange server no longer preserves X headers which came from "Anonymous submissions" (basically email from outside Exchange). Any header seen before the update to rollup 8 should have been preserved, however Nexus was implemented with rollup 8 installed.

Note: the oxmail SPAM headers are acted upon by the Junk mail filtering we have built into the Nexus implementation, but the headers are not preserved and passed on the IMAP or other cleints.

10. Other problems

Cannot log in (project account)
If a project account was migrated and a user couldn’t log in to Nexus, they just needed to connect to *anything* that was Webauthed using that project account's username and password. After a few minutes, they would then be able to log into Nexus using that username and password.
Birthdays late on iPhone and iPod
Birthdays may have appeared off by one day when synced to an iPhone or iPod from Outlook 2003 or Outlook 2007 Contacts. Apple described how to fix this on an Apple support page
Problem sending messages with attachment to Exchange account
Nexus/MS Exchange cannot accept attachments of the wrong MIME type, so Word .docs sent as application/applefile instead of application/msword (for example when using Thunderbird on a Mac) will be rejected. mimeTypes.rdf in the Thunderbird profile stores this kind of setting. Deleting that and re-running Thunderbird (with the normal profile) fixed the problem. Further details can be found on the non-delivery page.
Email does not get routed outside Nexus
 If you redirected mail to another mail server at the oxmail level, and you did not forward mail from Nexus onto this other location, then you can end up with email in two places. Exchange will shortcut the email routing, so if anyone in Nexus sent something to the address Nexus has associated with you, it ended up in your Nexus account. Measures were put in place to ensure email forwarding was automatically updated to match the oxmail routings where possible. For details of this process see the Email Routing page.

A second common cause of this issue was where the Nexus account had been assigned the email address username@herald.ox.ac.uk (this would have been migrated from the settings on Herald). When the user was selected from the GAL, email would route to Nexus. When their long form address was used it would have been routed outside. If the user did not wish to keep their Nexus account separate from their main account, elsewhere, then we advised you to let us know.