Contents
This section of the Nexus pages contains information about how the migration from the previous Herald email system to Microsoft Exchange, locally called Nexus, was achieved.
The following announcement was made regarding this issue on 30 June 2009.
OUCS has always heavily discouraged username@machine for use as an email address, principally because it causes confusion and email address changes when services are replaced or renamed. That position will not change with the introduction of the Oxford Nexus groupware solution.
We recognize however some units have 'Direct Deliver' status for their email, i.e. where email is sent direct to the unit Mail Transport Agent (MTA) regardless of what is in front of the "@" symbol. These units have to use username@machine-type addresses to route users' mail to the right place, including to Herald and, in future, Nexus. This may be direct routing from the unit MTA or it may be from another machine within the unit.
For the above reason, Nexus will be able receive email sent to username@nexus but we intend to refer to such addresses as "routing addresses" rather than "email addresses" and ask you always to inform your users that their address is of the form first.last@unit.ox.ac.uk. This should avoid any future confusion or email address changes as the Nexus service develops end evolves.
In the same email of 30 June 2009:
Please note that during the migration period, we will make our best endeavours to have herald forward emails to username@herald onto the user's Nexus account, once their migration has happened. After all Herald accounts have been migrated and that service approaches decommissioning (not before 2010) then these routings will cease to work. Adequate warning will be given of this but we strongly urge you to stop using username@herald routing addresses in scripts etc. that email people as soon as you feasibly can. You can make LDAP lookups for people's "real" email addresses in scripts and send to them. Please see http://www.oucs.ox.ac.uk/services/oak/sp/ldap/using_ldap_client_software_with_oak.xml for more information on how to do this in various different scripting languages.
Separate provision will be made for projects and student clubs and societies that currently have account@herald email addresses information about this will follow in the near future.
The following items were migrated from Herald/Webmail:
The following items didn’t get migrated:
3.2. How did the SPAM setting map across?
Nexus has SPAM filter settings of Low, Medium, High and Off, which mapped to a Herald equivalent of >7, >5, >3, and 0 respectively. The value from Herald was mapped across as per below:
The ‘automatically delete SPAM over a certain threshold’ setting was not migrated to Nexus.
3.3. How did the items get migrated?
Various techniques were used to push items from Herald to Nexus, all data was encrypted in transit via HTTPS or SSL secured IMAP.
Some messages couldn’t be migrated as some element of their MIME structure was corrupt, or they couldn’t be stored in Exchange for some other reason. When a corrupt message was detected its subject was logged (and only the subject was logged). At the end of the migration process, the system walked the Herald mailbox and located all messages with subject lines that matched known corrupt messages. All those messages were bundled in one or more zip files. The majority of the messages in the zip file would have actually been in the Exchange datastore, but because only the subject of the corrupt message was logged we were unable to determine which message on Herald with that subject was the corrupt one, and thus we had to place all messages into the zip file.
This ensured no data was lost during the migration process and left it down to the end-user's software to try to figure out the content. Some clients were more tolerant of these corrupt messages than others, but all would display an approximation of the "true" intended message.
For some, the corrupt messages email/attachment were confusing and generated support requests, but this was the best we could do when moving the email between these different systems.
We found a particular issue with files which had a mime type of application/applefile. A number of corrupt messages we saw had files of other types marked as applefile and this caused issues. If a user receivedan email with their corrupt messages in it or an email saying their migration was stopped due to the number of corrupt messages, we recommended searching for attachments with that mime type first as they were usually the cause of the problem.
We also found that the zip files we produced would not open on some Mac machines, although they were fine on Windows, Linux/UNIX and other Macs. If the zip file appeared corrupt when opening on a Mac, we advised opening it on another machine, preferably a PC.
3.4. Error Codes in the Migration Picker
If an account failed to migrate it was marked the next morning (circa 9am) in the migration picker. For each failed account, a failure code was available. Codes were added/removed as issues were found and solved. The last used codes are listed below with an explanation for each.
NB: The owner of the account being migrated should have got a message detailing what they needed to do or a generic "Unknown Error" message.
Quotes (") in folder names couldn’t be migrated. They can be stored on Nexus, but the migration tool in use wouldn’t migrate them. We emailed users with quotes in folder names.
[HELPDESK] If a user said they had changed their folder names, we advised them to double check this in WebReg and say ‘thanks’ if they had, or assist them if the rename had not worked.
Slashes on the end of folder names sometimes caused email to be misfiled after migration. A slash at the end implied another folder existed inside that one. As with quotes we emailed users.
[HELPDESK] If a user said they had changed their folder names, we asked them to double check this in WebReg and say ‘thanks’ if they had or assist if the rename had not worked.
Exchange is case-insensitive and dovecot (the IMAP server on Herald) was case-sensitive. Users were being contacted on a three-weekly cycle to rename these folders. On migration these folders were merged, however the migration process checked for this condition, emailed the user and failed the migration.
Some folder names such as "Calendar" are reserved in Exchange. If these folder names were in use, they were re-mapped on migration as below:
| Herald folder | Oxford Nexus folder | |
| Calendar | Calendar_Herald | |
| Contacts | Contacts_Herald | |
| Notes | Notes_Herald | |
| Tasks | Tasks_Herald | |
| Outbox | Outbox_Herald | |
| Junk | Junk_Herald |
5.1. Checking what is happening
We arranged that, on logging on to https://webmail.ox.ac.uk/ a message stated that a migration was in progress and when it started was also displayed. The migration process was not dependant on the size of the mailbox, but rather the number of messages in the mailbox. Each mailbox could be migrated at a rate of approximately 20,000 messages per hour, and 36 migrations could be run in parallel across the six migration servers in the migration cluster.
The username.herald.ox.ac.uk IMAP connection address was remapped to the Nexus IMAP server. Connections were presented with an appropriate Herald certificate and could connect using the same credentials. We hoped this made the migration as seamless as possible to IMAP clients.
autodiscover.emailsuffix.ox.ac.uk" which needs to be
an alias for autodiscoverredirect.nexus.ox.ac.uk. We contacted ITSS in batches with text
similar to the below:
As I hope you're well aware, we are proceeding well with early
adopters to the (Exchange) Nexus groupware service right now. This message
concerns units that have Outlook users. Please read on, even if you only
have a few Outlook users, as this may make their and your lives a little
easier.
Outlook 2007 can be configured so that it discovers all of its
'technical' server settings automatically. This is called "autodiscover" and
can make life a lot easier for users and their support staff.
See Outlook 2007 for a little background.
In order for Autodiscover to work, every domain that is also an
email domain needs to register an alias (not an A record) pointing to
autodiscoverredirect.nexus.ox.ac.uk.
For example, the @oucs.ox.ac.uk domain
needs an alias record of autodiscover.oucs.ox.ac.uk in the DNS.
[modification to original note] Note that the situation below no longer exists - IT staff can use the regular DNS interface to enter their autodiscover alias in the alias section without any OUCS involvement
We feel that we should not create these uninvited, so please could you email
(hostmaster email link) with a
request to create an alias for the domain that you administer? For your Outlook 2007
users, it will make an enormous difference in the easy configuration of the client.
If you could do this soon, we will have a good chance of creating the aliases before
your users are scheduled to be migrated.
As a final note, we expect that Outlook users - who are currently connected to Herald
via IMAP - will not need to change any settings at the time of the migration (as
username.herald.ox.ac.uk will route initially). However, in order to gain all of the
benefits of Exchange (the calendar, tasks, out of office, etc. etc.), the settings should
be changed to connect via 'Outlook Anywhere'.
With Outlook 2007, autodiscover is the most painless way of achieving this.
Autodiscovery can also fail if the account is not found in the Global Address List. See My account does not appear in the Global Address List above.
Your Out of Office settings cannot be displayed, because the server is currently unavailable. Try again later.
This may also have been caused by the lack of an autodiscover alias in the DNS. Please see the above item 'Outlook cannot autodiscover my account' as this could resolve the issue. Note that the easy work-around was to set up the Out of Office information in Outlook Web Access.
From: Microsoft Exchange
Sent: 21 July 2009 15:07
To: XXXXXXXXXXXXXXXXX
Subject: Undeliverable: XXXXXXXXXXXXXXXXX
Delivery has failed to these recipients or distribution lists:
Test User<mailto:IMCEAEX-_O%3DNEXUS_OU%3DEXCHANGE%2B20ADMINISTRATIVE%
2B20GROUP%2B20%2B28FYDIBOHF23SPDLT%2B29_CN%3DRECIPIENTS_CN%
3DTestuser1@ad.oak.ox.ac.uk>
The recipient's e-mail address was not found
in the recipient's e-mail system. Microsoft Exchange will not try to redeliver
this message for you. Please check the e-mail address and try resending this
message, or provide the following diagnostic text to your system administrator.
Sent by Microsoft Exchange Server 2007
The solution to this issue can be found on the non-delivery page.
Subject: Outlook Message Manager (Nexus) (KEY:EF858013EDF13B4796FCC546AB439DFD)
The 'message' itself was empty but the headers stated:
Thread-Topic: Outlook Message Manager (Nexus) (KEY:
EF858013EDF13B4796FCC546AB439DFD)
Message-ID:<7105CC05C1D8264BB17497808993B2394190FE0E01@EXMBX01.ad.oak.ox.ac.uk>
Accept-Language: en-GB, en-US Content-Language: en-US
Content-Type: text/plain;
charset="iso-8859-1" Content-Transfer-Encoding:
quoted-printable MIME-Version: 1.0
These messages were produced by Outlook and were not meant to be viewable to the user. After a short time Thunderbird rendered them hidden from view again.
Subject: Retrieval failed using IMAP4 protocol for
message: 14222
From: Microsoft Exchange 2007
To: XXXXXXXXXXXXXXXXXXXX
Exchange 2007 IMAP4 server failed to retrieve the
following message:
Subject: XXXXXXXXXXXXXXXXXXXXXXXXX
From: XXXXXXXXXXXXXXXXXXXXXXXXXXXx
Sent Date: 23/03/2009 13:40:31
The message could not be retrieved using the IMAP4
protocol. The message has not been deleted and may be accessible using
either Microsoft Outlook or Microsoft Office Outlook Web Access. You
can also try contacting the original sender of the message to
find out about the contents of the message.
Retrieval of this message will be retried when the
server is updated with a fix that addresses the problem.
nslookup username.herald.ox.ac.uk
The responses should have included herald.nexus.ox.ac.uk rather than an imapXXX.herald.ox.ac.uk address.
username.herald.ox.ac.uk canonical name = herald.nexus.ox.ac.uk.
If all else fails, we advised reconfiguring the IMAP client using the appropriate Nexus instructions and connecting via imap.nexus.ox.ac.uk.
The Exchange development blogged on this issue and their blog post implied that the Exchange server no longer preserves X headers which came from "Anonymous submissions" (basically email from outside Exchange). Any header seen before the update to rollup 8 should have been preserved, however Nexus was implemented with rollup 8 installed.
Note: the oxmail SPAM headers are acted upon by the Junk mail filtering we have built into the Nexus implementation, but the headers are not preserved and passed on the IMAP or other cleints.
A second common cause of this issue was where the Nexus account had been assigned the email address username@herald.ox.ac.uk (this would have been migrated from the settings on Herald). When the user was selected from the GAL, email would route to Nexus. When their long form address was used it would have been routed outside. If the user did not wish to keep their Nexus account separate from their main account, elsewhere, then we advised you to let us know.