This is to inform you about an issue with Oxford University email sent to Hotmail/Live/MSN email addresses. A University department misconfigured a mailing list of a large number people just before the weekend of 24/25 September in such a way that all its members could post to it. Many list members (external to Oxford) did that and it resulted in more than a million email messages being generated with large numbers of those destined for Hotmail/Live/MSN email addresses. Procedures Microsoft has in place to detect and stop spam email resulted in their servers blacklisting the servers in Oxford that sent outgoing email. This meant that mail sent to any domains for which the Hotmail servers handle mail was being rejected. It also caused a worse side-effect (manifesting itself as a mail loop, in some cases rapidly filling the Nexus mailbox) for anyone who had set their Nexus account to forward their email to such addresses.
In order to mitigate the ongoing problems caused by non-delivery, and the mail loops, OUCS put a trap in place to stop any mail for hotmail.com, hotmail.co.uk, live.com, live.co.uk, msn.com from leaving the University's servers. This has resulted in a large backlog of emails waiting to be delivered.
OUCS staff responded extremely quickly to the initial mailing list incident and had fixed it by lunchtime on Monday 26 September, however blacklisting went into place sometime on Tuesday 27 September. OUCS staff immediately followed the advertised Microsoft procedures for dealing with such incidents and OUCS has been trying tirelessly to resolve the ongoing block throughout the last week and over the weekend.
As of today, Monday 3 October, we believe the Microsoft blacklisting has been removed but we have to be extremely careful not to re-trigger it with too much mail too quickly. OUCS intends therefore to start releasing the queued mail in batches later today. We would like to bring to your attention, therefore, that email you may have sent to Hotmail/Live/MSN email addresses since Tuesday 27 September may now only just be delivered.
We estimate it might take a day but once the backlog has been cleared OUCS will remove the trap so that emails to hotmail.com, hotmail.co.uk, live.com, live.co.uk, msn.com from Oxford University should be delivered normally. We were not trapping Hotmail domains for other countries so emails sent to them should already be being delivered normally.
We would like to say how sorry we are that you and the wider University have experienced such a disruptive incident at such a critical time of that academic year. We will be requesting a full investigation in to how a department was able to cause such disruption and how to prevent this in the future. We will also be looking into how to improve communications with Microsoft and a mechanism to deal with its blacklists rather more quickly should anything similar ever happen again.
Oxford University Computing Services