2. Network Infrastructure

2.1. Connection to Janet

The connection of the University network to Janet (SuperJanet3) is provided by means of a 34 Mbps ATM link shared with Oxford Brookes. Outgoing traffic bandwidth is set by UKERNA (which operates Janet) on the instruction of JISC at 8Mbps. The incoming bandwidth is unconstrained, except by the effective 25Mbps capacity of the line (and the capacity consumed by Brookes (which has risen to as high as 30% of Oxford's at times).

Growth in incoming traffic has risen inexorably [figure 1], doubling approximately every year. It is clear that Oxford will only barely survive with no serious congestion before the connection to SuperJanet4 (at 622Mbps) is commissioned early in 2001. Making use of the Web cache compulsory for non-ac.uk traffic has only had a marginal effect on growth, since the local hit rate is only modest. Of course, the value of this move is experienced in other directions (savings in traffic charges — see below).
The outbound traffic is rather less predictable, albeit significantly smaller [figure 2]. There has been some growth during the year, but nothing to cause alarm bells to ring. However, one group of researchers (in particular) did experience congestion on the link early in 2000 (especially when making interactive connections to remote computers). After some investigation, OUCS was not absolutely able to pin down the cause of this congestion, but it was concluded that the most straightforward solution would be to upgrade the link from 8Mbps to 12Mbps. This was arranged with UKERNA, at a modest cost, and was implemented in May 2000. There have been no reports of similar problems since, so this may have resolved the problem, at least for now. The ultimate solution will be the connection to SuperJanet4 early in 2001.

It is noteworthy that the patterns of usage of incoming and outgoing traffic through the day are remarkably similar, indicting activity matched to a UK work cycle [figures 3 & 4]. The peaks and troughs of activity on incoming traffic are predictable (the troughs being in the early hours of the morning when Oxford users are at their quietest). But the fact that the outgoing traffic follows much the same pattern suggests it too is being largely generated by Oxford users, rather than by, say, American surfers devouring Oxford's Web pages!

2.2. SuperJanet4

UKERNA's contract with Cable & Wireless for Janet (SuperJanet3) expires at the end of March 2001. Considerable effort, coordinated by JISC, has been expended nationally to develop a replacement. Comprehensive Spending Review funds were secured to ensure a capital injection sufficient to be able to control ongoing costs. The general principle was expounded (though not actually made compulsory) that Janet henceforth would be responsible for interconnecting regional networks (which usually have the misnomer `MANs').

OUCS undertook investigations, in conjunction with Oxford Brookes and UKERNA, to determine the most effective way for Oxford to be connected to this proposed national backbone. At one time, it appeared possible that Oxford might consider joining the proposed MAN Lense (covering the area bounded by Guildford, Southampton and Brighton). However, this made no real sense (economically or otherwise), since Oxford's traffic is national and international, not regional.

Ultimately, UKERNA agreed to develop a `Thames Valley Network', with a connection to the backbone at Reading, and providing for the two Oxford universities, together with Reading and Rutherford Appleton Laboratory. UKERNA is providing the funds, from its national budget, and is managing the acquisition and installation (and possibly its ongoing operation). Tenders were called, and the contract awarded to Scottish and Southern Electric, which will provide 622Mbps lines interconnecting the 3 sites (thus providing valuable resilience, as well as a useful direct link between Oxford and RAL). These will be installed in January 2001. A valuable feature of the contract is the ability to upgrade at modest cost to 2.5Gbps circuits within 2 years if needed. The links to be provided will be based on DWDM technology, as for Janet itself.

While these developments are exciting and very timely, there are some cautions that must be noted:
  • If Parkinson's Law continues to hold (capacity is consumed at whatever level it is provided), then there may be concerns about the ability of internal networks to cope; the new Backbone (see below) is timely, but may need a `mid-term upgrade' earlier than expected; the biggest bottleneck may prove to be departmental or college networks, some of which are still operating at 10Mbps;
  • The current technology employed in the Oxford Firewall is unlikely to be able to cope if traffic does grow to fill the 622Mbps lines; much of Oxford's defences against hacker attack are dependent on this device; it may be very expensive to upgrade it to technology capable of keeping pace with this level of traffic volume growth;
  • If the proportion of trans-Atlantic traffic remains much the same (about 40-50% of the total incoming traffic), as seems likely, then there could be a dramatic increase in the charges Oxford is levied by JISC (see below); in any case, because of Oxford's relatively well-developed network, it is likely to be able to ramp up its usage faster than most other HEIs — with charges based on relative shares of traffic, this could increase Oxford's charge substantially (the only redeeming factor is that, if charges continue to be based on the previous year's usage, there will be a lag before this increased usage translates into increased charges).

Traffic patterns will need to be carefully monitored once TVN is installed so as to be ready to deal with such problems before they become critical. OUCS has also been investigating the possibility of obtaining a secondary Internet connection from a commercial provider, to which it might endeavour to divert much recreational traffic; at this stage, this does not really seem to be cost-effective, but is being kept under review.

2.3. Charges for Janet Use

This is the second year that charges have been levied by JISC to help pay for Janet. The HE Funding Councils have imposed a ceiling on how much they will pay from top-sliced funds for Janet, at about £20 million. This year, the amount to be raised from charges was £3 million. This sum was divided between all HEIs and other users based on incoming trans-Atlantic traffic volumes, excluding the hours of 1 am to 6 am, and excepting traffic directed through the national Web cache. The period of usage on which the charge was based was August 1998 to April 1999 (this enabled HEIs to budget for their charges, while still providing a degree of responsiveness to institutions' attempts to reduce their load [figures 5 & 6]).
For Oxford, the charge this year came to £143,000 which sum was met by a General Board subvention.
For 2000/2001, the charges will be derived in the same way, except that the period of usage on which charges are based will be May 1999 to April 2000. With the introduction of the compulsory Web cache at Oxford in December 1999 (see below), `countable' traffic dropped dramatically, thus reducing the cost for the year 2000/2001 to £96,500.

It is very difficult to know how these charges may vary in future. The present arrangement has a number of anomalies (eg see above), which are certain to be redressed before long. During the year, JISC undertook a consultation exercise to seek opinion from the HE community; Oxford responded to this, among other things pressing for continued (or greater) top-slicing; it is thought that around 50% of the community also pressed for top-slicing model. However, there was a considerable divergence of opinion on other related matters, with a vocal minority seeking full cost recovery based on actual current usage. Furthermore, it is understood that the Funding Councils are very much against further top-slicing. JISC has as yet given little indication how it will respond to the quest for an improved cost-recovery model.

For 2001/2002, indications are that charges will be formulated in the same way as for 2000/2001. It seems very likely that, subsequently, charges will take into account all incoming traffic, not just trans-Atlantic traffic (originally, this was the fastest-growing component of Janet's costs, but this is no longer the case). It is also not clear whether the present incentive to use the national cache will remain in the longer term. This is something that OUCS will continue to monitor and influence as best it can [figure 7].

2.4. Web Cache

From the beginning of December 1999, use of the University Web cache was made mandatory. The reason for this was to reduce the charges for network traffic imposed on the University by the Janet authorities. These charges (which are based on the proportion of incoming transatlantic traffic attributable to the University) are waived for traffic passing through the national caches, and making use of the University cache enables us to ensure that calls that cannot be satisfied from that cache can be referred to the national cache.
Although many universities have imposed similar constraints by instructing users to configure their browsers to use the cache, and barring all other calls, it was felt that the diversity of methods of access within the University, and the lack in many cases of immediate support, meant that such a solution was not possible.
A solution was therefore sought, and found, that enabled this to be implemented in a transparent manner, by automatically redirecting web traffic through the cache, without the end user needing to make any change. A number if exceptions had to be made, mostly for sites for which access was restricted (for licensing reasons) to specified addresses within the Oxford domain.

The hardware used for this, the Acedirector, routes requests to one of the seven proxy servers which are connected to it, which themselves pass on requests to the national cache. Domains within the UK will be requested directly.

During Hilary Term, about 55 million requests for Web objects were handled by the cache farm each week. During Trinity this increased to over 60 million per week. The amount of data served to Web browsers per week was 350 megabytes in Hilary, and 400 megabytes in Trinity [figures 8, 9 & 10]. Approximately 25% of the data requested by local Web clients is served from the local caches.

2.5. University Backbone Network

The number of networks connected to the Backbone are basically static from the previous year [figure 11]. A slight reduction has arisen in moving to the Gigabit Backbone network, since router to router interfaces used on the old FDDI backbone are not counted.
Backbone traffic continued to rise during the year, justifying the decision and timing of the replacement of the FDDI 100Mbps technology, which has served Oxford so well since 1992, with a faster alternative [figures 12, 13 &14]. Initially, there was considerable doubt as to the most appropriate replacement technology — the only candidates being ATM (155Mbps or 622Mbps) and Gigabit Ethernet (1000Mbps). At first, ATM seemed the best choice, because it was a stable and proven product, has been employed by many other universities, and held out the promise of use for multiple mode traffic (eg video as well as data).
However, further investigations revealed that few manufacturers were planning further development of ATM, many were pouring considerable resources into GE, few universities had been installing ATM lately, and its promise of multiple uses rarely seems to have been fully realised; furthermore, technologies such as Voice-over-IP suggested that a pure IP network may well be just as open to multiple uses.

Gigabit Ethernet does not have the same degree of intrinsic resilience as FDDI, so care was taken to design a configuration that offered similar security. The network was progressively deployed during the year, and has proceeded with minimal difficulties, and almost no disruption to traffic. At year end, there were still about 25% of networks to be transferred. The target for completion, December 2000, seems likely to be met.

As far as the geographic distribution of the backbone is concerned [figure 15], there has only been very modest development this year. In any event, resources have been devoted very substantially to deployment of the Gigabit Ethernet [figure 16].

A case was developed during the year for the development of an alternative fibre route to the Headington hospitals, in order to provide resilience on this Backbone spur. As this will also provide resilience for Oxford Brookes' connection to Janet, it is hoped that UKERNA may assist in its funding. At this stage, the bulk of the funding will be provided by savings on the new Backbone, together with some supplementary funding from the General Board. The route has been surveyed, planning applications lodged, and tenders are expected to be called in late Summer 2000.

2.6. Dial-Up Service

The year-on-year growth in use of the Dial-Up service has slowed down during 1999/2000, as indicated in figures 17 & 18. As a consequence, it has not been necessary to increase the number of lines to the same extent as in previous years [figure 19]. The level of usage is still monitored regularly, so as to detect any untoward queuing, but there appears to be no evidence of this [figure 20].
It seems likely that the slow-down in usage growth has come about through a number of factors, including (a) the community served by this facility is reaching saturation, (b) an increase in use of other ISPs, such as Freeserve, to gain initial Internet connection, and (c) a continuing creep in the Backbone network's reach.
It is also thought that the length of calls may have increased, perhaps explained by various cheaper or free after-hours phone call tariffs. The effect of the new NTLworld service on the NTL side of our Dialup service remains to be seen.

For the longer term, OUCS has no plans to reduce or abandon this service, which a few universities appear to be doing. In such a case, users would have to dial-up another ISP and gain connection that way. Economically, this makes some sense; however, the chief impediment is that access to many electronic resources is governed by the IP address of the user. Dialling-up OUCS ensures it is within Oxford, whereas dialling up an external ISP would result in access being barred. Since Oxford has probably the largest collection of electronic resources in the country, this is a much greater factor for us than most others.

Up: Contents Previous: 1. General Overview Next: 3. Network-based Services