A Busy Week!

If you saw my recent Blog “When things get hot”, you will know the challenges faced by IT staff when server room cooling fails. Well, the air conditioning in that server room failed again on 8th December which was really unexpected because it had received a major overhaul. In the light of this further incident, we are now putting in place permanent 24×7 monitoring of the temperature in that room until we cease using it, and we have mobile cooling units on site that we can use should the need arise – these can keep the room sufficiently chilled in the event of any further failure of the main system. In fact, we will only be using this room for a few more months before moving all the kit into a new state of the art datacentre shared with Aberdeen University and Aberdeen College. The 24×7 monitoring will be in place in time for the Christmas break, and as we do every year, a number of IT Services staff are on call over the holiday.

As if that was not enough, on Tuesday 11th December we had a very unusual technical problem on our storage system, which hosts “home” and “shared” network drives for all staff and students. IT Services staff worked through the day, and through the night until 2am in conjunction with global support engineers from the manufacturer before a chap called Adam in their Australian support centre identified the problem and we got the “home” and “shared” drives back. Big sigh of relief! We will meet with the manufacturer early in January to carry out a review of what happened. Meantime, once again big thanks to a number of IT staff who worked well into the evening and night to sort this.

For staff and students, over these few days they would see a short outage of some services early Saturday morning, and the loss of network drives on Tuesday. Behind the scenes, however, staff from IT Services had a heavy programme of work to keep services running and secure for the whole of that week. With one of the server rooms operating at reduced capacity, they had to move some services to the other server room. Systems like e-mail, the web site, our Moodle Virtual Learning Environment, the Portal and many others kept operating throughout all of this period. A lot of the week was spent in conjunction with our Estates Department arranging for the cooling to be fixed – and I’m pleased to say that the faulty parts have been replaced and cooling is working again. IT Services staff also had to work for several days to re-establish the backup systems which had been significantly affected by the cooling and technical problem. All this is almost finished as I write. Staff and students don’t see that work, but it is essential to ensure that all our services are protected and properly backed up – certainly before the holiday period. Apart from the work to re-establish our backups systems, we have put a freeze on all other changes now until the University re-opens in January.

 

Chopped Pork and Ham

. . otherwise of course known as SPAM. I was wondering today how SPAM came to be coined as a term for junk e-mail. As I suspected, it seems to go back to a Monty Python sketch from the 1970’s – have a look at this article.

I guess that’s lost on the younger elements of our staff and student population but I remember singing the SPAM song. . .

What’s certainly not lost is junk e-mail. We all get the unwelcome e-mails in our inbox – many are offering a whole range of services and products, the nastier ones pretend to be a bank or other authority and are trying to persuade you to part with your PIN number and/or password. All this SPAM is at best a huge nuisance, and at worst a real security threat. What are we doing about this?

Fighting SPAM successfully is a constant battle. People who send SPAM are constantly changing their approach, and as one route gets shut down they find another. A bit like the malaria parasite – as soon as your body defences recognise it and attack it, it simply changes its coat. The trick is to prevent all the undesirable e-mail getting through, but making sure you don’t actually prevent legitimate email.

We use an external company for this – all our incoming mail goes through their service. There’s a good reason for that – an external company managing this for lots of organisations is better placed to spot bulk e-mail messages going to lots of organisations and block it.

It may interest you to know that in a typical month we receive 3.3 million e-mails into our University every month. 2.6 million of these are identified as SPAM and you never see them – that’s nearly 80% of all our e-mail. There are some that get through, but rest assured – they are a tiny fraction of the ones that are blocked.

Finally – just a reminder. If you receive any e-mail, from anyone, asking you for your PIN number, or password, or any other security details, NEVER reply. Just bin it. If it looks like it came from IT Services, it didn’t. If they offer a link to some web site where you can allegedly check or change your security details, DON’T CLICK.

When Things get Hot

 Anybody who works in IT today will know that when important IT Services fail, IT staff feel the heat. When they are working normally you don’t normally get thanked. Let’s face it – when did you last write to the Electricity Company to thank them for such a wonderful job delivering electricity? So, for us this situation to some extent goes with the job, but I’ll use this Blog today to say thanks to a number of RGU IT staff who did a great job over the weekend of 17th November. 

We have two main server rooms from which we deliver our main IT services. Servers in each room are running all the time so that we can use the full capacity for all of our services. If one room fails, however, we can continue to run critical services from the other. 

Now, servers pump out lots of heat and one of the most common causes of failure is around cooling. That’s exactly what happened to one of our rooms in November – one of the cooling units failed, and then the second cooling unit, now under a heavier load, started to wobble. The temperature climbed rapidly and it wasn’t long before some of the servers started to automatically shut down to protect themselves – a nightmare scenario for IT Staff. 

IT Services and Estates staff managed to get the temperature under control, but it was clear that the air conditioning had to be repaired quickly and that meant having to shut down ALL the cooling for several hours. IT Services staff prepared a plan of action, and worked over the weekend to move essential services into the other computer room ahead of the shutdown. When the time came, the cooling was shut down as were most of the servers in the room. We were able to continue running the most essential University IT services throughout the day from the backup room and once the air conditioning was repaired things were back to normal fairly quickly. 

We were able to do this because for several years we have been building resilience into our overall IT Architectures. We have dual communications links, dual server rooms, and we use technologies that allow us to move services from one room to the other, and keep copies of critical data in both computer rooms. 

It all paid off that weekend, and indeed it has paid off on a number of occasions. More than once, we have had some kind of problem with our network links, or servers or server rooms, but you would never have known because we were able to keep essential services running. 

So, my thanks to all the IT Staff who make this possible!

Files on Fire

 

Last week we had our annual Health and Safety internal conference at RGU. We heard first hand from another University which had experienced a major fire, and was willing to share with us their lessons learned. We’d had a similar talk two years ago from another University which had experienced a major fire. You would think these were rare occurrences, but it was suggested that we should search on the Internet for “University fire explosion” – try it yourself and see what comes up.

It’s a great feature of the HE sector that we have such open sharing of experience and lessons learned. Following each of these major fires, the institutions quickly discovered that many staff were still storing the only copy of some electronic files and documents locally, on their laptops or desktops. They weren’t backed up. In some cases, there were also some local servers with important departmental data on them. They weren’t backed up either. 

For each presentation, we were shown images of burnt and/or soaking wet IT equipment being dried out in large dehumidifier arrangements before some IT recovery firm then set about recovering the precious data. In many cases this was important research data. Amazingly, the recovery was able to retrieve a lot of this data, but it took some weeks and the data was inaccessible during that time. For equipment unfortunate enough to be near the seat of a large fire – forget it.

None of this is anything new. You shouldn’t have to have experienced a car crash to know the importance of wearing a seatbelt. So, we shouldn’t have to experience a fire to know the importance of making sure our important electronic information is securely stored and backed up. If you place it on one of the University Shared Network Drives, or any of the main University systems such as Moodle – it’s all safe and secure.

If, however, you have the only copy of something important stored on your desktop computer, and no backup anywhere else, then as you close the door tonight to go home just imagine that’s the last time you see your office in one piece.

How does that feel?

 

 

 

 

Telephone Service

I don’t imagine you spend any more time thinking about the phone on your desk than you do thinking about the light bulb in the ceiling. It’s just there, it works, so why the heck are we changing it?

Up until the 1990’s, all phones were what we call “analogue”. If you have an old looking phone on your desk, chances are it’s an analogue one. With an analogue phone each phone has it’s own wire which goes all the way back to the University’s telephone exchange system. 1,500 phones = 1,500 wires, some going quite a distance.

In the 90’s, it became clear that telephone calls could be carried over computer data networks, which by then were becoming standard in buildings anyway. This is called “Voice over IP”, or “VOIP” for short. At a very basic level, this saves on wiring. For our big new campus buildings, we save hundreds of thousands of pounds by using VOIP instead of analogue, and for that reason alone nobody puts analogue phones into new buildings any more.

But that’s not all – as well as saving money, VOIP allows many more features than are available with analogue phones. I explained some of these in a previous post on Unified Communications.

At RGU, about half our phones are already on VOIP, but with the new Masterplan building opening up next year, it will all be VOIP and we are just now taking the opportunity to move all the remaining analogue phones across to VOIP. This is well underway and those of you with old analogue phones are going to see these replaced over the next couple of months.

If you have an analogue phone, this is what will happen:

1) Before we can do anything else, we have to get the new VOIP phone onto your desk and make sure that it’s working – we don’t want to leave you without a working phone. So, first you will be given a new “VOIP” phone which will sit on your desk along with your analogue phone. At that point, all your incoming calls will come in to your analogue phone, but you will be able to make outgoing calls from either phone. We’ll do that for all analogue phone users before moving to the next stage.

2) Then, phone by phone, we have to move your extension number from the old analogue phone to the new VOIP phone. We’ll tell you when that’s happening. After that, you use your VOIP phone for everything.

3) Then, we’ll take away your old analogue phone.

4) Once we have all staff safely across onto VOIP phones, then we will start to look at making other features available.

For those of you moving into the new Masterplan building down at Garthdee, you’ll get your new phone well ahead of that in your existing office. Then when the time comes to move, it’s just a question of unplugging the phone and plugging it in at your new desk down at Garthdee (someone will do that for you).

You can see more details of the project here:

My Slender Friend

Often, when I discuss with a friend or colleague “thin client” I get (understandably) a completely blank look. Most desktop computers in use at RGU today are what we in the trade call “fat client”. That means that the box that sits on your desk (or under it) is a fully fledged personal computer, usually running Microsoft Windows or Apple OSX, with software installed on it to meet your needs – just like the one you probably have at home. So you’ve learned something – your home computer is a “fat client”.

And if your home computer is anything like the norm, it’s a bit of work to keep it up to date. There are always new versions of software to load, updates being downloaded, things go wrong, and it has to be backed up so that you don’t lose any of your files and settings (you do backup your home computer . . . don’t you?). When you get a new computer, there is all the pain of transferring across all your software and files and of course always some software that stubbornly refuses to work on the new computer.

If that’s a battle at home, try managing 3000 of these things in an environment like the University. They also consume a lot of energy and output a lot of heat – particularly where you have lots of PC’s in an IT Lab.

So, if that’s “fat client”, what’s “thin?”. With “thin client”, all your software, settings and data are run on central servers. At RGU, you access these through our “MyApps” service which I have mentioned in a previous post. The box that sits on your desk has just about nothing in it – it just has to connect you to MyApps. That’s why it is called “thin”. That means it’s dead simple, uses little energy and it’s a piece of cake to install it. If it goes wrong – just get another one.

We are shortly going to start installing these across the University to replace traditional PC’s. This will substantially reduce our energy bills and simplify our IT support. What does a “thin client” look like? It looks just like a “fat client” except it’s thinner and totally silent. You still have a keyboard, you still have a mouse, you still have a monitor, and the software you use is still the same.

I’ve just received one, and I will use it from now on as my main University computer. I am writing this blog entry on it – it’s just the same as using Microsoft Word on any computer.

We won’t be putting thin client everywhere – some specialist software applications run better with a “fat client” computer so we will keep these where they are required. Wherever possible, however, we will over the next few years be buying “thin client” computers instead.

Andy McCreath

Cyber Security

The whole subject of cyber security is growing in prominence across the UK – indeed across the world. It is recognised as a significant challenge to organisations and national economies, and with the key role that Universities play in research and education they are as vulnerable as any other sector. Universities own areas of intellectual property of immense value and theft of that intellectual property would be an issue not just for the Universities concerned, but for the wider economy as well.

The UK Government’s National Security Strategy lists “hostile attacks upon UK Cyber Space” as amongst the top 4 priority risk areas next to terrorism, war and major accidents/natural hazards.

In November 2011, the Cabinet Office published a “UK Cyber Security Strategy

This strategy sets out how the UK Government will tackle the threats, but also in a way that ensures that “. .  cyberspace remains an open space – open to innovation and the free flow of ideas, information and expression.”

That’s important for Universities. We are very open communities, with large student populations, members of the public, and staff involved in a wide range of activities. Our IT infrastructure needs to be open enough to permit all of that, but secure enough to protect important information resources, personal and confidential information.

Like many organisations, we will be keeping our security measures under constant review, particularly in the light of growing cyber security threats. We want to make it easy for staff and students to access our systems but there will always be the need for some security and it is important that all of our users respect this and follow guidelines and instructions where they are provided. You may feel that you are not involved in anything that is secret or confidential. If you are connected to our network, however, you are just as likely to be a target and if you don’t follow security guidelines your equipment could provide an easy entry point for an attacker.