** About Us > Briefing Papers > At the Parting of the Ways

IT security gets added prominence when staff move to pastures new,
especially if it is a system administrator that is leaving.

By Stephen Bishop

While most departures are amicable and take place in an orderly
manner, we know of circumstances where key members of staff have
left under a cloud and with little or no warning.

In some cases, this has lead to a significant damage limitation
exercise - in effect an ad hoc security audit carried out in short
order - and pushing aside all other work.  But, as in many other
areas, preparation is everything and there are practical steps
that companies can take to avoid this kind of trouble.

* The Problem

In many organisations, there is one individual who reigns supreme
over the network and key systems. He or she is not only formally
responsible for the smooth running of the service, but also
does most of the actual work, from dealing with suppliers to
cabling racks.

We all know the type, and they're the people that we turn to
when things go wrong or tricky technical issues need resolving,
but that's the problem: no-one else really knows the extent of
their knowledge. We trust them implicitly but don't really know
with what it is that we trust them.

People like this often seem to be part of the fixtures and
fittings, but most employees move on eventually and the break
can be very sudden, perhaps for personal reasons rather than the
simple need to earn more money, and it can be wrapped up in bad
feelings on both sides. Put simply, the gamekeeper can suddenly
become a potential poacher, and there may be no other assistant
gamekeepers able to step into the role.

The obvious worry is that the person on the way out knows a large
number of administrative username-password combinations and other
credentials, covering applications, servers, networking equipment
and physical access within the building. But does anyone else
know the complete set of critical passwords, or at least has
access to a sealed envelope containing these items for use in an
emergency? There are many instances where this process has not
been put into place. Worse still, are there undocumented ways
into the network from the outside, for example, remote access to
firewall management or SSH servers running in unexpected places?

This is a real issue. We have seen the effect that sudden
departures can have on system security, having been called in
by clients to give practical assistance and assess the overall
impact on the organisation. In one case, we received a request
for immediate help for a small company that was about to dismiss
the network manager: fortunately we were able to send along a
consultant the very next day, to join the emergency audit team,
roll up his sleeves and get stuck into remedial work. Looked at
from a philosophical point of view this can be an interesting
opportunity to do more-or-less the whole range of IT security
tasks in the space of a single day, and demonstrate one's
ability to think on one's feet, but it isn't easy on the nerves.
In particular, it's easy to see spooks, with every connection
looking like an all-out attack.

* Policy and Procedure

Organisations cope best when they have carried out some
preparation, and an orderly response is always better than applying
panic measures.

Perhaps a good way of starting is to sit at your desk, close
your eyes and imagine your system manager being run over by a bus
(later on, it may be a nice idea to imagine the person making a
full recovery in hospital). Then think of all the things that could
go wrong but with no one available to fix them. This should help
focus the mind and provide an incentive for doing some preparation.

Here is a list that should be considered right now, although this
is not exhaustive and may need considerable expansion:

- Avoid single control: don't have key systems in the hands of a
single administrator - and role rotation is a good idea, as it
forces all involved to work in a more orderly manner

- Maintain documentation: keep track of network structure, systems,
users, events and responsibilities (and carry out regular reviews
of this information)

- Build procedures: try to wrap all significant changes to
equipment and personnel in documented procedures, and keep them
up-to-date as technology moves on and the organisation changes

- Avoid manual maintenance: reducing the organisation's dependence
on manual procedures has its own benefit, as well as easing
administrative handovers

- Keep logs: retain access records, making sure that these can
be viewed quickly and easily if there is a security alert

Note that we have not mentioned the creation and promotion of
an Information Security Policy in this list, as this should be
a given. In fact, all the suggestions itemised here should have
their equivalents in a properly managed security policy.

Similarly, we recognise that quality management along the lines
of ISO 9001 provides a good framework into which our practical
recommendations can be placed.  We've also left out specific
mechanisms such as file integrity monitoring. These are of general
value, but to be of any real benefit in the current context they
would have to be run separately from normal system management,
which is fine in theory but difficult to achieve in practice,
particularly in smaller organisations.

* Practical Steps

Of course, the crunch comes when a network manager or senior
system administrator departs. The worst possible case is that
of a formerly trusted system administrator being caught in some
misdemeanour and marched off the premises, meaning that the
gnawing doubts start immediately.

The primary fear is that some form of access is still available
to the individual concerned, either by normal channels that have
not been closed down or by undocumented, back-door connection
points and alternative credentials. After all, many employees
these days need some form of remote access in order to do their
job, and in the case of a network manager or system administrator
this may be quite a complex, many-headed arrangement.

Similarly, there is a worry that key systems have been compromised
in a way that may leave them open to a simple attack in the
future. This may not even be deliberate, for the person leaving
may simply know about unfixed vulnerabilities that have not
been documented and of which erstwhile colleagues are blissfully
unaware.

But this is no time to sit and worry.  There will be a number of
actions that have to be carried out in a fairly short space of
time, possibly by new or inexperienced staff.

Ironically, the first thing to do is to have a meeting: a short
gathering of all the relevant staff with the aim of assigning
responsibilities and making sure that resources are used in the
best possible way. Of course, if external help is needed, it is
best to ask for it as soon as possible.

One obvious division of labour is to have some staff focus on
detection, in other words looking to see if anything anomalous
is happening, while others work on prevention, taking concrete
steps to close any holes as soon as possible.

These are some of the more immediate actions that need to be
tackled:

- Remove access: disable relevant accounts, change all
administrative passwords and collect any keys or other physical
tokens

- Scan perimeter: check the network's visibility from the Internet,
making sure that all services offered are there for a good reason

- Carry out a network inventory: make sure that there are no
unexpected systems on key networks (this may involve physical
inspection to find things like unexpected modems)

- Quarantine relevant PCs: system administrators' own desktop PCs
should not simply be passed on to others, but kept in isolation
until any security issues are resolved

- Maintain user awareness: let all staff know that there is a
heightened state of security concern, with an increased need to
report any unusual events, but without going into unnecessary
detail or spreading alarm

- Debrief leavers: assuming that the departure is reasonably
amicable, a final review may be of value to the individual's
successors

Beyond this are the personnel issues such as the signing of
non-disclosure and similar documents, subject, of course, to the
individual's contract of employment.

* In Perspective

Having been involved at the sharp end, we know that this is
not an insuperable problem and, with some effort, most of the
threats can be countered. Being realistic, we know that the
doubts never disappear completely, although they usually blend
into a background level of concern that is the burden of all of
us involved in security and network management.

If nothing else, we hope that this article has acted as a reminder
of a niggling issue that all companies have to face at some time
or other.

* About Us

IDsec is an independent company specialising in network security,
and has provided penetration tests and intrusion detection
systems since 1997. We can assess the security of your enterprise
and advise on long-term protection: as we have for a range of
blue-chip clients in the banking, telecoms, manufacturing and
utility sectors.

IDsec Limited 31-33 College Road, Harrow, Middlesex HA1 1EJ,
United Kingdom

T: +44 20 8861 2001 F: +44 20 8861 3433 W: www.idsec.co.uk

All prices exclude VAT and are subject to confirmation.

Copyright (C) 2012 IDsec Limited

about/briefings/sysadmin-leaving.txt 20120510     (5.11)