Copyright Chris Johnson, 2002.
Here is a printable version.
Safety-Critical Systems Development
Open Assessment 2002-2003
A. Introduction
Risk homeostasis is a controversial theory.
It suggests that individuals have an implicit, preferred or target level
of risk.
One consequence of this theorem is that any safety improvements that
reduce the risk exposure of an indiviudual will potentially lower the
perceived threat below the target level.
This creates the opportunity for people to alter their behaviour.
In particular, they may trade performance objectives for a slightly increased
level of risk.
For example, car drivers that have access to advanced braking systems
and other protection mechanisms may drive faster and brake later than
drivers whose cars do not offer this higher degree of protection.
Many people disagree with the basic ideas behind risk homeostasis
theory.
Does anyone really have an implicit target level of risk?
Is anyone calculating enough to identify performance benefits that can
be precisely traded off against safety improvements?
Your task is to select an appropriate analytical technique to gather
evidence about this theory.
Gerald Wilde provides an on-line introduction to Risk Homeostasis.
Be aware, however, that there are different perspectives on this issue
and that you should also read more widely about the subject.
B. Your Task
Your task is to find evidence that will support or weaken the arguments
made about risk homeostasis.
You can use any method that you think is justified to support your
arguments.
The only caveats are that the evidence MUST be original and that at no
point should you endanger anyone who may be connected with the project.
The following paragraphs identify some of the techniques that can be
used to provide evidence for and against risk homeostasis.
C. Methodologies for Examining Risk Homeostasis
There are many different ways in which you can provide or disprove
assertions about risk homeostasis theory.
These are summarised in the following paragraphs:
- Experimental techniques.
A number of objections can be made about the use of experimental
techniques to examine issues in risk and decision making.
In particular, expressed preferences in the laboratory may provide
poor predictions of individual behaviour in the `real world'.
However, many researchers have simulated situations in which people make
`risky' decisions under laboratory conditions.
These laboratory experiments are used to provide insights
into elements of risk homeostasis theory.
In order for these experiments to be valid you must consider the
experimental design, including issues such as dentifying dependent and
independent variables, counterbalancing etc.
You must also choose an appropriate statistical test with which to
analyse any results that you might obtain.
Some of these issues are mentioned in Chapter 9 of Wilde's book but you
will also have to read more widely into experimental design.
For example, there is an introduction to experimental design at
William
Trochim's site at Cornell and another by Paul Cohen at Colorado
State
In particular, you should note that Chapter 9 of Wilde's book covers the ways in which primitive (and more advanced)
forms of computer games are commonly used to examine risk homeostasis
theory.
One way in which you could actively extend research in this area would
be to write a computer game to test risk homeostasis theory under
laboratory conditions.
Do NOT simply copy the games in the Wilde book.
- Epidemiological techniques.
One way of finding evidence for and against risk homeostasis is to
gather data about incidents and accidents before and after some safety
innovation is introduced.
For example, if cycle helmests are made compulsory then we can look to
see what impact that has on injuries to cyclists who wear these hats.
Problems arise in trying to interpret the statistics.
If injuries rise does this support the homeostasis theory and show that
cyclists misjudge their target level of risk,
In other words, they do more dangerous things beyond the benefits
provided by a helmet.
Alternatively, we might see that they types of injuries changes.
The helmets have the effect of increasing the frequency of incidents
because cyclists ride more recklessly but that the outcomes are less
severe because the helmets intervene to prevent head injuries.
In any event, if you choose the epidiemiological approach you must find
a NEW data soure
General pointers to this data are available via OSHA, the HSE and other regulatory organisations via the links page.
Gerald Wilde's book has several examples of this approach being applied
to safety data, especially Chapter 5.
- Other approaches.
This assessment is deliberately open-ended.
It is intended to provide scope for individual initiative and research.
I would strongly encourage any other ideas about techniques that might be used in this
open assessment.
However, I would also urge you to email me or discuss them after a
lecture to ensure that they will provide credible evidence about risk
homeostasis.
D. Transferable Skills
It is important to stress that this assessment will provide anumber of
generic skills.
Although we are focussing on evidence that might support or weaken
arguments about risk homeostasis in general, the methods that you are
using are identical to those that safety-critical organisations might
use in order to assess the risks that are associated with particular
products or designs.
Statistical surveys or epidemiological studies can be used to identify
benchmark figures for the performance of existing safety-critical
systems.
The existing frequency of injuries involving particular types of devices
provides a target for development, any new systems must at least be as
safe as previous applications.
Experimental studies of risk taking behaviour provide means of assessing
the actual performance of system designs prior to their full operational
use.
The same empirical methods that you might employ in this project can also be used
on a commercial setting.
The same criticisms can be raised about these methods as well.
Epidemiological or experimental work can often be compromised by
problems of under-reporting and gathering raw information about previous
incidents involving particular devices.
Similarly, operator behaviour in lab-based studies may have little
relationship to their real-world performance.
E. Assessment Criteria and Submission Details
This exercise is degree assessed.
It contributes 30% to the total marks associated with this course.
The body of the report should not exceed fifteen A4 pages.
The report must be printed out and must be submitted in a secure binder (i.e., one that will keep the pages together and in the correct order).
It must include:
- A title page containing your student as well as your contact details (email address etc);
- A table of contents and appropriate page numbers;
- A section on the methodology that you used.
This should begin with a statement of the hypothesis that you chose to
prove or disprove.
Please note that your project might look only at a specific aspect of risk
homeostasis, such as the existinace of implicit risk targets rather than
behaviour changes.
This should include some consideration of alternative approaches and a considered justification of the reasons why you chose the method that you did.
- A results sections.
This should describe the findings that you obtained.
It should also discuss any problems that arose during the study that
might make it difficult to interpret your findings.
- Conclusions.
You must provide a clear statement about whether or not your study
supports the ideas behind risk homeostasis.
If your study indicates a mixed set of conclusions then you should state
which aspects of the theory you are willing to support.
You should also identify the limitations of the theory as it currently
exists,
In addition to the fifteen pages associated with the body of the report, you may also include appendices.
These should contain:
- the listing of any code used during the study together with suitable acknowledgements for the source of code that has been borrowed from other programmers;
- source data for any statistical study.
The intention is that another analysts should be able to recreate your
results in order to validate your findings.
If the source data comes from the web or is available in another
electronic form it is acceptible to simply provide a reliable URL or a
disk.
It should be handed in at the start of the lecture on Wednesday 11th December 2002.
Extensions will only be granted in exceptions circumstances and they should be requested prior to the deadline.
Extensions for medical reasons should be reported as soon as possible and should be supported by forms from a medical practitioner.
Extensions for equipment failures may be granted provided that you let me know as soon as they occur; so that I can make sure they get fixed as soon as possible.
Please make sure that you keep back-up copies of all of your work towards this exercise.
The following marking scheme will be applied:
- 15 for the method;
- 10 for the results;
- 15 for the conclusion;
- 10 for the technical documentation.
All solutions must be the work of the individual submitting the exercise.
If any code or design ideas are borrowed from course notes, books or other students then those sources MUST be clearly acknowledged.
All questions about this exercise should be addressed to Chris Johnson.
Back to the main index...