Department of Computing Science, University of Glasgow, Glasgow, G12 8QQ,

Tel: +44 141 330 6053, Fax: +44 141 330 4913

1. Introduction

Accident reports document the causes of previous failures and guide the operation of future systems. However, a number of factors complicate the generation of these documents. The increasing integration of heterogeneous production processes makes it difficult to communicate the causal factors that lead to major failures. The increasing requirement to consider managerial and organisational factors is extending the scope of accident reports. Other problems stem from a cynicism that many engineers express about the veracity of accident reports (Snowdon and Johnson, 1999). Greater faith seems to be placed in the anecdotal accounts that are passed between designers and managers than in the official accounts published by investigation agencies. One reason for this is that formal reports focus on a single view of an accident. In contrast, anecdotal accounts often focus on multiple alternative explanations that may, or may not, be grounded in the available evidence. This paper, therefore, presents a simple graphical technique that can be used to represent and reason about the lines of reasoning that support particular conclusions. The intention is not to replace existing textual documents. In contrast, Conclusion, Analysis, Evidence (CAE) diagrams provide an overview for the arguments that are presented in accident reports.

A report prepared by the Marine Incident Investigation Unit (MIIU) is used to illustrate the argument in this paper. This document describes the events leading to a fire aboard the Leonardo da Vinci, a dredger based in the Netherlands. The report also provides an overview of the crews’ response to the fire and the subsequent involvement of local shore-based fire services and of the tugs that came to support the vessel. The MIIU report summarises the incident as follows:

This case study was chosen for a number of reasons. It typifies the relatively large number of incidents that do not lead to a fatality but which do have the potential to develop into major accidents (Reason, 1998). It stems from a combination of technical, managerial and operator failures that also typifies many incidents (Johnson, 1999).

A further reason for choosing this case study is that the MIIU report is covered by the Australian Navigation (Marine Casualty) Regulations. Under sub-regulation 16(3), if a report, or part of a report, relates to a person’s affairs to a material extent, the Inspector must, if it is reasonable to do so, give that person a copy of the report or the relevant part of the report. Sub-regulation 16(4) provides that such a person may provide written comments or information relating to the report. These comments are then, typically, included in the closing sections of the report. The MIIU argue that the introduction of these alternate viewpoints provides greater confidence in their findings.

Conclusion, analysis and evidence diagrams were specifically developed to provide a graphical overview of the argument that is presented in accident reports. They stem from the observation that many root-cause analysis techniques advocate the separation of evidence and conclusions (US DOE, 1992). This has become embodied in the format of most accident reports. The events that occurred during an accident are, typically, described in the opening chapters while the conclusions are presented at the end. This is useful because readers can form their own interpretation of events before reading the investigators’ analysis. However, such an approach can create problems when people are forced to recall important evidence that is presented many pages before the concluding chapters. Problems can also arise if readers are never explicitly told which items of evidence support a particular conclusion.

The following procedure is used during the generation of CAE diagrams:

1. List all of the conclusions that are identified in the report. This is usually a trivial exercise because most accident reports list their conclusions in the final chapter. Next to each conclusion, indicate the page numbers where they are proposed.
2. List each line of analysis that supports or weakens the conclusions identified in stage 1. This is, typically, a non-trivial exercise. Few accident reports explicitly record the lines of analysis that support their findings. The reader is expected to piece together the implicit inferences that support specific conclusions. Again, a note is made of every page number on which the line of analysis is presented.
3. List the evidence that either weakens or strengthens each line of analysis. This is relatively straightforward because stage 2 helps to identify the evidence that is needed to support a particular line of argument. Page numbers are recorded for each item of evidence. The warrant or source for the evidence (e.g., expert witness, simulation, regulation) is also noted.
4. Construct a graph based on the products of the first three stages. A separate CAE diagram is produced for each of the conclusions identified in stage 1. These are drawn as the roots for each of the graphs. The arguments for and against a conclusion are connected to the root. Solid lines connect arguments that support a conclusion. Arguments that weaken a conclusion are connected by dotted lines. Finally, each item of evidence is linked to the relevant arguments. As before, evidence in support of an argument is shown by a solid line whilst negative or contradictory evidence is shown by a dotted line. Figure 1 illustrates the results of this process for the MIIU case study.

In previous papers, we have shown how CAE diagrams can be used to reason about the internal consistency of arguments presented in accident reports (Johnson, 1999a). This paper uses CAE diagrams to chart the differences that exist between alternative accounts of the same accident. This helps to identify areas of conflict between the "official" view of an accident and eyewitness testimony. As we shall see, however, the technique is more general. It could also be applied to examine differences between, for example, the version of events advocated by a regulator and that proposed by an operating company.

This section shows how CAE diagrams provide an overview of the argument in the MIIU report. The case study document contains two sets of findings and so, as explained previously, this leads to two related sets of CAE diagrams. The first relates to the causes of the injury to the Technician. The second focuses on the causes of the fire in the first place.

Figure 1: CAE Diagram for the Causes of the Technician’s Burns

The MIIU report identifies the following causes for the burns that were suffered by the technician during the fire aboard the Leonardo da Vinci:

The next stage in the composition of a CAE diagram is to identify the evidence that supports each of these lines of analysis. The following paragraph reproduces sections of the MIIU report that directly address the causes of injury to the technician. As can be seen from the page numbers associated with each of these citations, the evidence and analysis of these causal factors is widely dispersed throughout the accident report. Each quotation is also separated by a wider discussion of the contextual factors that set the scene for the incident:

Figure 1 shows how these quotations can be used to construct a CAE diagram followiung the procedure outlines in Section 1.2. The previous citations also illustrate a common problem with many accident reports. They fail to accurately distinguish between inferences on the part of the investigator and those facts for which there is substantiated evidence. This distinction, typically, forms a focus for subsequent litigation. These quotations also illustrate the way in which many accident reports fail to provide any warrant for the evidence that supports their arguments. There is a section on the sources of information for the MIIU report. These are listed as follows:

The MIIU report provides no information about the forms of analysis that were used to reach findings such as "the exits and emergency exits in the engine room… were not clearly marked". Later sections will show how this led the Chief Engineer aboard the Leonardo da Vinci to object to sections in the investigators’ account. Similarly, it is difficult to assess the reliability of assertions such as "The Technician… followed the only route out that he knew" without knowing more about the procedures that were used to validate this assertion. Readers must trust to the skills and expertise of the investigator. In this incident, the enquiry was relatively straightforward. Greater doubt can be created when, for example, human factors experts diagnose "high workload", "poor crew resource management" and "loss of situation awareness" without little or no supporting evidence (Johnson, 1999a).

Previous paragraphs have focussed on the loss of contact between the electrician and the technician as a primary causal factor in the injury. This argument takes attention away from an alternative line of analysis that centres on the loss of lighting. Little consideration is given to the technical reasons for the delay in cycling the emergency generators when the fire alarm was first raised:

Figure 2 extends the previous CAE diagram to consider the role that an emergency lighting system might have played in the course of the accident. A double box is used to denote any elements of the diagram that are not direct citations for the MIIU report. As can be seen, the analysis of the lighting is marked as "supposition" because it is not directly addressed in the original document. Part of the justification for this omission may stem from the fact that smoke also hampered the technician’s escape. An emergency lighting system may not then have helped under these circumstances. This line of reasoning is also shown in Figure 2. Note that there is a dotted line between the analysis that lighting would have helped and the analysis about the smoke. As previously mentioned, this dotted line indicates that the smoke weakens any argument about the lighting system. It is important to emphasise that our primary concern is not with the particular analysis shown in Figure 2. Although we have done little more than construct arguments from the evidence that is already present in the body of the MIIU report. In contrast, we are concerned to represent the legitimate concerns or questions that arise when people read an accident report. If these concerns are to be addressed then it follows that accident investigators must explain both what did and what did NOT cause an accident (Snowdon and Johnson, 1999). Followed to its logical conclusion, this analysis might lead to accident reports that are many times their current length and which are correspondingly more expensive to produce. CAE diagrams are, therefore, intended to provide a concise graphical overview of these alternative hypotheses.

Figure 2 employs a simple reference scheme to indicate arguments and evidence that have been presented in previous CAE diagrams. A2 refers to the argument that the technician had not received any formal training on emergency procedures (see Figure 1). E3 refers to evidence that the emergency exits were not clearly marked. This is important because, as we shall see, the same item of evidence may be used to support several different arguments. Such numbering schemes also help to detect inconsistencies where, for example, a single piece of evidence may simultaneously be used to strengthen and weaken a more general argument. This numbering scheme also reduces the problems of scale that arise when mapping out the complex arguments in many accident reports. The closing sections of this paper go on to describe how tool support is also being recruited to address these problems.

Figure 2: CAE Diagram Showing Analysis of the Emergency Lighting System

The injury to the technician would not, of course, have occurred if the fire had not broken out in the first place. The following list summarises the MIIU’s analysis of the causal factors that contributed to this hazard:

As before, a careful reading of the body of the report helps to identify the evidence that supports these conclusions. It is then possible to map out the arguments used by the investigators. The CAE diagram shown in Figure 3 illustrates these different lines of reasoning. This diagram also illustrates the way in which a range of media can be used to support particular lines of argument. In this case, photographic evidence is used to show that the spindle bonnet of the number one shut-off cock had become detatched. Elsewhere we have shown how video material and even desktop virtual reality simulations can be used to support the on-line presentation of future accident reports (Johnson, 1999b). In contrast, this paper focuses on the arguments that are put forward by these documents rather than the media that are used to present them.

Figure 3 also illustrates how the MIIU report fails to distinguish between evidence and analysis. The first citation presents a causal analysis within the main body of the report. The second citation shows how this version of events is simply repeated within the findings of the report. This goes against the separation of evidence and argument that is a common feature of many analytical techniques (US DOE 1992):

Figure 3 again illustrates how the report provides little or no detail about the procedures and mechanisms that were used during the investigation. The reader is not informed about the interview techniques that were used to verify the period during which the handles for the numbers 1 and 2 diesel filters were missing. This lack of explicit warrant and the blurring of analysis and evidence can lead to the generation of further hypotheses that may or may not be supported by the findings of the investigation (Snowdon and Johnson, 1999). For instance, the MIIU report also identified further concerns that were not considered to be primary causes of the accident:

Figure 3: CAE Diagram for the Causes of the Fire

The CAE diagram in figure 3 can be extended to represent the lines of argument that are presented about both the O rings and the general maintenance procedures aboard the Leonardo da Vinci prior to the fire, see Figure 4 A10. The findings about lax maintenance procedures are also supported by the investigator’s argument that all fuel oil leeks should be dealt with as soon as practicable, A11. This analysis is linked explicitly in Figure 4 to the evidence about the Chief Engineers’ workbook, E10. This is important because the previous citations only include an implicit link between the investigator’s comments about good maintenance procedures and the delay in resolving the leaks that are noted in the Chief Engineers’ workbook. The juxtaposition of this apparent failure against the recommendation for good practice is clearly intended to make the reader infer that the Chief Engineer did not deal with the oil leak "as soon as practicable". Figure 4 makes this inference explicit by linking the implied argument to the evidence that supports it.

The evidence that is cited from the Chief Engineer’s workbook not only supports the argument about a failure to deal promptly deal with all oil leaks. This evidence is also linked to an argument about problems with the fuel filter change-over lever. This illustrates how the same observations may be used to support more than one argument. Such dependencies increase the rhetorical value of that evidence to the overall conclusions in an accident report. If it can be contradicted then two arguments may fail. CAE diagrams, therefore, indicate lines of attack that may subsequently undermine large areas of an accident report.

Figure 4 also illustrates how CAE diagrams denote evidence that weakens particular arguments that are made by the investigators. This is denoted by a dotted line between evidence and an argument. In this case, it can be argued that the implication of lax maintenance procedures, A11, is not supported by the observation that the greasers had only recently finished wiping down the engines, E12. Such intervention prevented the fire from spreading. It is important not to obscure or under-emphasise the importance of such observations because it is critical to identify those defences that worked to protect systems and their operators.

Figure 4: Representing Secondary Findings and Contradictory Evidence in an Accident Report

This section has quoted the investigators’ secondary findings in full because they widen the scope of CAE diagrams that might otherwise provide a partial view of the arguments in the MIIU report. A second justification is that these secondary findings often form the focus for litigation and dispute in the aftermath of an accident. Primary findings are, perhaps paradoxically, less prone to such rebuttal. The secondary argument shown in Figure 4 are of particular interest here because they formed the focus of debate when the Chief Engineer of the Leonardo da Vinci responded to the MIIU’s account under sub-regulation 16(4) of the Navigation (Marine Casualty) Regulations. The following sections use CAE diagrams to analyse this response to the investigators’ arguments.

As mentioned above, the Chief Engineer made a written response to the arguments that are presented in the MIIU report. The inclusion of such commentaries into the final document is an important strength of the Australian Maritime safety system. This helps to increase the reader’s confidence in the findings of such documents. The Chief Engineer’s comments were as follows:

Both the investigator’s analysis and the Chief Engineer’s response are critical for our understanding of this incident. In particular, they provide valuable insights into the reasons why the preconditions, or initial causes, for the incident were not detected before the fire started. The official report suggests that the causes of the fire, i.e. the loose Number 1 shut-off cock spindle bonnet, might have been discovered if the crew had replaced the ‘O’ rings that were known to need replacing. However, this implication is contradicted in the Chief Engineer’s statement. He argues that it would not have led to the detection of this primary cause of the accident because the ‘O’ rings are part of a different assembly.

Figure 5 illustrates how this dispute over the significance of the oil leak from the spindle of the Number 1 engine fuel filter can be represented within a CAE diagram. The graphical notation provides an overview of arguments that rebut the official findings of an accident investigation. These opposing arguments, A12 and A13, are labelled by both their page number within the MIIU report and their source, in this case the Chief Engineer. As before, we have used reference numbers to indicate arguments and evidence that are presented in previous diagrams.

Figure 5: Using CAE Diagrams to Represent Alternative Views of an Accident

Figure 5 only captures a small part of the Chief Engineer’s comments on the MIIU report. His testimony continues:

The Chief Engineer argues that the presence of replacement parts for the shut-off cock need not imply that lax maintenance was a central cause of the fire. Figure 6 shows how this objection, A14, relates to the analysis, A9, in the official report.

Figure 6: Chief Engineer’s Objection to Evidence of Lax Maintenance

Figures 5 and 6 raise a number of wider issues. In figure 5, the objection is denoted as two arguments, A12 and A13, that attack the evidence, E10, about records in the Chief Engineer’s workbook. However, in Figure 6, the Chief Engineer’s criticism, A14, is aimed at the analysis or interpretation of the spare shut-off assemblies. This clearly reflects a subjective judgement on the part of the analyst. There is no predefined algorithm that can be used to categorise the natural language comments of eyewitnesses into objections against particular lines of analysis or evidence. There are a number of heuristics or rules of thumb that can be applied. Such objections should be linked to items of evidence if they attack the validity of that evidence, as shown in Figure 5. Otherwise, objections should be linked to the investigator’s analysis, as in Figure 6. However, experience in drafting CAE diagrams has shown that things are seldom so clear cut. In Figure 5, argument A13 does not directly contradict the evidence about the workbook entries. It does, however, attack the interpretation of that evidence as well as the subsequent implication that is based upon this interpretation. The Chief Engineer accepts that the workbook entry about the ‘O’ rings does exist. However, he denies that this evidence has any baring on the possible detection of the spindle bonnet problem. Therefore, the objection could have been linked to either the evidence of the ‘O’ rings problem, E10, or the subsequent analysis, A11 and A10. It is important to stress that this detailed level of analysis indicates an important benefit of CAE diagrams. They provide a graphical overview that can be used to represent and reason about the full implications of the objections and criticisms that are often raised during the drafting of an accident report.

The Chief Engineer also addressed the personal injury to the technician. In particular, he criticised the MIIU’s conclusions that the technician did not receive adequate training during the short period that they were working on the vessel and that poor signing should have been identified as a problem during routine inspections on the Leondardo da Vinci:

As before, these objections can be integrated into CAE diagrams of the arguments in the official report. Figure 7 splits the previous paragraph into two lines of argument. The analysis labelled A15 focuses on the lack of a formal induction course. A16 focuses on the failure of previous AMSA inspections to identify the poor signage. In the former case, this is interpreted as an objection to the evidence that the technician had not been put through a safety induction programme on the Leonardo da Vinci. In the later case, this is shown as an objection to the analysis that the exits and emergency exits were not clearly signed. The official report argues that this is the case on the basis of the technician’s failure to find an exit that avoided the seat of the fire. However, the Chief Engineer argues that independent inspectors passed the signs.

Figure 7: Chief Engineer’s Objection to the Official Causes of the Technician’s Injuries

Previous sections have shown how CAE diagrams represent the conclusions, analysis and evidence that are presented in accident reports. They provide an overview of the complex arguments that are constructed in many of these documents. The same diagrams can also be used to reason about potential weaknesses in an argument, for example, where several lines of analysis depend on the same item of evidence. We have also shown how these diagrams can be extended to represent the opposing arguments that are often made by eyewitnesses in the aftermath of an incident. They provide a graphical representation of the multiple viewpoints that can help a reader’s understanding of an incident or accident (Johnson 1999, Reason 1998). However, all of this is of limited benefit if analysts cannot apply CAE diagrams to support the subsequent design and operation of safety-critical systems.

This section explains how techniques from design rationale (Buckingham Shum, 1995) can be used in conjunction with CAE diagrams to provide a link between the analytical techniques of accident investigations and constructive design.

Design rationale notations provide a graphical overview of the arguments that support particular development decisions (Moran and Carrol, 1995). For instance, the Questions, Options and Criteria (QOC) notation was originally developed to support a range of engineering decision-making activities within the Rank Xerox Corporation. Its syntax is almost identical to that used in CAE diagrams; CAE diagrams were originally developed from the QOC notation. The first step in producing a QOC diagram is to identify the key development questions. These are then linked to the options that answer those questions. Finally the criteria that support or weaken options are added to the diagram. As in CAE diagrams, a broken lines is used to denote negative criteria. Figure 8 illustrates how the QOC (Questions Options and Criteria) notation can represent the design options that were considered to improve situation awareness amongst crews on the Heath Reef in Australia (Johnson, 1999a). The first option is to force all ships to notify their position to an existing monitoring system. This is supported by the criteria that it would provide an external means of ensuring that crews comply with regulations. The Reefrep system could monitor and log the reporting behaviour of each vessel. The development of such a system is not supported by the affect that it would have upon crew workload. The second design option is to use crew training procedures as a means of ensuring adequate levels of situation awareness. This is not supported by the possibility of performing external checks.

Figure 8: QOC diagram showing design options for improved situation awareness.

Ideally, we would like to produce a similar diagram for the various recommendations that were presented in the MIIU report. This is usually a trivial task; most accident reports list their findings for the future development and operation of similar systems. However, the MIIU case study is atypical. It does not explicitly identify any recommendations that might be derived from the analysis of this incident. Instead, the reader is forced to infer the lessons that might be drawn for the future improvement of these systems. Figure 9 presents some of these inferences.

Figure 9: QOC diagram showing design options for fire avoidance.

As mentioned, figure 9 is derived from a subjective analysis of the MIIU report. The question of how to reduce the likelihood of an engine room fire can be addressed by three possible design options. The first option is to specify a maximum period within which any logged fault must be resolved. This is supported by the argument that it will ensure the timely correction of faults once they are discovered. The second option is to establish a detailed maintenance regime for all possible fire hazards. This is supported by the criteria that it will involve a regular inspection of all known fire hazards but is weakened by the criteria that it requires engineers to accurately predict all possible fire hazards. The final option relies upon training improvements to help staff detect faults. This avoids the need for engineers to identify all possible fire hazards and then create explicit inspection procedures as suggested by option two. It is important to note, however, that QOC’s do not enforce an exclusive selection between competing alternative. It would be perfectly possible to exploit all three of the options identified in Figure 9.

Figure 9 omits one recommendation that did emerge from the investigation but which was not directly addressed within the MIIU report. This is unfortunate because the recommendation has considerable implications for the readers’ view of this accident:

The Chief Engineer’s testimony contains the observation that the shut-off cock was re-designed in the aftermath of the fire. It might have been better to include this information within the main body of the report. The fact that this accident might have been avoided through improved design, rather than through increasingly elaborate training and maintenance procedures, introduces a further dimension to the development options shown in Figure 9.

Figure 10: Design options for fire avoidance revised after Chief Engineer’s testimony.

A major limitation with the previous diagram is that it provides little or no indication of the status or source of the options and criteria that are represented. In other words, we have no means of assessing whether or not it is possible to identify all possible fire hazards (Option 2). Nor does the previous diagram show how a particular option will guarantee that faults, which are detected, can be resolved within some maximum time period (Criterion 1). Such problems can be avoided by integrating design rationale techniques, such as the QOC notation shown in Figure 10, with the findings of accidents and incident investigations.

Figure 10 integrates CAE and QOC diagrams for the fire aboard the Leonardo da Vinci. The CAE diagram represents the MIIU's finding that technical failure and lax maintenance procedures led to the fire. A link is then drawn to the QOC diagram to show that this finding justifies designers in considering how to reduce the likelihood of fire within the engine room. It is important not to underestimate the benefits that such links provide. By linking development documents directly to the products of accident investigations, it is possible to ensure that designers base their subsequent development decisions at least partly upon those problems that have arisen with previous applications.

Figure 11: Using Previous Accidents to Justify Asking the Questions in QOC Diagrams.

Further links are shown between the QOC and the CAE diagrams. For example, the option of specifying a maximum time limit for fault resolution, O1, is supported by the MIIU argument, A11, that all leaks are potential hazards and must be dealt with as soon as practicable. The option of establishing detailed maintenance procedures to detect all possible faults, O2, is supported by the argument that not only had the shut-off cock spindle bonnet worked loose but there were other problems with the spindle of the Number 1 change-over lever, A10. Finally, the option not to attempt to predict all possible faults, O3, is supported by the Chief Engineer’s observation that the inspection of a particular fault, such as the change-over lever problem, might miss other faults, such as the one that developed in the shut-off spindle bonnet, A13.

It is important to emphasise that we are not primarily concerned with the particular causes of the fire aboard the Leonardo da Vinci. In contrast, we are concerned to demonstrate that designers can use past accidents to inform subsequent development. It is also important to note that the previous diagram also indicates the various lines of argument for and against particular design options. By combining the CAE and QOC diagrams, it is possible to see that more information is needed to explain the re-design of the spindle bonnet. It also raises questions about why the reasons for this design failure were not presented within the MIIU report when operator error and maintenance failure were considered in detail.

A number of areas for future work have been identified.

Our use of CAE diagrams has much in common with Peter Ladkin’s work on Why-Because Graphs (Ladkin, Gerdsmeier and Loer, 1997). Both approaches have, in the past, rewritten the prose that is presented in accident reports before annotating diagrams. This improves the clarity of the exposition and helps to reveal the argument in the accident report. We could have rewritten sections of the MIIU report so that each paragraph is easily classified as a Conclusion, a line of Analysis or an item of Evidence. There is, however, a danger that this rewording may lose the initial meaning of the investigator’s prose. We have, therefore, rejected this approach. Verbatim citations have been taken from the MIIU report so that they can be directly integrated into CAE diagrams.

The use of direct quotations raises a number of further problems. In particular, the natural language in an accident report cannot easily be reclassified into the simple distinctions supported by CAE diagrams. Previous sections have shown that particular paragraphs in the MIIU document present evidence, analysis and conclusions side by side. This rhetorical technique carries its own dangers; it can be difficult to distinguish between suppositions and facts or inferences. It is possible to represent the more fine-grained argumentation structures that are used in natural language. A number of detailed diagrammatic techniques have been used to represent and reason about the structure of argument in other domains. The gIBIS approach is an example of such a technique (Conklin and Begeman, 1989). However, it can be extremely difficult for readers to recall the many different types of nodes that are denoted in these diagrams (Moran and Carrol, 1995). Further work is needed to determine whether the costs of introducing additional syntactic features are offset by the greater precision in representing the arguments that are presented in accident reports. Similarly, more work needs to be conducted to determine whether it is appropriate to constrain the semantics of the links between CAE and QOC diagrams. Our initial development of this technique has exploited informal guidelines about the precise nature of these connections. However, it is likely that these guidelines may have to be codified if the approach is to be used by teams of accident investigators and systems designers.

CAE diagrams were developed to provide a graphical overview of the increasingly complex arguments that are presented in accident reports. The increasing integration of production processes and the recognition that accident reports need to address a range of contextual and environmental issues has led to longer and longer documents. However, the integration of CAE diagrams and design rationale, shown in Figure 11, raises a number of questions about the scalability of our approach. The MIIU report was only twenty-one pages long. Even so, the complexity of the argument makes it difficult to represent on a single page of A4 paper. Correspondingly, we might expect that many more pages would be required to capture the argument in the 488 pages of the Cullen report into the Piper Alpha fire. The length of such documents not only presents a challenge to our research but also helps to justify it. It is impossible for most readers to retain a coherent overview of the argumentation in such reports. The only solution to the scalability problems that frustrate the application of CAE diagrams would seem to be the provision of tool support. We are aware that the proliferation of hypertext links can lead to a complex tangle, which frustrates navigation and interpretation by interface designers and regulatory authorities.

Accident reports help to ensure that designers and engineers learn from the mistakes of the past. Unfortunately, it can be difficult for readers to locate the many different pieces of evidence that support particular arguments about the causes of an accident or incident. These items of information can be scattered throughout the pages of an accident report. A second problem is that readers are often forced to reconstruct complex chains of inference in order to understand the implicit arguments that are embedded within these documents.

This paper has argued that the graphical structures of Conclusion, Analysis, Evidence (CAE) diagrams avoid these problems. CAE diagrams explicitly capture relationships between evidence and analysis. They also provide a graphical overview of the competing lines of argument that might contradict particular interpretations of human ‘error’ and systems ‘failure’. They can also be used to capture the different viewpoints that reflect important disagreements between official accounts and eyewitness testimonies. However, these diagrams do not directly support subsequent development. We have, therefore, integrated design rationale techniques with the argumentation structures of CAE diagrams. This offers a number of benefits. For example, arguments that support or weaken particular design options can be linked to the findings that are documented in accident reports.

The Australian Maritime Incident Investigation Unit’s report into a fire in the engine room aboard the Leonardo da Vinci has been used to illustrate this paper. This case study has revealed the strengths of CAE diagrams, for example in providing an overview of the arguments that were presented. It has also helped to identify weaknesses. For example, the rhetorical techniques used in the report often made it difficult to identify the evidence that supported particular lines of argument. However, we were able to show that CAE diagrams can capture the alternate views that were presented in the Chief Engineer’s submission under clauses 16(3-4) of the Australian Navigation (Marine Casualty) Regulations. We also identified an important omission in the MIIU document. The official report failed to mention the design improvements that were made to the spindle bonnet, which caused the fire.

Thanks are due to the members of the Glasgow Accident Analysis Group and the Glasgow Interactive Systems group who provided valuable help and encouragement with this research. In particular, I would like to thank the Australian Marine Incident Investigation Unit for their openness in providing access to the accident report that is used to illustrate this paper. I would also like to commend the innovative way in which sub-regulations 16(3-4) of the Australian Navigation (Marine Casualty) Regulations helps readers to understand the broader context of accidents such as the fire that is considered in this paper.

S. Buckingham Shum, Analysing The Usability Of A Design Rationale Notation. In T.P. Moran and J.M. Carroll (eds.), Design Rationale Concepts, Techniques And Use, Lawrence Erlbaum, Hillsdale, New Jersey, United States of America, 1995.

J. Conklin and M.L. Begeman , gIBIS: A Tool For All Reasons, Journal Of The American Society For Information Science, 200-213, May, 1989.

US Department of Energy, Root cause analysis guidance document. DOE-NE-STD-1004-92. Office of Nuclear Energy, Washington DC, 1992.

C.W. Johnson, Visualising the Relationship between Human Error and Organisational Failure. In J. Dixon (ed.) Proceedings of the 17th International Systems Safety Conference, The Systems Safety Society, Unionville, Virginia, United States of America, 101-110, 1999.

C.W. Johnson, A First Step Towards the Integration of Accident Reports and Constructive Design Documents. In M. Felici, K. Kanoun and A. Pasquini (eds.), Computer Safety, Reliability and Security: Proceedings of 18th International Conference SAFECOMP'99, 286-296, Springer Verlag, 1999a.

C.W. Johnson, Improving the Presentation of Accident Reports over the World Wide Web. In J. Dixon (ed.) Proceedings of the 17th International Systems Safety Conference, The Systems Safety Society, Unionville, Virginia, United States of America, 396-405, 1999b.

P. Ladkin, T. Gerdsmeier and K. Loer, Analysing the Cali Accident With Why?...Because Graphs. In C.W. Johnson and N. Leveson (eds), Proceedings of Human Error and Systems Development, Glasgow Accident Analysis Group, Technical Report GAAG-TR-97-2, Glasgow, 1997.

Maritime Incident Investigation Unit, Investigation into a Fire in the Engine Room Aboard the Netherlands Dredger – Leonardo da Vinci off the Port of Dampier, WA, on 11 June 1998. Report 131, ISBN 0 642 20018 1, Department of Transport and Regional Development, Canberra, Australia, 1999.

T.P. Moran and J.M. Carroll (eds.), Design Rationale Concepts, Techniques And Use, Lawrence Erlbaum, Hillsdale, New Jersey, United States of America, 1995.

 J. Reason, Managing the Risks of Organisational Accidents, Ashgate, Aldershot, 1998.

P. Snowdon and C.W. Johnson, Results of a Preliminary Survey into the Usability of Accident and Incident Reports. In J. Noyes and M. Bransby (eds.), People in Control, 258-262, The Institute of Electrical Engineers, Savoy Place, London, United Kingdom, 1999.