The Attributer’s Blog – Governed

In the previous blog article (Conflicted, April 2019) The Attributer examined the potential conflict between the decisions made by autonomous systems and those made by their human operators. The case study was the alleged failure of the MCAS[1] system on the Boeing 737 Max aircraft and the part it may have played in two recent crashes of this aircraft. The casualty rate for these two accidents totalled 346 lost lives. That level of catastrophe draws a lot of media attention and detailed expert analysis. We draw our information for this article from a BBC News web-site article published on 17 May 2019 and accessed on that same date. [2]

Our purpose is to look at a specific aspect of safety and security management – that of corporate governance and its importance in applying both SABSA and STPA[3]. In a tutorial on STPA[4], Dr John Thomas makes clear the need for governance at the highest level of national government. For a discussion on the SABSA Governance Model see Figure 5 in the SABSA white paper W101: Architecting a Secure Digital World.[5]

The SABSA Institute takes a deep interest in the systems engineering aspects of security and safety architecture and their management. SABSA Thinking is based on systems thinking[6]. Everything is a system or a system of systems. All that differs from one system to another is the definition of scope. This civil aviation case study clearly exemplifies the importance of system scope. We call the in-scope system the ‘system of interest’ or SOI.[7]

We might say that the SOI scope is the 737 Max aircraft. However, reading the reports of what happened in the cockpits of those two crashed airplanes you soon recognise the importance of the flight crew and the actions they took. That leads you to examine their experience, their training, the systems that support their training, the resources available to them for incident management, their interaction with ground staff and air traffic controllers, and of course, the design of the MCAS sub-system, the flight testing, and the certification of the aircraft. 

The SOI scope also includes the manufacturer and its supply chain companies. It includes industry regulators and independent safety auditors and inspectors. It also includes the shareholders of all the commercial interests, especially the shareholders of Boeing itself. It doesn’t take much imagination to see that within a ‘system of interest’ a ‘conflict of interests’ is easily created unless the governance lifecycle is well-constructed for independent oversight and honestly executed.

The life-cycle of an aircraft is complex. Every stage of that life-cycle must be governed to give assurance that it is executed to standards of quality and safety that guarantee the safety of passengers and crew, and any other third parties on the ground where a plane might crash. Ultimately the top of the governance hierarchy is the regulating body in any industry that is regulated – in this case the FAA[8] in the US. They are responsible for certifying aircraft manufactured by Boeing. Flight safety is one of the key concerns of the FAA. The 737 Max was certified by the FAA. Currently that certification is on hold.

The FAA is a relatively small government agency when compared with the corporate might of Boeing. In the US alone, Boeing employs more than 135,000 people, and claims to support more than a million more jobs across thousands of American suppliers. It is the world’s biggest aerospace company, with annual revenues of more than $100bn, and profits of more than $10bn. That’s a lot of economic and political power. It also leads to Boeing being the main pool of expertise for engineers and auditors with the knowledge and skills to certify a complex aircraft. Now we begin to see how conflicts might arise. One of the goals of architecture work is to identify and resolve conflicts.

In fact, the FAA outsources 90% of its certification work to Boeing. Allegedly, the FAA would need an extra 10,000 employees to do the same work, and it would cost US taxpayers around $1.8bn per year. The manufacturer is self-certifying in so many ways that it carries the burden of managing the significant challenges of intra- and inter-domain systemic risks and their unintended consequences. Independent oversight is the central challenge that they face, whilst simultaneously protecting their commercial interests and trading in a competitive marketplace.

So, taking this scope analysis a stage further, we can see that the entire global civil aviation industry is the real scope of the SOI here, not just the MCAS software, nor the 737 Max. Why did Boeing build the 737 Max? The answer is because of competition from Airbus and their development of the A320 into a faster, more efficient workhorse for short and medium range transport. Boeing needed a response to this and the most obvious one was to redevelop the 737, despite some serious technical difficulties over mounting the new engines on a re-engineered wing design. At 1^st January 2019 Boeing had an order book for more than 4,500 of the 737 Max, which tells you how right they were in their judgement of the market demand. The 737 family is the bestselling aircraft in the history of civil aviation.[9]

In previous Attributer articles we have pointed out the synergy between SABSA and STPA, the safety engineering methodology from MIT[10]. The key to that synergy is the focus on systems engineering, and the work to expose and correct undesirable system behaviours caused by emergent properties of systems. The SABSA community has much to learn from the modern safety engineering community led by Dr Nancy Leveson’s team at MIT. The SABSA Institute will pursue all possible avenues for collaboration with a multi-disciplinary approach to the security of safety critical systems.

The SABSA Institute will issue a Call for Participants for this initiative in the near future.

[1] Maneuvering Characteristics Augmentation System
[2] https://www.bbc.co.uk/news/resources/idt-sh/boeing_two_deadly_crashes
[3] Systems theoretic process analysis
[4] http://psas.scripts.mit.edu/home/wp-content/uploads/2014/03/Systems-Theoretic-Process-Analysis-STPA-v9-v2-san.pdf Slide 4
[5] https://sabsa.org/architecting-a-secure-digital-world-download-request/
[6] Enterprise Security Architecture: A Business-Driven Approach. 2005. Sherwood, Clark and Lynas. Chapter 5
[7] As defined in ISO 42010: Systems and software engineering – Architecture Description
[8] FAA: Federal Aviation Authority
[9] http://www.b737.org.uk/sales.htm
[10] Massachusetts Institute of Technology