Safety Culture Assessment: What are we Trying to Achieve? Joelle - PowerPoint PPT Presentation

Safety Culture Assessment: What are we Trying to Achieve? Joelle Mitchell Safety Psychology Conference Sydney, October 2012

Outline • What is safety culture? – Theory and research • Are perception surveys useful? – Assumptions – Evidence – Challenges • Is safety culture a valuable concept?

Culture: Theory and Research

Culture Theory • Safety culture is understood as a component of organisational culture • Schein (1985) is commonly referenced A248801 29/10/2012 4

Schein’s Model Artefacts Espoused Values Basic Assumptions A248801 29/10/2012 5

How is culture built? A248801 29/10/2012 6

How Is Culture Built? • A definable group with a shared history • Experience and solve challenges • Solutions work well enough to be considered valid • Taught to new members as the correct way to perceive, think, and feel in relation to the problem • Become basic assumptions • Informs future behaviour A248801 29/10/2012 7

Key Defining Features • Abstract construct (not a concrete phenomenon) • Relatively stable and enduring • Multiple dimensionality • Shared group norms • Socially constructed • Provides meaning about observed things A248801 29/10/2012 8

Safety Culture: A Definition • “The product of individual and group values, attitudes, perceptions, competencies, and patterns of behaviour that determine the commitment to, and the style and proficiency of, an organisation’s health and safety management.” – International Nuclear Safety Advisory Group (1991) • The way we do things around here • Why we do things the way we do them around here A248801 29/10/2012 9

Safety Culture Research • Perception surveys • Normative sets of factors • Many, many questionnaires • Many, many factors • Factor structures difficult to replicate • A few factors are common to most studies A248801 29/10/2012 10

Criticisms • “We have been too focused on a traditional research paradigm… that has produced very reliable results about very unimportant things . • …we have lost touch with some of the important phenomena that occur in organizations… simply because they were too difficult to study by the traditional methods available.” – Schein (1993) A248801 29/10/2012 11

Criticisms • “As applied by safety researchers, the culture concept is deprived of much of its depth and subtlety, and is morphed into a grab bag of behavioural and other visible characteristics, without reference to the meaning these characteristics might actually have, and often infused with normative overtones .” – Guldenmund, 2010 A248801 29/10/2012 12

Perception Surveys

Why Perception Surveys? • We want a leading indicator – To intervene before something goes wrong – To prevent major events • We like to benchmark and measure change – Between-groups comparisons – Trends • Easy to analyse • Lots of nice graphs • Relatively cheap • Relatively minimal time “off-tools” A248801 29/10/2012 14

Problems with Perceptions • Perceptions are subjective – Difficult to interpret meaningfully • Perceptions are volatile • Self-report issues – Honesty – Selective Memory • Construct Validity – Mono-method bias – What are we actually measuring? • Predictive Validity A248801 29/10/2012 15

Core Assumption • Safety Culture has a causal impact on safety performance – Safety culture can be reliably measured with perception surveys – Therefore, perception surveys provide a leading indicator of safety performance • Is this assumption accurate? A248801 29/10/2012 16

Core Assumption Part 1 • Can safety culture be reliably measured with perception surveys? A248801 29/10/2012 17

Is a Normative Approach Appropriate? • How does a culture develop? • Are all organisations likely to hold the same basic assumptions? • Are basic assumptions likely to impact organisational outcomes in uniform ways? A248801 29/10/2012 18

Do perception surveys measure safety culture? Artefacts Espoused Values Basic Assumptions A248801 29/10/2012 19

So What? • What cultural issues are identified after major incidents? – High level decision-making – Top-level leadership practices – Initiatives, priorities and targets – Allocation of resources – Routine violations • Unlikely to be detected by perception surveys A248801 29/10/2012 20

Core Assumption Part 2 • Safety culture is not reliably measured with perception surveys • But… • Do perception surveys provide a leading indicator of safety performance? A248801 29/10/2012 21

What has research found? • Weak to moderate correlations between perception survey scores and injury frequency rates (e.g. TRIFR) • Used to demonstrate tool validity A248801 29/10/2012 22

Common Interpretation Safety Perceptions Safety Behaviours Injury Rate A248801 29/10/2012 23

Another possibility… Safety Behaviours Injury Rate Safety Perceptions A248801 29/10/2012 24

…and another… Injury Rate Safety Safety Perceptions Behaviours A248801 29/10/2012 25

…and another! Safety Perceptions Variable ‘X’ Injury Rate A248801 29/10/2012 26

Is it a Useful Correlation? • Can we have poor safety outcomes and a low TRIFR? • Longford Gas Plant 1998 • Texas City Refinery 2005 • Deepwater Horizon 2010 A248801 29/10/2012 27

Predictive Value • Perception survey scores correlate with occupational injury rates • HOWEVER… • Low TRIFR ≠ Safe operations • Do perception surveys predict major incidents? A248801 29/10/2012 28

Case Study • Snorre Alpha, Norway – Antonsen, 2009 • Positive perception survey results • Gas blowout 12 months later • No lives lost – luck not design • No significant changes to the organisation or facility over that 12 months A248801 29/10/2012 29

Contradictory Findings Perception Survey results Incident investigation findings Safety a priority Production targets prioritised Rigorous risk assessments Lack of risk assessments and poor understanding High levels of compliance Non-compliance was normal Good safety communication Poor communication climate Incidents reported Not all incidents reported Learning and change from Limited use of own and others’ incidents incident information A248801 29/10/2012 30

Key Question • Are perception surveys the best way to spend limited safety resources? • Perception surveys do not predict major events – some correlation with occupational injuries • There are other lead indicators for occupational injury – accuracy – cost-effectiveness – time-efficiency A248801 29/10/2012 31

Safety Culture is a Valuable Concept

Qualifiers • Don’t try to ‘measure’ culture • OUTCOMES: – understand the meaning of observable things – uncover basic assumptions – develop effective improvement strategies – identify potential change barriers • NOT: – as a benchmarking exercise – as a replacement for other technical and systems processes A248801 29/10/2012 33

How? • Triangulate your methodology! • Use qualitative methods – Interviews – Observations – Focus Groups – Action Research A248801 29/10/2012 34

But we want to benchmark! • Use existing measurable data – Reporting rates – Inspection findings – Workplace observations – Audit findings – Quality Assurance – Incidents – Near misses – Outcomes and lessons learnt • Changing themes • Follow-up on previous findings A248801 29/10/2012 35

What about Perception Surveys? • Are the questions the right ones? • How was the survey developed and validated? – Industry – Company – Links to safety outcomes • Use data to drive qualitative inquiry – Open-ended questions – Interviews – Focus groups • Remember the limitations of perception data A248801 29/10/2012 36

Final Thoughts • The ultimate goal is to keep people safe • Safety resources are not unlimited • Target the tools and strategies that are most likely to lead to improved safety outcomes • Take an evidence-based approach A248801 29/10/2012 37

Thank you! Questions?

References • Antonsen, S. (2009). Safety Culture: Theory, Method and Improvement . Surrey: Ashgate Publishing Group. • Guldenmund, F. W. (2010). (Mis)understanding safety culture and its relationship to safety management. Risk Analysis, 30 , 1466-1480. • International Nuclear Safety Advisory Group (1991). Safety Culture. Vienna: International Atomic Energy Agency. • Schein, E. H. (1985). Organizational Culture and Leadership. San Francisco: Jossey-Bass. • Schein, E H. (1993). Legitimating clinical research in the study of organizational culture. Journal of Counseling and Development, 71 , 703-708. A248801 29/10/2012 39