Some Thoughts about Reports Peter Findlay (QAA)
Point of departure • Reports are a form of message • Reports follow the general rules which govern all messages • Communication theory applies here • We need to think about the communication process • It is not simple
Structure of presentation The message The sender The receiver (HEI, (panel, Agency) stakeholders) (The report) • Context • Form • Context • Purpose • (Language) • Reading • Writing • Content • Understanding, Actions
‘ Encoding ’ • Encoding means – Knowing the purpose of the message – Knowing what you need to write about – Knowing how to write it – Knowing who will read it – Knowing how they will read it
‘ Decoding ’ • Decoding means – Knowing how to get the message – Knowing how to read it – Understanding it correctly – Knowing what you should do when you have read it
NOISE
‘ Noise ’ in communication – ‘ stuff happens ’ All the surrounding events, activities, experiences that take place at the time of the message • E.g. – the compiler of the report is going to retire in one month ’ s time • E.g – there is an election taking place with new higher education policies ahead • E.g - the institution is in financial trouble
Encoding and Decoding • Choice of language and tone • Common understanding of the code used (i.e. language, terminology and expressions • The level of understanding of the code used will differ between reader groups • The code has to be learned • The easiest commonly understood code is the simplest language
The report is the outcome of the process • Agency policy and method • Agency officer and review team • Documentation and information • Visit and confirmation etc • Drafting the report • Editing the report • Finalising the report • Completing the report is the AIM of all parts of the process
Sending the Message Stages in producing the report
Building the team for the project
The visit and the meetings
Gather the information…..
Doing the analysis
….and now for the report…is it that simple?
Report production - writing Consulting Drafting Checking Finalising Compiling Editing Publishing Not that much fun??
And now for the report……it can sometime feel like this!
Challenges in report production • Individual task, highly responsible • Weak contributing drafts from panel • Wrong form and content from panel • Judgements don ’ t relate to the main body • Inadequate evidence-base to support evaluation • Getting the language right…. • Unsure about exact purpose • Unsure about exact readership • Time management and Deadlines….
Yes – deadlines…….!!
Expectations: Who will read the whole report right through? How many people do How many people do you think should read you think will actually the whole report really read the whole 1 ( ideally )? report? 10 25 50 100 2000 500 5000 1000
Who ‘ owns ’ the report? • The expert panel? • The member of the panel who writes it? • The Agency officer who writes it? • The Agency committee that approves it? • The Agency as a legal entity? • The institution? - - Can you own part of the report? - What are the implications?-
The message
Reports – the key considerations • Length • Structure • Content • Style - language • Tone
Ideals • Length – as short as possible • Structure – as clearly defined as possible • Style – simple, precise, clear, focused • Tone – neutral, measured, objective • Content – minimum description, maximum analysis and evaluation • No need to describe detailed procedures
Detailed description? WHO FOR?
Style - virtues • Well structured paragraphs ‘ beginning, middle, end ’ • Sentences which are simple and clear (never longer than two lines of text!) • Arguments clearly based on evidence • Point to relevance to the review questions/agenda/ framework • Regular summaries of each section
Style: - avoid • Over-complex sentences • Jargon and cliché ‘ cutting edge ’ ‘ pushing the envelope ’ etd • Emotive or strongly judgemental words ( ‘ wrong ’ ‘ chronic ’ ‘ harmful ’ ‘ bad ’ ‘ impressive ’ ‘ outstanding ’ ) • Obscure words ( ‘ inchoate ’ ‘ serendipitous ’ ) • Ambiguity • Too much unnecessary detail • Personality • Too much reference to the process (the review visit etc)
The jargon of quality • ‘ the quality management system ’ • ‘ PDCA cycle ’ • ‘ key performance indicators ’ • ‘ feedback loops ’ • ‘ quantitative data analysis ’ • ‘ action planning ’ • ‘ student learning opportunities ’ • ‘ Constructive alignment ’ ==Who needs it??
Good words ….QA favoured vocabulary • confidence • responsibility • accountable • reliability • soundness • effective
The receiver (s)
‘ Transparency ’ for whom? • The communication of reports is controlled by the format and language used • Transparency for one set of readers may be a limiting device for other readers • Language can deliberately be used to target specific reader groups and exclude others • Is it possible to communicate with all possible reader groups?
Language – soft or hard? Diplomatic: “ the university may like to consider the advisability of conducting regular student surveys ” = Blunt: “ The university must introduce student surveys immediately ’
Use of Coded Language • “ the university may like to consider the advisability of conducting regular student surveys ” • “ further strengthen the existing arrangements ” = you have not done this properly yet • Introduce more regular reviews = it ’ s not planned
Babel or ‘ Euroglish ’ Q-Speak? OR – IS IT ONE TONGUE , MANY VOICES? ?
Language politics • National languages of small nations don ’ t have same impact or comparability (but stay in the culture) • Translation into the lingua franca (English) brings risks of mistranslation, confusion, misunderstanding • We don ’ t realise how varied our understanding of key terms in English can be • Many Englishes …..(US -UK- and others)
Beware! You can ’ t always be sure what English means to the reader!
Shared English? – Differing readings, Confusions and Misunderstandings…(2006 ENQA WORKSHOP) • Standards • Programme • Accreditation • Validation • Experts • Evaluation • Audit • Assessment • Enhancement
Assessment Definitions • ‘ the evaluation or estimation of the nature, quality, or ability of someone or something ’ (OED) A general term that embraces all methods used to judge the performance of an individual, group or organisation. (INQAAHE)
Assessment - students • Assessment in UK HE usage more generally refers to the assessment of someone i.e. the student Assessment is a generic term for a set of processes that measure the outcomes of students' learning, in terms of knowledge acquired, understanding developed, and skills gained. It serves many purposes. Assessment provides the means by which students are graded, passed or fail. (AUQA)
Some terms in student assessment • Assessment strategy • Alignment of assessment • Continuous assessment • Formative assessment • Summative assessment • Self-assessment • Peer assessment • Group assessment Assessment provides information about levels of academic achievement and performance, and is therefore closely related to the determination of academic standards
Institution and programme assessment CHEA: Assessment A diagnostic form of quality review and evaluation of teaching, learning, and programs based on a detailed examination of curricula, structure, and effectiveness of the institution, its internal review, and quality control mechanisms.\ Bologna/Bergen: Assessment The process of systematic gathering, quantifying and using information to judge the effectiveness and adequacy of a higher education institution or a programme. It implies evaluation of core activities. It is a necessary basis for a formal accreditation decision.
UK usage The assessment of an institution or programme is more usually described in other terms in UK HE e.g.: • Programme review • Course evaluation • Institutional audit • Institutional review
but there are also exceptions….to this rule! The research assessment exercise (RAE) Teaching quality assessment (TQA)
Aim of the message What do your report readers need? - Understanding of complexity? - Clarity of message? - Brevity of message? - Simple signals?
How much attention?
Alternatives to the ‘ 40 page narrative ’ • A checklist approach? • Nothing but the recommendations? • Short summary for the public (published) plus detailed Annex for the institution (private)? • Agreed Action Plan? – future orientation?
Is a checklist approach the answer?
Recommend
More recommend
