SETIG Systems (Un)Conference Update AEA 2016 2015 Follow-up - PDF document

SETIG Systems (Un)Conference Update AEA 2016 2015 Follow-up Survey

Connecting with like- Over half of 2015 participants are using new minded colleagues knowledge or skills learned from the who are interested in learning together (Un)Conference. and sharing resources about systems. Yes, occasionally 48% Great Not at all 23% conversation with other No time or opportunity, but 23% evaluators would like to (or plan to) about systems evaluation. Yes, frequently 7% (n=31) The fostering of a community, which is Fewer participants connected with someone definitely they met through the 2015 (Un)Conference. promoted by the kind of event you organized, is very important - keep on doing it! Being new to systems No 71% evaluation the most useful things … were [the] basics... and being able to "listen in" on the conversations in various rooms by those with more experience Yes 29% and expertise. (n=31) For those that did connect, the majority connected with one person. • connecting with like-minded others 1 person 56% • the informal unconference environment • great resources shared 2 people 22% • a novel way to learn and connect 3 people 11% 4 people 0% 5 or more 11% (n=9)

• length (three days too much) • encourage more active participation and conversation • develop clearer focus 2016 Western Hemisphere attendance was lower than the previous year. And registration continues to be double actual attendance. 249 Registered Day 1 Attended 127 220 Day 2 2015 80 Western Eastern 188 Day 3 51 Hemisphere Hemisphere 69 attendees 29 attendees 191 2016 69 People still experience problems with the The Eastern Hemisphere session met more technology. participants’ expectations. Strongly disagree Disagree Neutral Agree Strongly Agree 47% 49% Registration process straightforward 8% 4% 25% 63% Western Hemisphere Americas Eastern Hemisphere Received information 50% 46% 8% needed prior to 4% 4% 50% 42% (Un)Conference 24% None or few problems 4% 15% 6% 30% 38% 92% with (Un)Conference 76% 4% 4% 8% 50% 33% software 4% 20% 49% 27% I increased my knowledge of the topic 4% 58% 38% n=46 n=24 n=46 n=24

Some Western hemisphere sessions were rated higher than others. The Eastern Hemisphere session had more positive The first session was usually rated higher than the second . outcomes. 4.5 Support for a Systems Approach: Strategies for Promoting Buy-in 4.5 Assisting the Design of Complex Programs: Real-life Examples Strongly disagree Disagree Neutral Agree Strongly Agree 4% 11% 27% 42% 16% Feel more Designing Useful Systems Visualizations: Tools & Strategies 4.2 4.4 connected to colleagues 4% 8% 33% 54% 4.3 Identifying System Leverage Points: Helpful Approaches Distinguishing Systems Evaluation: Where are the Boundaries? 3.3 4.3 11% 16% 47% 27% Good value 3.8 Using a Complex Adaptive Systems Approach: Real-life Examples for time spent 8% 50% 42% Designing a Systems-based Evaluation: Challenges and Strategies 3.3 3.3 Similarities & Differences between Design vs. 2.0 3.2 Systems Thinking: Applied to Evaluation Would attend 13% 40% 45% Importing Tools from the Design World into another 3.0 3.7 Systems Evaluation: Think Big! (Un)Conference 4% 9% 87% Basics of Systems in Evaluation: Getting 2.0 2.4 Oriented Bridging the Know-Do Gap: Praxis for 1.7 Systems-oriented Evaluation n=46 n=24 Eastern hemisphere sessions, both first and second , were Suggestions for improvement rated more highly. 5.0 Have fewer breakout rooms to ensure rooms are full. The cultural fit of systems thinking in indigenous evaluation contexts 5.0 5.0 Add brief position statements to each room to help set the A systemic approach to evaluating Whānau Ora in a Pasifika context 4.0 5.0 tone for discussion. 4.0 4.8 Ensure each room has a facilitator to take the lead with Using a Developmental Evaluation Approach in Systems Evaluation topic suggestions and shape discussions, especially in sessions with a lot of beginners. Thinking systemically when tackling a value for money in evaluation 4.2 4.7 4.0 Use a technology which is less likely to be blocked by 4.0 4.0 Systems thinking and complexity methods & tools we might use in evaluation some employer firewalls, i.e. government. 3.8 Basics of Systems in Evaluation: Getting Oriented Encourage and support more people to attend and fully participate in the (Un)Conference. Participants want to learn more Going forward participants will…. about… …Connect more Actual techniques and methods for systems evaluations. regularly to the Systems group at How design, evaluation, and systems thinking overlap. AEA in some way. Brief occasional What defines systems thinking from regular evaluation. meetings aren't a substitute. …Process lots of How to balance different values that stakeholders bring new insights I can when tough decisions need to be made about resource add into my work - allocations. far more than one. Determining the boundaries of a system and how to facilitate that conversation with clients.

2016 Committee Members Bethany Laursen 
 Judy Oakden 
 Jan Noga 
 Kylie Hutchinson 
 Ginger Fitzhugh Jenny Lawlor 
 Jen Nunez 
 Alana Robilliard Special thanks to Heather Britt Brandon Coffee-Borden Bob Williams