Cheshire Puss, she began, rather timidly, as she did not at all know - PowerPoint PPT Presentation

“Cheshire Puss,” she began, rather timidly, as she did not at all know whether it would like the name: however, it only grinned a little wider. “Come, it's pleased so far,' thought Alice, and she went on. `Would you tell me, please, which way I ought to go from here? “That depends a good deal on where you want to get to,” said the Cat. “I don't much care where—” said Alice. “Then it doesn't matter which way you go,” said the Cat. “--so long as I get somewhere ,” Alice added as an explanation. “Oh, you're sure to do that,' said the Cat, `if you only walk long enough”

L o g ic Mo de ls fo r Stra te g ic Pla nning & E va lua tio n E RC We binar De c e mb e r 18 &19, 2017

L e a rning Ob je c tive s We b ina r a tte nde e s will b e a wa re o f the : • E va lua tive inq uiry c yc le • Ro le o f mo de ls in stra te g ic pla nning • T e rms use d in c re a ting a the o ry o f c ha ng e a nd lo g ic mo de l • Ste ps in c re a ting mo de ls • Ste ps to a lig n mo de ls a nd stra te g ic pla n 3

E va lua tio n Ove rvie w NSF Appro a c h, Princ iple s & E xpe c ta tio ns 4

E va lua tive Stra teg ic Pla n I nq uiry Pro c e ss 1 c rite ria fo r wha t c o nstitute s e vide nc e de fine institutionalize 3 ho w e vide nc e ho w e vide nc e , c a n b e o nc e analyze d de sign re port/ use a na lyze d, and synthe size d & synthe size d, use d c an be use d Annua l Report E va lua tion Pla n me asure analyze 2 pro c e sse s & me tho ds fo r o b ta ining e vide nc e 5

Stra te g ic Pla nning Compr ehensive Vision Cur r ent State

Theory of Change and Logic Cre a ting Fo c us Models help identify key program components Strate gic Plan Ho w w ill w e do o ur w o rk? Ho w do w e mo nito r pro g re ss a nd de fine suc c e ss? E valuation Plan Wha t e vide nc e do w e ne e d to ma na g e a nd impro ve the pro je c t? Ho w w ill w e g a the r a nd use e vide nc e 7

Ma pping & Asse ssing Pro g re ss Ge Get Compr ehensive Theory of Change Vision Cur r ent State Do

Ge Get Do SUPPORT 9

De ve lo ping Mo de ls Stra te g ic Pla ns are Blue prints fo r Suc c e ss 10

Do Ge Get T he o ry o f Cha ng e E xa mple Inc re ase d re se arc h c apabilitie s Re se arc h and ne w kno wle dg e E ng ine e ring Dive rse , g lo bally c o m pe titive , te am - Wo rkfo rc e o rie nte d wo rkfo rc e De ve lo pme nt Im pro ve d value c hain, te c hno lo g y Inno vatio n transfe r, and e ntre pre ne urial c ulture E c o syste m C ulture o f Partic ipants fro m all bac kg ro unds Inc lusio n partic ipate and suc c e e d Infrastruc ture & Im pro ve d m anag e m e nt, Manag e m e nt infrastruc ture , and im ple me ntatio n 11

Mo de ling a T he o ry o f Cha ng e fo r yo ur Stra te g ic Pla n 1. Use a she e t o f e a se l po st-it. 2. Ide ntify the fo unda tio na l c o mpo ne nts a nd the ir ma jo r thrusts fo r yo ur pro je c t a nd write e a c h o n a 5x7 po st-it. 3. Ide ntify the o ve ra rc hing re sult fo r e a c h c o mpo ne nt/ thrust a nd write e a c h o n a 5x7 po st-it. T a ke Home E xerc ise 1 12

Ge Get Do ? Re se arc h ? E ng ine e ring Wo rkfo rc e De ve lo pme nt ? Inno vatio n E c o syste m ? C ulture o f Inc lusio n ? Infrastruc ture & Manag e m e nt 13

Simple L o g ic Mo de l DO GET Strategies Results Short-term Intermediate- Long-term Inputs Activities Outputs Impact Outcomes term Outcomes Outcomes 14

L o g ic Mo de l Co mpo ne nt De finitio ns T erm Definition I nputs I nc lude fi nanc i al, hum an, o rg ani zati o nal, c o m m uni ty o r syste m s re so urc e s e sse nti al to i m ple m e nt the pro je c t. Ac ti vi ti e s T he spe c i fi c ac ti o ns that m ake up the pro je c t. T he y c an i nc lude to o ls, pro c e sse s, pro duc ts, e ve nts, te c hno lo g y and o the r aspe c ts o f the i nte rve nti o n de plo ye d to ac hi e ve de si re d re sults. Outputs I nc lude de sc ri pti o ns o f the type s, le ve ls and audi e nc e o r targ e ts fo r the pro je c t. Co untable attri bute s o f the ac ti vi ti e s i f ac c o m pli she d. (F re que nc y, I nte nsi ty, T arg e ts) Outc o m e s T he c hang e s i n pro je c t parti c i pants o r o rg ani zati o ns, as a re sult o f the pro je c t. Can i nc lude c hang e s i n aware ne ss, kno wle dg e , ski ll, and be havi o r. (Spe c i fi c , Me asurable , Ac ti o nable , Re ali sti c , T i m e d) I m pac t T he ulti m ate c hang e i n an o rg ani zati o n, c o m m uni ty o r o the r syste m . Ofte n o c c urs afte r the g rant c yc le has e nde d. 15

Pro g ra m-le ve l L o g ic Mo de l E xa mple s 16

Insight Policy Research (2015) EFRI Outcome Monitoring System

Go a ls Cha ra c te rize d b y Me a sura b le Outputs & Outc o me s Goals Outputs & Outcomes Metric Breakthrough New products #/5 years/#university partners Technologies New methods New processes Papers # of journal publications/5 years/# partners Stakeholder IAB member satisfaction % of membership renewals averaged over a 4-year period Satisfaction Leveraged funding $ other new sources : $ NSF/5 years Researcher satisfaction Likert scale satisfaction Student Outreach Graduate research grants # of grants for theses and dissertations Student participation # of student members/5 years Student Development Student projects # of student publications/presentations/5 years Mentorships Median ratio researcher : graduates/5 years Technology # Degrees # (BS + MS + PhD)/5 years Commercialization Licensing # of new licenses/5 years Students hired by IAB member % of participating graduates hired by IAB member firms averaged/5 years Consulting # of consulting contracts for researchers to IAB member companies/5 years Knowledge Transfer Website Quality of information dissemination on website Prof org memberships # of professional memberships held by IAB members/5 years Papers # of co-publications (researcher and industry member)/5 years Conference presentations # of conference presentations/5 years Workshops # of seminars and workshops held Gibson, E. and Daim, T. (2016). A measurement system for science and engineering research center performance evaluation. Engineering and Technology Management. 2016 Proceedings of PICMET ‘16: Technology Management for Social Innovation.

L og ic Models & Stra teg ic Pla n T heories of Cha ng e “Vision” Impre ssio n & Re a lity Re pre se nta tio n Ab stra c tio n 22

Pro po sa l L o g ic Mo de l Stra te g ic Pla n 1. F o r e ac h fo und atio nal c o mpo ne nt and strate g y, c o mple te the ac tivitie s c o lumn. 2. L ist the e xpe c te d re sults (g o als) and sho w inte rre latio nships if ne e d e d . 3. F ill in the g aps (inputs, o utputs, sho rt- and inte rme diate te rm o utc o me s) to sho w the links be tw e e n yo ur “ d o ” and “ g e t.” 4. Che c k to assure the links fro m le ft to rig ht are in a lo g ic al, fe asible , se que nc e . 5. E nsure that the mo d e l re pre se nts the pro je c t (w / o unne c e ssary d e tail. 6. Re vise d and upd ate the mo d e l pe rio d ic ally to re fle c t c hang e s in the pro je c t. 23

L o g ic Mo de l & Stra te g ic Pla n Long-term Short- & Outcomes & Outputs Intermediate Inputs Activities Impact Outcomes 24

Que stio ns to Guide Re vie w o f the L o g ic Mo de l & Stra te g ic Pla n 1. Are the ma jo r inputs, a c tivitie s, a nd o utputs c o nsiste nt a nd suffic ie nt to a c hie ve de sire d o utc o me s? 2. Are the stra te g ic g o a ls o utc o me o rie nte d? 3. Are the re missing stra te g ic g o a ls? 4. Ho w do c o lle a g ue s no t fa milia r with yo ur pro je c t, inte rpre t yo ur mo de l? 25

Mo de ls fo r E va lua tio n Pla nning Context Process Outcomes Quality, Use & Relationships Effectiveness Satisfaction & Capacity INFLUENCES LONGER -TERM SHORTER- TERM ACTIVITIES OUTPUTS IMPACT OUTCOMES OUTCOMES INPUTS Form at iv t ive Evalu luat i t ion Sum mativ ive Evalu luat i t ion Are we doing the Are we doing the work Are products/services accessed What difference have “right” work? “right”? and used as intended? we made? 1 2 3 4 What have we learned about what it 5 takes to do & sustain this work? 26

Re so urc e s • https://www.wkkf.org/resource- directory/resource/2006/02/wk-kellogg-foundation- logic-model-development-guide • https://fyi.uwex.edu/programdevelopment/logic- models/ • http://ctb.ku.edu/en/table-of- contents/overview/models-for-community-health-and- development/logic-model-development/main • http://www.pointk.org/client_docs/File/logic_model_w orkbook.pdf • http://www.sagepub.com/sites/default/files/upm- binaries/23938_Chapter_3___Creating_Program_Logic _Models.pdf 27

Ge ne ric Stra te g ic Pla n T e mpla te Project Thrust 1: Description … Ge Get Do Do 28