Programme BRICK Programme Evaluation: How, why and what? The plan - PowerPoint PPT Presentation

External Evaluation of the BRICK Programme BRICK Programme Evaluation: How, why and what?

The plan …  Practical evaluation - meeting objectives  BRICK evaluation - aims and approach  BRICK evaluation - emerging findings  Learning - other Catalyst programmes  The future - legacy and sustainability  Q&A

Practical Evaluation Meeting objectives  Managing resources  Evaluation Plan  Research tool design - evidence gathering  Coverage - sample size  Experiencing the programme offer  Analysis  Reporting 

Best elements - what has worked? What have been the key successes?  ◦ Process of management or delivery ◦ Impacts on organisations ◦ Impacts on teams or individuals ◦ Softer or less intangible benefits ◦ Continuous learning ◦ Ability of participants to influence the design and delivery of the project ◦ Case study material – assessment and marketing

Learning points - what could have worked better? What have you all learned?  ◦ What hasn’t worked? ◦ What would you change with the benefit of hindsight? ◦ What are the barriers or challenges? ◦ What are the catalysts for success? ◦ Are there any differences between cohorts? ◦ What are the key successes/learning points? ◦ Have lessons been learnt and good practice shared across partners?

BRICK Evaluation  External evaluation by an independent consultant  Cover training and materials over the final year of the programme as well as documentation from years 1 to 3 inclusive  Complement internal evaluation work  Assessment of the medium/long term impact of participating in the BRICK programme of early BRICK attendees  Assessment of how the BRICK programme has progressed, achieved against objectives and legacy

BRICK Evaluation Methodology Inception Document BRICK Delegates Analysis meeting Review Online survey Qualitative Development Data Analysis Interviews with at Quantitative A . Project Management D. Analysis and Reporting and agreement B. Desk Research C. Fieldwork Research Tool least 25 Reporting of an Evaluation Design participants Plan Ten case studies Stakeholders Regular Draft Report Telephone progress Final Report interviews reporting stakeholders and programme sponsors Workshop observation

BRICK Evaluation - Emerging Findings Unless stated based on 71 responses to online survey of  people accessing the following elements of the BRICK programme offer:  Workshops  Mentoring  BRICK +  Online resources  Heritage Helpline  Conferences Complemented by stakeholder consultations 

In which region or country is your organisation based? 20% 17% 15% 13% 13% 10% 10% 10% 7% 7% 7% 6% 4% 4% 5% 3% 0%

How would you describe your organisation? 48% 50% 40% 30% 18% 20% 13% 10% 10% 6% 3% 3% 0%

How have you received information about the BRICK Programme? Prince’s Regeneration Trust website 61% Direct invitation (BRICK and/or PRT) 56% BRICKwork website 49% Word of mouth 29% Heritage Update 23% Heritage Lottery Fund 23% Architectural Heritage Fund 20% English Heritage 9% Association of Preservation Trusts 7% Local council 6% Historic Environment Scotland 4% Twitter 4% Facebook 4% Cadw 3% Historic Environment Division - DoE Northern Ireland 1%

How would you rate the BRICK workshops? 70% 61% 60% 55% 51% 51% 50% 48% 49% 46% 50% 43% 43% 41% 39% 40% 40% 40% 38% 40% 30% 20% 20% 10% 8% 7% 10% 5% 2% 2% 2% 2% 0% Excellent Good Average Poor Very Poor

Most useful aspect of the BRICK workshops? ‘Questions were answered in a way that made me sense to me instead of the terminology going over my head’ ‘Ability to meet other people involved in similar projects, and to network’ ‘Meeting with other people who were already further down the project path than ourselves, gave us hope !’ ‘The talks, on the whole, were excellent, informative, inspiring and useful’ ‘Making contact with experts’

Most useful aspect of the BRICK workshops? ‘The inspirational talks by speakers who were already actively carrying out projects’ ‘The BRICK workshop provided a good, clear and informative picture of the elements involved’ ‘I came away feeling I had tools and knew how to go about approaching funders’ ‘Talking about governance and how to handle and run organisations with willing volunteers and recruit people to become involved’

Suggested improvements to the BRICK workshops? ‘more case studies from rural areas would have been more relevant’ ‘more practical workshops would have been welcomed with more relevant case studies’ ‘slightly higher level information - in addition to info for organisations earlier in the project process’ ‘more ways of fundraising, explanation of all funding opportunities that are available’ ‘not so many topics per day to give more opportunity for discussion to learn from other experiences’

How would you rate the mentoring support? (n=18) 70% 61% 60% 50% 50% 39% 40% 33% 28% 28% 28% 28% 30% 20% 17% 17% 11% 11% 11% 11% 10% 6% 6% 6% 6% 6% 0% The quality of the Enabling your Helping to unlock Increasing your mentoring project to secure barriers to the confidence in funding project undertaking a proceeding community project Excellent Good Average Poor Very Poor Don't Know N/a

Impact of mentoring support? (n=18) 39% 39% 40% 33% 33% 33% 33% 30% 28% 28% 28% 28% 28% 22% 22% 22% 17% 20% 17% 17% 17% 17% 17% 11% 10% 0% Increasing Upskilling Improving Developing a Developing a Marketing and Informing the capacity to trustees organisational sound business project vision PR ownership, raise funding governance case repairs and reuse of the building Substantial Significant Some Very little None N/a

Describe the impact of the mentoring programme on you and/or your organisation?

Suggested improvements to the mentoring programme ‘more time input’ ‘can you clone another 6 Laura Norris's so she can visit us more often please?’ ‘ensure that as many stages and needs are covered as possible and that communication remains open until a conclusion has been reached’ ‘Felt stretched as a programme and this was frustrating. I wonder whether we were actually the right sort of organisation to be accepted into the programme - the selection process was fairly brief as I recall’

How would you rate the BRICKworks digital platform and wider website? (n=17) 70% 65% 59% 60% 53% 53% 50% 35% 40% 29% 24% 25% 29% 6% 30% 13% 12% 12% 20% 12% 12% 6% 10% 0% Overall as a Ease of Enabling Promoting Facilitating Offering and Searching for A source of resource finding users to projects and online securing pro volunteers news information connect and celebrating discussions bono advice and potential on resources share success from trustees and events resources professionals Excellent Good Average Poor Very Poor Don't Know Not used

Impact on governance 48% 50% 42% 40% 30% 30% 20% 20% 20% 14% 10% 3% 3% 2% 0% 0% No knowledge A little Some Fairly confident Full confidence knowledge knowledge Prior to BRICK Post BRICK Average attribution of impact to the BRICK programme - 50%

Impact on understanding of Fundraising 48% 50% 40% 40% 30% 25% 22% 20% 20% 17% 10% 3% 3% 2% 0% 0% No knowledge A little Some Fairly confident Full confidence knowledge knowledge Prior to BRICK Post BRICK Average attribution of impact to the BRICK programme - 52%

Impact on understanding of business planning and operational insight 50% 47% 44% 40% 30% 25% 21% 20% 16% 11% 10% 5% 5% 3% 0% 0% No knowledge A little Some Fairly confident Full confidence knowledge knowledge Prior to BRICK Post BRICK Average attribution of impact to the BRICK programme - 46%

Impact on understanding of digital innovation 50% 41% 40% 34% 29% 30% 20% 20% 15% 8% 8% 8% 10% 5% 2% 0% No knowledge A little Some Fairly confident Full confidence knowledge knowledge Prior to BRICK Post BRICK Average attribution of impact to the BRICK programme - 44%

Impact on understanding of collaborative project visioning and options development 60% 51% 50% 40% 32% 30% 25% 22% 20% 14% 8% 10% 5% 3% 3% 2% 0% No knowledge A little Some Fairly confident Full confidence knowledge knowledge Prior to BRICK Post BRICK Average attribution of impact to the BRICK programme - 46%

To what degree to you think the BRICK programme been successful in supporting community groups trying to save their local heritage and create a better future for their local area? 50% 45% 39% 40% 30% 20% 10% 10% 4% 2% 0% Completely To great extent Somewhat Very little Not at all

Learning from other Catalyst Programmes Applying the learning   Barriers  Catalysts Legacy and sustainability   Legacy actions  Challenges  Catalysts

Applying the Learning Barriers (n=59) 40% 34% 30% 22% 19% 20% 17% 10% 5% 3% 0% I/we have I/we have I/we have I/we have I/we haven’t I/we have been able to been able to been able to been able to been able to not tried to apply apply the apply some apply a little apply any apply the everything majority training and support