Introducing 2) Develop skills to utilize the online GENIE GENIE - PDF document

3/18/2014 The Guide for Effective Nutrition Interventions and Education Learning Objectives 1) Explain the process used in developing and validating the Guide for Effective Nutrition Interventions and Education (GENIE). Introducing 2) Develop skills to utilize the online GENIE GENIE checklist and become familiar with GENIE’s resources. 3) Apply GENIE’s criteria to benefit your professional practice. Outline Meet Today’s Presenters Rosa Hand, MS, RDN, LD 1) Why was GENIE created Senior Manager Dietetics Practice Based Research Network 2) How was GENIE created Academy of Nutrition and Dietetics -Reliability and validity testing Jenica Abram, MPH, RDN, LDN 3) Introduction to the GENIE checklist and ConAgra Foods Foundation Nutrition Education Research Fellow scoring Academy of Nutrition and Dietetics 4) GENIE’s online resources Katie Brown, Ed.D., RDN, LDN National Education Director 5) How GENIE can be used Academy of Nutrition and Dietetics Foundation The GENIE Team Rosa K Hand, MS, RDN, LD 1 Senior Manager , Dietetics Practice Based Research Network Why was GENIE created? Jenica K Abram, MPH, RDN, LDN 1 ConAgra Foods Foundation Nutrition Education Research Fellow Katie Brown EdD, RDN 2 Nutrition education is National Education Director important Paula J Ziegler , PhD, RDN, CFCS 1 Senior Director , Research and Evidence Analysis What does ‘good’ look like? J. Scott Parrott, PhD 3 Associate Professor Alison L Steiber , PhD, RDN 1 Chief Science Officer 1 Academy of Nutrition and Dietetics, 2 Academy of Nutrition and Dietetics Foundation, 3 Rutgers University 3/18/2014 1

3/18/2014 Academy Fellowships GENIE fellowship Provides opportunities: • First time this model has been used • Work solely on one project • Received 28 applications • Mentorship by Academy staff • Has resulted in 3 accepted conference • Meet experts in the area sessions, 2 pending manuscript submissions, • Publish and present results of work and 1 pending abstract submission Goal is a mutually beneficial experience • Provided proof of concept for the fellowship model Fellow is selected by Academy staff based on application and interview • Unique opportunity for collaboration between Foundation and Research team Background What is GENIE? Who should use GENIE? • A simple-to-use online checklist • Nutrition education program planners • A rigorously validated tool for designing, modifying or • Program reviewers comparing nutrition and funders education programs • A tool kit of resources for program planners Study Flow Chart • Identify thought leaders to participate in testing (n=10). How was GENIE created? • Identify proposals to be reviewed (n=16). Proposals sent to thought leaders for review and evaluation without knowledge of the new tool. 3 Step Process • Expert Panel Meeting 1. Establish • Day One-Expert panel to come to consensus on quality of each proposal (criterion Criteria development • establishment) Face and and expert panel • Day Two- Expert panel to determine essentiality and clarity of each element of GENIE (Face and Content Validity Content validity). • Reviewer testing Systematic literature • review 2

3/18/2014 Study Flow Chart Results Reviewers agreed with experts • Develop data collection tool and procedures • Strong relationship between expert and reviewer for evaluating proposals, based on thought leader group input. score groups • Recruit 13 reviewers from Academy's Dietetics Practice Based Research Network Reviewers agreed with each other (DPBRN) 2. Criterion • Good agreement between reviewers across • Training in person at Academy Headquarters; completion of three calibration proposals Validity and Inter- proposals and across categories • Assessment of 12 proposals completed by 13 rater Reliability reviewers Reviewer characteristics had no influence • Differences among reviewers accounted for a low proportion of differences in proposal scores Conclusions GENIE Format GENIE performs well across reviewers and across various proposals- can be applied to a GENIE Categories 1) PROGRAM DESCRIPTION AND IMPORTANCE variety of programs 2) PROGRAM GOAL 3) PROGRAM FRAMEWORK 4) PROGRAM SETTING, RECRUITMENT AND RETENTION PLAN Trained reviewer GENIE scores agreed with 5) INSTRUCTIONAL METHODS expert panel assessments- high degree of 6) PROGRAM CONTENT 7) PROGRAM MATERIALS validity 8) EVALUATION 9) SUSTAINABILITY Trained reviewers agreed with each other- high degree of reliability sm.eatright.org/GENIE The GENIE Checklist 3

3/18/2014 GENIE Criteria Checklist Features GENIE checklist, con’t . GENIE checklist, con’t . GENIE checklist, con’t . GENIE checklist, con’t . 4

3/18/2014 GENIE checklist, con’t . Checklist Features GENIE Score Page GENIE Score Page GENIE Score Page This seems like a lot of work… 5

3/18/2014 In one POS intervention, which factor was a 3. PROGRAM FRAMEWORK better predictor of people using the information? 3.1 The proposed program uses research or best practice examples to show how a model or A) Participants needed to like the information framework integrates with the program goal. 6. PROGRAM CONTENT B) Participants needed to understand the 6.2 The proposed program content is based on best information practice examples or related research with citations from relevant research or government/health society guidance. Participants needed to like the information “… information characteristics (e.g. display size, “This finding highlights the color scheme), which are key determinants need for communication of consumers’ attention to nutrition efforts and research to move beyond a focus on information and liking of the information, “understanding of nutrition may offer a window of opportunity to information” and to improve the effectiveness of nutrition emphasize more the liking information in terms of targeted dietary and attractiveness of change.” information formats.” Hoefkens, Christine, Zuzanna Pieniak, John Van Camp, and Wim Verbeke. "Explaining the effects of a point- of-purchase nutrition Hoefkens, Christine, Zuzanna Pieniak, John Van Camp, and Wim Verbeke. "Explaining the effects of a point- information intervention in university canteens: a structural equation modeling analysis." International of-purchase nutrition information intervention in university canteens: a structural equation modeling analysis." Journal of Behavioral International Journal of Behavioral Nutrition and Physical Activity 9.1 (2012): 111. Nutrition and Physical Activity 9.1 (2012): 111. What can we learn from effective advertising 7. PROGRAM MATERIALS models, such as AIDA (Attention, Interest, Desire, 7.1 The program cites and explains that the materials Action)? have social and cultural relevance including language, reading level, food likes/dislikes, What can we learn from behavior change models, household status, food/diet needs, interests, such as the Transtheoretical Model? age/development stage matched, learning style and/or format. How can we apply evaluations from other EVALUATION interventions into your design? 8.3 The proposed program's evaluation method is cited, based on related research, best practice or What can we learn from other types of POS includes pilot testing to support its use. interventions? 6

3/18/2014 Resource Kit The Effectiveness of Interventions to Increase Screen shot Resource Kit tab Physical Activity A Systematic Review Signs were effective for both obese and non-obese people, but greater for those who were obese; Among obese people, linking stair use to the potential for weight loss was more effective than signs for general health benefits; Effects on African Americans was mixed; This type of intervention is effective, when adapting the messages for the population. Hoefkens, Christine, Zuzanna Pieniak, John Van Camp, and Wim Verbeke. "Explaining the effects of a point-of-purchase nutrition-information intervention in university canteens: a structural equation modelling analysis." International Journal of Behavioral Nutrition and Physical Activity 9.1 (2012): 111. Videos 39 Tutorials 42 7

3/18/2014 Additional Resources GENIE Resources Sample Proposals 45 First Draft- Page 2 Scoring Benchmarks 8

3/18/2014 Criterion 1.1 Example First Draft Coded Color Coding Color Coding Revised Drafts How GENIE can be used Your hospital wants to start a nutrition outreach program in the local middle school. It doesn’t make sense for you to design a program from scratch, so you decide to select an existing curriculum to implement. 9