Really Work? Empirical Evidence from Taiwan Thursday, February 21, - PowerPoint PPT Presentation

Does Research Ethics Education (REE) Really Work? Empirical Evidence from Taiwan Thursday, February 21, 2017 Karen, Yuan-Hsuan Lee, Ph.D. Assistant Professor, National Tsing Hua University Chien Chou, Ph.D. Chair Professor, National Chiao Tung University 1

Research Background • Research integrity is an essential part of research (Godecharle et al. 2014; Horbach & Halffman, 2016; Steneck, 2006). • Scandals of research misconduct strengthen the requirement of research ethics education for stakeholders of research (Anderson et al., 2012). 2 Picture retrieved from REE available at https://ethics.nctu.edu.tw

Research Objective & Question Objective: • This study aimed to investigate the effectiveness of the Online Research Ethics Education program (REE) in Taiwan regarding students’ comprehension as well as their cognitive and behavioral attitude in research ethics. Research aims: • Assess students’ research ethics comprehension through the development and validation of the Online REE item bank • Evaluate students cognitive and behavioral attitudes in research ethics 3

Special Features of the online REE program • Anchored instruction • Case studies • Interactivity • Formative assessment & Instant feedback 4

Study 1 Study1: Development and validation of the REE Item Bank Basic concepts in research ethics Research ethics test Ethical considerations in the research Assessment procedure and Linking: Items reviewed by Item Balanced Protection of content and test Bank Incomplete research subjects development experts. Block Publication and Design authorship (van der Tested on the calibration Conflict of Linden, 2004) sample (N=566) interest Statistical procedures 3-PL Item Response Models applied to examine Classical Test Theory item quality 5

6

Study 1 Study 1: Development and Validity of the REE Item Bank Participants • 1325 graduate students recruited from universities located in the northern, middle, southern, and eastern parts of Taiwan participated in the study. Group 1: Online posttest Online REE course (6hr) (N=730) posttest Control (N=595) • Interviews with the learners: • 30 minutes of semi-structured interviews on 11 participants form Group 1. 8

Study 1 Differences in Research Ethics Comprehension Online Control (n=730) (n=595) Scales p t M SD M SD Comprehension 71.84 17.88 48.18 15.61 25.3 <.001 (scaled score via linking) 9

Study 1 Result: Interviews with the Learners • Awareness of the consequence of misconduct: • “I learned the Korean research ethics scandal from REE, and felt pity that a man with high academic reputation would fail due to his research misconduct.” (S04) • Perceived confidence in avoiding plagiarism • “When you use the literature, you know how to be careful to avoid plagiarism and how to rewrite it to conform to the regulations.” (S05) • Metacognition in learning from online REE program • “I like the videos, and also the tests. After finishing the test, I can know what I still did not understand.” (S09) 10

Study 2 Study 2 ： Assessment of cognitive and behavioral attitudes Participants • 1268 graduate students recruited from universities located in the northern, middle, southern, and eastern parts of Taiwan participated in the study. Online REE course(6hr) posttest Group 1: Online (N=566) Online REE course Group 2: Blended posttest + face2face Instruction (N=501) Control (N=201) posttest 11

Study 2 Study 2 Instrument • The cognitive aspects measure students’ perception toward moral judgments, e.g., • I think every researcher has the duty and obligation to blow the whistle if he/she knows of others’ research misconduct. (1: strong disagreement – 5: strong agreement) • The behavioral aspects measure students’ inclination to fulfill a moral behavior, e.g., • I will be the whistleblower if I know of others’ research misconduct. (1: Not at all like me– 5: Very much like me) 12

Study 1 Differe Diff erences in C ces in Cogni ognitiv tive and and Be Behavio avioral ral Atti Attitud tude Blended Online Control (n=566) (n=501) (n=201) Scales F p Post hoc M SD M SD M SD 3.94 0.50 4.06 0.34 3.92 0.38 11.313 <.001 Cognitive Online > Integrated, Control Behavioral 4.10 0.55 4.25 0.42 4.04 0.41 16.940 <.001 13

Summary • The online REE is valid and reliable in assessing students performance. • Students who took the online REE courses exhibited higher scores in their 1. REE comprehension 2. Cognitive attitude 3. Behavioral attitude • Qualitative interviews showed that the courses enhanced students’ 1. Awareness of the consequence of misconduct in research, 2. Perceived academic writing skill to avoid plagiarism, and 3. Metacognition in the course content. 14

Conclusion • The REE program is effective in promoting students’ comprehension as well as their cognitive and behavioral attitudes toward research ethics. • Moreover, the nature of this online-based program has the affordance to accommodate a massive number of learners. 15

References • Anderson, E. E., Solomon, S., Heitman, E., DuBois, J. M., Fisher, C. B., Kost, R. G., … Ross, L. F. (2012). Research ethics education for community-engaged research: A review and research agenda. Journal of Empirical Research on Human Research Ethics: An International Journal , 7 (2), 3–19. https://doi.org/10.1525/jer.2012.7.2.3 • Godecharle, S., Nemery, B., & Dierickx, K. (2014). Heterogeneity in European research integrity guidance: Relying on values or norms? Journal of Empirical Research on Human Research Ethics,9(3), 79–90. doi:10.1177/1556264614540594. • Horbach, S. P. J. M., & Halffman, W. (2016). Promoting virtue or punishing fraud: Mapping contrasts in the language of “scientific integrity.” Science and Engineering Ethics . https://doi.org/10.1007/s11948-016-9858-y • Steneck, N. H. (2006). Fostering integrity in research: Definitions, current Knowledge, and future directions. Science and Engineering Ethics , 12 , 53–74. • van der Linden, W. J. (2004). Optimizing balanced incomplete block designs for educational assessments. Applied Psychological Measurement , 28 (5), 317–331. https://doi.org/10.1177/0146621604264870 16

Thank you for listening! 17

Students’ satisfaction toward the online REE program Learning Satisfaction Online Blended Scales (N=1539) (N=237) t p M SD M SD Difficulty of the -0.194 4.07 0.736 4.08 0.740 0.423 learning material Usability of the 4.03 0.786 4.04 0.744 -0.191 0.424 instructional content Interface design 3.74 0.830 3.75 0.804 -0.177 0.430 Interactivity 3.72 0.767 3.76 0.729 -0.781 0.217 Aids in  結果顯示： 3.71 0.782 3.76 0.767 -0.932 0.176 comprehension 18 Note ： 1. Strongly disagree, 2. Disagree, 3. Average, 4. Agree, 5. Strongly agree