Department of Psychology and Philosophy Assessment Report AY2017-18 - PowerPoint PPT Presentation

Department of Psychology and Philosophy Assessment Report AY2017-18

Assessment Procedures 2017-18 • According to the 5 year assessment plan filed in Fall 2017, Goal 2 –Interpret, design, and conduct basic psychological research was scheduled to be assessed. Assessment of this goal required the development of rubrics for and assessment of artifacts from the capstone courses AY2017-18. • Due to Work to Rule and failure to reach a contract agreement, the department decided to assess Goal 1-Describe key concepts, principles, and overarching themes in psychology. This goal was originally scheduled for assessment in AY 2018- 19 but was moved up due to the ease of assessment with the ACAT Comprehensive Content measure commercially available. The department agreed that administering this test would not violate work to rule agreement and the review could be completed during the summer months with additional compensation for summer work. • The ACAT allows for selection of specific content domains to be assessed. The empirical research domain was selected as a direct measure for Goal 1. The additional domains of Development, Social, and Learning and Cognition were selected to provide information on knowledge base in 3 of 4 Content Domain areas within the Psychology major at FSU. • All capstone instructors agreed to administer the exam during the last week of April or the first week of May 2018. • The department decided on providing an incentive for completing the task. The assessment committee obtained incentive choices of : $20 University Bookstore Credit, Two Cinema Movie Passes, or $20 in vouchers for taxi services. • The cost of the ACAT was $23.50 per student for a total cost, including shipping, of $2,171.00. A total of 85 exams were ordered for the 85 non-overlapping student enrollment in the capstone courses. A total of 71 students completed the exam and the unused forms were returned .

ACAT Assessment Data Administered to Capstone courses during the last week of April and first week of May 2018 total possible 105, Sample completing=71. • History and Systems (n=20) • Seminar: Humor (n=20) • Seminar: Prejudice (n=20) • Thesis (n=18, n=10) • Internship (n=17) • Students provided incentive in form of choice of $20.00 Bookstore Credit, two cinema passes, or $20.00 in taxi fee. Evenly divided between bookstore and cinema passes, no taxi fee chosen. • 71 students completed the exam and chose NOT to complete the alternative assignment in order to receive the incentive.

ACAT 2018 FSU Psychology Capstone Students Gender (N=71) 70 67 60 Number of Students 50 40 30 20 10 4 0 Women Men

ACAT 2018 FSU Psychology Capstone Students Native versus Transfer (N=71) 60 51 50 Number of Students 40 30 20 20 10 0 Native Transfer

ACAT 2018 FSU Psychology Capstone Students Self Report GPA (N=71) Self Report GPA Ranges Overall Major 35 32 30 30 27 Number of Students 25 21 20 15 13 9 10 5 2 2 0 2.1-2.5 2.6-3.0 3.1-3.5 3.6-4.0

ACAT 2018 Overall Score (N=71) FSU Psychology Compared to National Sample Scores by Percentile 29.6 FSU National 30 23.9 25 20 18.4 18.3 17.7 16.1 Percent 15 13.3 11.7 10 8.5 8.5 8.1 7 6.3 4.3 4.1 5 2.8 1.4 0 0 0--4 4--11 11--23 23--40 40--60 60--77 77--89 89--96 96--100

ACAT 2018 FSU Psychology Capstone Students Average Percentile by Content Areas (N=71) 100 90 80 70 Percentile 60 50 39 40 32 31 31 27 30 20 10 0 Overall Experimental Developmental Learning and Cognition Social

P&P Dept. Supplemental Survey Classification (N=80)* 60 *Numbers varied from the ACAT and P&P Dept. Survey 56 50 Number of Students 40 30 21 20 10 1 0 Sophomore Junior Senior

P&P Dept. Supplemental Survey Course in the Biological Domain (N=80) 60 50 Number of Students 39 40 37 30 20 11 10 6 0 Biopsych Health S&P NONE

P&P Dept. Supplemental Survey Course in the Developmental Domain (N=80) 60 55 50 Number of Students 40 30 24 23 20 10 0 Child Adolescent Adult

P&P Dept. Supplemental Survey Course in the Learning/Cog Domain (N=80) 60 50 Number of Students 40 37 34 30 22 20 10 1 0 Learning Cognitive BX Mod NONE

P&P Dept. Supplemental Survey Course in the Sociocultural Domain (N=80) 60 50 Number of Students 40 32 31 30 20 20 16 10 2 0 Personality Social Cultural Women NONE

P&P Dept. Supplemental Survey Number of Courses taken in the Biological Domain (N=80) 70 64 60 Number of Students 50 40 30 20 10 7 6 3 0 NONE One Course Two Courses Three Courses

P&P Dept. Supplemental Survey Number of Courses taken in the Developmental Domain (N=80) 70 60 60 Number of Students 50 40 30 20 16 10 4 0 One Course Two Courses Three Courses

P&P Dept. Supplemental Survey Number of Courses taken in the Learning/Cog Domain (N=80) 70 64 60 Number of Students 50 40 30 20 15 10 1 0 NONE One Course Two Courses

P&P Dept. Supplemental Survey Number of Courses taken in the Sociocultural Domain (N=80) 70 60 56 Number of Students 50 40 30 18 20 10 2 2 2 0 NONE One Course Two Courses Three Courses Four Courses

P&P Dept. Supplemental Survey Number of Students Taking Each Domain Course (N=80) 60 55 50 Number of Students 39 40 37 37 34 32 31 30 24 23 22 20 20 16 11 10 0 Bio Women Cultural BX Adol Adult Social Person Learn Health Cog S&P Child Course Names

Issues and Concerns for Second Administration of ACAT and Student Survey: • Inability to link FSU student survey information to ACAT data due to inconsistency or missing ID numbers on FSU survey. • Discrepancy between student reported course enrollment on ACAT and FSU survey. Course names not consistent and students may have not correctly identified similar course listings. • FSU survey formatting needs to be changed for ease of analysis and comparison. • Collect data on 300 level courses taken to date. • Pre-assign student ID numbers in future. • Scheduled testing times instead of class administered if possible. • Limit to only juniors/seniors • Administer in fall to obtain info on students graduating in Fall.

Key Points: • Assessed Goal 1: Describe key concepts, principles, and overarching themes in psychology. Assessment committee decided to assess in four content areas: Developmental, Experimental, Social, and Learning and Cognition. • Scores on the ACAT revealed that compared to a national sample, our students would fall in the range of a low in Developmental of 27% to a high of 39% in the Experimental domains. • Examination of the number of courses students reported taking in the different departmental domains offered by the department yields some insight into the percentages obtained.

Key Points: ACAT Experimental Design and Course Experience • All 71 students completing the ACAT had successfully completed Research Methods 1 and Research Methods 2 at FSU. • The ACAT score students achieving at or below the 39% level for this domain may reflect the performance of students completing the course requirements at FSU. • A grade of C- or better is required to progress through the sequence and on to the capstones. This grade reflects written performance and statistical performance as well as an understanding of experimental design. The sequence is heavily biased toward writing and understanding statistics while the ACAT references a good deal more advanced experimental design concepts. • The ACAT score may indeed accurately reflect the “content” emphasized in these courses and therefore may be more satisfactory than first glance would suggest.

Key Points: ACAT Developmental Domain and Course Experience • While Developmental domain scores were only in the 27 th percentile, students reported taking the most courses (123) overall in that domain. • However, the child development enrollments of 55 far exceeded the enrollments in adolescent and adult development. • A total of 60 students took only one course in this domain suggesting that the bulk of exposure to developmental content would have come primarily from child development. • The domain test of the ACAT is evenly divided over the three areas of development and therefore may not accurately reflect the “developmental” perspectives that are being highlighted in our department.

Key Points-ACAT Scores on Human Learning and Cognition Related to Course Experience • The Score on the Human Learning and Cognition ACAT results indicates students scoring at or below the 31% level. • When examining the courses taken in this domain Learning and Cognitive enrollments were approximately equal with a little over one-third of the students taking each course and 64 completing only one course in the domain. • Given this discrepancy in course content exposure the results on the ACAT may reflect an absence of students completing the content in the curriculum.

Key Points: ACAT Scores on Social Domain related to Course Experience • About 1/3 of the students reported taking Social and 1/3 reported taking Personality. Slightly less than 1/3 reported taking either Cultural or Psychology of Women. • However, a total of 56 students reported taking only one course in this domain. The exposure to the concepts on the Social ACAT domain test content may be limited in those students not having taken Social.