Mysteries of the Real Estate Licensing Exam Mysteries of the Real Estate Licensing Exam REEA Annual Conference REEA Annual Conference June 2011 June 2011 Tadas Dabsys, Vice President Tadas Dabsys, Vice President PS I S ervices LLC PS I S ervices LLC
ARELLO-Accredited Vendors � Applied Measurement Professionals � http://www.goamp.com � Pearson VUE � http://www.pearsonvue.com � PSI Services LLC � http://psionline.com 2 2
Benefits of ARELLO-Accreditation � Independent Evaluation � Confidence and Protection � Defensible Standards � Reasonable Consistency in Quality � Facilitates License Recognition � www.ARELLO.org (ARELLO Guidelines for Accreditation) 3 3
Stakeholders � Applicants � Public � Regulators � Educators � Publishers � Industry 4 4
Licensing Exam Purpose � Regulation � Public Protection � Competency Assurance � Minimum Competence 5 5
Licensing Exam Development � Validity � Reliability � Fairness � Legal Defensibility � Standards � Standards for Educational and Psychological Testing � (AERA, APA, & NCME, 1999) 6 6
Credibility � Subject Matter Experts � Qualifications � Experience � Training in Job Analysis � Test Development Experts � Psychometricians � Item Writers 7 7
Job Analysis � Job Analysis Survey � Procedure � Content Development � Format Development � Sampling – Representative Sample � Target Groups � Sources for Addresses/Contact Information 8 8
Analysis of the Survey Results � Demographic Information � Professional Activities � Knowledge � Rating and Respective Response Scales � Summary Statistics � Identification of Qualifying Professional Activities/Knowledge 9 9
Content Specifications � Development of Content Specifications � Establishing Content Base for Inclusion in Exams � Linking Knowledge Areas to Professional Activities � Development of Operational Definitions � Test Outline and Content Weighting 10 10
Linking Back to the Job Analysis � Activities and Tasks � Subject/Content Area of the Task � Knowledge Statement (KSAs) � Knowledge � Skills � Abilities 11 11
Item Development � Based on Job Analysis and Content Specifications � Written by Professionals � Reviewed by Subject Matter Experts � Checked Against References � Sensitivity Review � Continual Review and Updating � Quality Assurance 12 12
Basic Item Writing Principles � One Correct Answer � Relevant and Appropriate � Realistic � Important to Related Task � Limit Use of Negative Words � Clear and Concise � No Clues that Give Away the Answer � Entry-Level Reading Level 13 13
Basic Item Writing Principles � Response Options: � Fit the Stem Logic Grammatically � Equal in Length � Unique Meaning � Plausible - yet definitely incorrect � Avoid “All the Above” and “None of the Above” � Avoid Multiple Options 14 14
Balancing Cognitive Levels Recall: Able to state the definition of a “latent defect .” Application: Applies knowledge of the definition of “latent defect” by recognizing and classifying certain described property conditions as meeting the definition of a latent defect. Analysis: Applies knowledge of the definition by recognizing certain described property conditions as meeting the definition and interpreting the information to determine what action should be made. 15 15
Sample Items Recall - Terminology Item A land contract is also known as 1. an option. * 2. a contract for deed. 3. an executory contract. 4. an agency contract. Application - Practical Item A young couple have enough income to make monthly house payments easily, but they do not have a sufficient down payment to meet lender requirements. Which would their agent MOST LIKELY suggest? * 1. A land contract. 2. An assigned mortgage. 3. A reverse annuity mortgage. 4. A deed in lieu of foreclosure . 16 16
Quality Assurance � Test Question Review � Examination Review � Monitoring Web Sites � Updating References � REEA and ARELLO � Satisfaction Surveys 17 17
Item Statistics � p-value: Percentage of candidates in a sample who answer an item correctly � pbis: Correlation within a sample between performance on an item and performance on the overall exam � option stats: Percentage of candidates who answer each option � option pbis: Correlation between performance on the options and the overall exam 18 18
Expertise in Content and Test Development � Producing Professional Results Questionnaire Committee Committee Review Content Domain Job Analysis Empirical Results Test Content Relevancy Specifications Form Ratings Item Development Cutoff Score 19 19
Variables Affecting Pass Rates � Examination � Education � Candidate – Study Habits, Health � Reference Material � Environment � Others 20 20
Reporting � Candidate Score Reports � Diagnostics � State Reports � Partner Site � School Reports � Topic Level for Schools and State 21 21
Tips for Increased Potential for Success � Internet � Candidate Information Bulletin � Content Outline � Reference Materials � Comments – During Exam ONLY � Security Agreement – Confidentiality 22 22
Security Agreement � I will not give or receive assistance while taking the test, including the use of unauthorized notes. � I will maintain the confidentiality of the test. � I will not have in my possession a cell phone, pager, or other unauthorized materials � I understand that violating the confidential nature of the licensing test can result in severe civil or criminal penalties, invalidation of test scores, reports to the Authorized Agency 23 23
Forensic Data Analysis Suspicious Candidate Activity � Test for significant differences in pass rates � Better performance on harder items � High scores in low test times � Suspicious improvement in scores 24 24
� Comments ! � Questions ? � Thank You 25
Recommend
More recommend
