ONLINE (PHASE 2) Kijoo Ko Korean Language Program, UC Berkeley - PowerPoint PPT Presentation

ONLINE (PHASE 2) Kijoo Ko Korean Language Program, UC Berkeley

▪ Korean Placement Test ▪ To assign students to the right level in the Korean language program (Brown, 1996; Rapi & Miconi, 2014) ▪ Computer Adaptive Language Test (CALT) ▪ Computer- tailored test to the student’s level (Meunier, 1994; Long et als., 2018)

(Meunier, 1994; Long et als., 2018) ▪ Efficiency ▪ Developers: can easily upload/update test items ▪ Students: can take the test at any place/time, can get the result immediately ▪ Teachers: can save time for making/administrating/grading tests, can analyze data easily ▪ Authenticity ▪ Can use multimodal inputs ▪ Flexibility ▪ Individualized test ▪ Self-paced, less boring/frustrating

▪ Levels ▪ 5 levels: Elementary, Intermediate, Advanced, 4 th , 5 th year Korean ▪ Tracks ▪ Heritage track: K1AX/BX, K10AX/BX, K100AX/BX ▪ Non-Heritage track: K1A/B, K10A/B, K100A/B ▪ Students ▪ 300-350 per semester ▪ Instructors ▪ 9 full-time lecturers (Spring, 2019), 1 tutor

Current: Time-consuming, labor-intensive test managed by individual KLP instructors Goal: To provide a reliable and practical measure of students’ level of Korean as one factor in assessing the appropriate course for a student.

(PHASE 1) ▪ When ▪ Fall, 2017 ▪ Who ▪ Kijoo Ko & Seung-Eun Chang ▪ How ▪ Support by CKS, BLC (BOLT) ▪ What ▪ Developed Item Set #1: 450 test items, 9 levels, 50 questions for each level ▪ Administered 4 Norming tests: Entrance & Exit tests in Spring, 2018 & Fall 2018

(PHASE 2) ▪ When ▪ Spring, 2019 ▪ Who ▪ Kijoo Ko ▪ How ▪ Support by BLC ▪ What ▪ Analyzed the norming test results ▪ Revised Item Set #1, Developed Item Set #2

(PHASE 2) ▪ Literature review ▪ Chinese & Japanese placement tests at UCB ▪ Korean placement tests at other universities ▪ Other official Korean tests ▪ Analysis of the norming test results ▪ Revision for Items Set #1 ▪ Addition for Items Set #2

▪ Why ▪ Do students really know the test item when it is taught? ▪ If not, we need to revise (question or level or distractor) or delete ▪ Who ▪ 598 students taking a Korean class at UCB in 2018 ▪ When ▪ Entrance test: beginning of the semester ▪ Exit test: end of the semester ▪ What ▪ Level X: previous-level contents ▪ Level Y: next-level contents

(Brown, 1996; Lee, 2000) ▪ Item Facility (IF) analysis ▪ IF score: (# of students answered correctly)/(# of students answered) ▪ 0.25 would be the result for guessing ▪ Item Discrimination (ID) analysis ▪ ID score: Difference between the IF score of students below level and the IF score of students above the level of the question For example, Group – L IF score 0.3 ID score 0.5 Group +L IF score 0.8 ID score < 0.2 : not good 0.2 < ID score < 0.4 : ok ID score > 0.4 : great

Q QL SL0 SL1 ID score Q QL SL0 SL1 ID score Q QL SL1 SL2 ID score Q QL SL2 SL3 ID score Q QL SL3 SL4 ID score 5004 1 0.95 0.93 -0.03 5145 2 0.57 0.53 -0.04 5118 3 0.62 0.5 -0.12 5175 4 0.34 0.24 -0.1 5183 5 0.2 0 -0.2 5061 1 0.84 0.85 0.01 5008 2 0.93 0.9 -0.03 5098 3 0.91 0.83 -0.08 5102 4 0.24 0.2 -0.04 5188 5 0.38 0.25 -0.13 Levels 1-5 5042 1 0.55 0.57 0.02 5010 2 0.93 0.91 -0.02 5015 3 0.89 0.88 -0.01 5106 4 0.8 0.77 -0.03 5114 5 0.94 0.83 -0.11 5047 1 0.91 0.93 0.02 5146 2 0.64 0.64 0 5014 3 0.83 0.85 0.02 5224 4 0.56 0.59 0.03 5189 5 0.51 0.42 -0.09 5057 1 0.89 0.92 0.03 5150 2 0.69 0.7 0.01 5016 3 0.76 0.79 0.03 5168 4 0.17 0.2 0.03 5254 5 0.63 0.58 -0.05 5060 1 0.89 0.92 0.04 5011 2 0.86 0.88 0.02 5221 3 0.25 0.28 0.03 5103 4 0.87 0.9 0.03 5131 5 0.41 0.36 -0.05 5073 1 0.81 0.86 0.04 5089 2 0.79 0.82 0.03 5222 3 0.24 0.27 0.03 5025 4 0.33 0.37 0.04 5182 5 0.32 0.29 -0.03 5038 1 0.95 1.00 0.05 5009 2 0.86 0.89 0.03 5164 3 0.37 0.41 0.04 5026 4 0.86 0.9 0.04 5111 5 0.94 0.92 -0.02 5007 2 0.86 0.91 0.05 5017 3 0.75 0.83 0.08 5125 4 0.65 0.71 0.06 5184 5 0.41 0.42 0.01 5041 1 0.88 0.93 0.05 5037 1 0.94 1.00 0.06 5141 2 0.57 0.67 0.1 5099 3 0.72 0.8 0.08 5101 4 0.89 0.96 0.07 5028 5 0.54 0.58 0.04 5005 2 0.5 0.61 0.11 5154 3 0.57 0.65 0.08 5024 4 0.78 0.86 0.08 5029 5 0.88 0.92 0.04 5039 1 0.86 0.93 0.06 5062 1 0.86 0.92 0.06 5087 2 0.69 0.8 0.11 5092 3 0.77 0.87 0.1 5180 4 0.7 0.79 0.09 5108 5 0.96 1 0.04 5148 2 0.67 0.79 0.12 5206 3 0.47 0.57 0.1 5123 4 0.39 0.49 0.1 5032 5 0.69 0.77 0.08 5049 1 0.93 1.00 0.07 5035 1 0.93 1.00 0.07 5003 2 0.5 0.63 0.13 5207 3 0.29 0.39 0.1 5173 4 0.36 0.46 0.1 5033 5 0.84 0.92 0.08 5086 2 0.69 0.82 0.13 5093 3 0.79 0.9 0.11 5237 4 0.52 0.64 0.12 5192 5 0.64 0.73 0.09 5036 1 0.93 1.00 0.07 5048 1 0.92 1.00 0.08 5090 2 0.71 0.84 0.13 5120 3 0.41 0.52 0.11 5100 4 0.43 0.55 0.12 5027 5 0.9 1 0.1 5065 1 0.92 1.00 0.08 5006 2 0.6 0.74 0.14 5012 3 0.78 0.9 0.12 5019 4 0.85 0.98 0.13 5110 5 0.72 0.83 0.11 5081 1 0.70 0.79 0.09 5085 2 0.71 0.85 0.14 5218 3 0.33 0.46 0.13 5022 4 0.69 0.82 0.13 5239 5 0.72 0.83 0.11 5059 1 0.91 1.00 0.09 5088 2 0.77 0.93 0.16 5208 3 0.42 0.55 0.13 5169 4 0.26 0.39 0.13 5250 5 0.39 0.5 0.11 5091 2 0.57 0.73 0.16 5161 3 0.15 0.29 0.14 5104 4 0.75 0.91 0.16 5186 5 0.73 0.85 0.12 5034 1 0.91 1.00 0.09 5078 1 0.83 0.93 0.10 5197 2 0.71 0.87 0.16 5018 3 0.79 0.93 0.14 5228 4 0.29 0.45 0.16 5107 5 0.87 1 0.13 5204 2 0.69 0.85 0.16 5166 3 0.18 0.33 0.15 5124 4 0.21 0.38 0.17 5130 5 0.69 0.82 0.13 5072 1 0.89 1.00 0.11 5044 1 0.76 0.87 0.11 5194 2 0.57 0.75 0.18 5220 3 0.68 0.85 0.17 5235 4 0.6 0.77 0.17 5251 5 0.54 0.67 0.13 5199 2 0.54 0.73 0.19 5013 3 0.7 0.87 0.17 5225 4 0.32 0.5 0.18 5255 5 0.54 0.67 0.13 5040 1 0.88 1.00 0.12 5058 1 0.78 0.92 0.14 5151 2 0.57 0.76 0.19 5159 3 0.34 0.52 0.18 5236 4 0.18 0.36 0.18 5256 5 0.6 0.73 0.13 5143 2 0.62 0.83 0.21 5097 3 0.72 0.9 0.18 5105 4 0.75 0.94 0.19 5253 5 0.85 1 0.15 5043 1 0.86 1.00 0.14 5079 1 0.86 1.00 0.14 5140 2 0.69 0.91 0.22 5213 3 0.41 0.61 0.2 5176 4 0.19 0.4 0.21 5244 5 0.42 0.58 0.16 5136 2 0.54 0.77 0.23 5094 3 0.73 0.93 0.2 5020 4 0.69 0.92 0.23 5243 5 0.57 0.75 0.18 5075 1 0.75 0.92 0.17 5053 1 0.61 0.79 0.17 5203 2 0.5 0.73 0.23 5223 3 0.1 0.31 0.21 5177 4 0.53 0.77 0.24 5112 5 0.72 0.92 0.2 5138 2 0.21 0.44 0.23 5155 3 0.24 0.46 0.22 5179 4 0.2 0.44 0.24 5187 5 0.71 0.92 0.21 5056 1 0.75 0.93 0.18 5084 2 0.64 0.89 0.25 5153 3 0.58 0.83 0.25 5167 4 0.18 0.43 0.25 5132 5 0.11 0.33 0.22 5063 1 0.73 0.92 0.19 5117 2 0.21 0.46 0.25 5156 3 0.3 0.55 0.25 5170 4 0.46 0.74 0.28 5193 5 0.57 0.83 0.26 5077 1 0.62 0.85 0.22 5200 2 0.57 0.82 0.25 5158 3 0.33 0.59 0.26 5021 4 0.64 0.94 0.3 5030 5 0.5 0.77 0.27 5080 1 0.77 1.00 0.23 5052 1 0.68 0.93 0.25 5202 2 0.5 0.75 0.25 5163 3 0.4 0.66 0.26 5232 4 0.53 0.83 0.3 5245 5 0.4 0.67 0.27 5149 2 0.21 0.48 0.27 5121 3 0.48 0.79 0.31 5227 4 0.24 0.56 0.32 5109 5 0.64 0.92 0.28 5070 1 0.68 0.93 0.25 5055 1 0.70 1.00 0.30 5198 2 0.38 0.65 0.27 5160 3 0.5 0.82 0.32 5238 4 0.35 0.68 0.33 5249 5 0.46 0.75 0.29 5139 2 0.64 0.92 0.28 5122 3 0.22 0.55 0.33 5174 4 0.55 0.89 0.34 5129 5 0.38 0.67 0.29 5050 1 0.54 0.85 0.31 5082 1 0.61 0.92 0.31 5144 2 0.46 0.75 0.29 5095 3 0.34 0.68 0.34 5126 4 0.52 0.87 0.35 5128 5 0.28 0.58 0.3 5134 2 0.46 0.76 0.3 5096 3 0.42 0.79 0.37 5231 4 0.32 0.67 0.35 5190 5 0.62 0.92 0.3 5074 1 0.53 0.85 0.32 5068 1 0.59 0.93 0.34 5205 2 0.46 0.78 0.32 5119 3 0.31 0.68 0.37 5023 4 0.6 0.96 0.36 5248 5 0.53 0.85 0.32 5142 2 0.21 0.53 0.32 5217 3 0.42 0.79 0.37 5127 4 0.36 0.73 0.37 5247 5 0.43 0.75 0.32 5067 1 0.50 0.85 0.35 5116 2 0.54 0.87 0.33 5209 3 0.33 0.71 0.38 5181 4 0.42 0.79 0.37 5242 5 0.59 0.92 0.33 5045 1 0.50 0.86 0.36 5196 2 0.54 0.89 0.35 5165 3 0.25 0.63 0.38 5178 4 0.09 0.47 0.38 5240 5 0.32 0.67 0.35 5064 1 0.56 0.93 0.37 5201 2 0.4 0.75 0.35 5219 3 0.35 0.73 0.38 5229 4 0.14 0.52 0.38 5031 5 0.63 1 0.37 5046 1 0.62 1.00 0.38 5133 2 0.54 0.91 0.37 5157 3 0.3 0.69 0.39 5233 4 0.37 0.77 0.4 5191 5 0.38 0.75 0.37 5051 1 0.54 0.92 0.38 5002 2 0.23 0.63 0.4 5214 3 0.38 0.77 0.39 5171 4 0.38 0.8 0.42 5185 5 0.52 0.92 0.4 5076 1 0.48 0.92 0.44 5147 2 0.15 0.58 0.43 5211 3 0.32 0.74 0.42 5230 4 0.18 0.63 0.45 5241 5 0.5 0.92 0.42 5066 1 0.47 0.92 0.45 5135 2 0.38 0.88 0.5 5152 3 0.51 0.93 0.42 5234 4 0.29 0.77 0.48 5246 5 0.3 0.77 0.47 5069 1 0.55 1.00 0.45 5115 2 0.07 0.6 0.53 5162 3 0.22 0.73 0.51 5226 4 0.23 0.77 0.54 5113 5 0.26 0.75 0.49 5054 1 0.47 0.93 0.46 5195 2 0.21 0.8 0.59 5212 3 0.31 0.88 0.57 5172 4 0.26 0.85 0.59 5252 5 0.43 0.92 0.49 5071 1 0.44 0.92 0.49