Global Perceptions on the Use of WeBWorK as an Online Tutor for - PowerPoint PPT Presentation

in CS Global Perceptions on the Use of WeBWorK as an Online Tutor for Computer Science Olly Gotel, Christelle Scharff, Andrew Wildenberg, Mamadou Bousso, Chim Bunthoeurn, Phal Des, Vidya Kulkarni, Srisupa Palakvangsa Na Ayudhya, Cheikh Sarr, Thanwadee Sunetnanta Presented by: Olly Gotel (ogotel@pace.edu) Pace University, New York, USA [FIE 2008]

in CS Outline • The Problem with CS and Teaching Programming … A Personal Perspective … • Systems for Automated Assessment of Programming Assignments • WeBWorK and WeBWorK-JAG • A Global Experiment - PERCEPTIONS • Conclusions

in CS Systems for Automated Assessment of Programming Assignments • Web-based systems to encourage practice (with feedback), and improve and reinforce students’ understanding of concepts • Types of questions – True / false, short answer, multiple-choice, programming • Grading programs – Correctness + quality + authenticity

in CS Not an exhaustive list! Some Existing Systems • BOSS (http://www.dcs.warwick.ac.uk/boss, Open Source, Computer Science) • CodeLab (http://www.turingscraft.com, Commercial, Computer Science) • CourseMarker (http://www.cs.nott.ac.uk/coursemarker, Commercial, Computer Science) • DevSquare (http://www.devsquare.com, Commercial, Computer Science) • Educosoft (https://www.educosoft.com/ecf, Commercial, Mathematics) • Gradiance (http://www.gradiance.com, Commercial, Computer Science) • JavaBat (http://javabat.com, Free, Computer Science) • MathXL (http://www.mathxl.com, Commercial, Mathematics) • MyCodeMate (http://www.mycodemate.com, Commercial , Computer Science ) • MyMathTest (http://www.mymathtest.com, Commercial, Mathematics) • OWL (http://owl.course.com, Commercial , Computer Science) • Quiz PACK (http://www.sis.pitt.edu/~taler/QuizPACK.html, Open Source, Computer Science) • Viope (http://www.viope.com, Commercial, Computer Science) • WebAssign (http://www.webassign.net, Commercial, Mathematics) • WebCAT (http://web-cat.cs.vt.edu, Open Source, Computer Science) • WeBWorK (http://webwork.rochester.edu and http://atlantis.seidenberg.pace.edu/~scharff/webwork, Open Source, Mathematics and Computer Science) • WileyPLUS (http://he-cda.wiley.com/WileyCDA/Section.rdr?id=101003, Commercial, Mathematics) • ASSYST, CFX, CodeWitz, Marmoset, Praktomat, RoboProf, SQL-Tutor, TRAKLA2…

in CS WeBWorK: About • http://webwork.rochester.edu • Project funded by NSF • Free, open-source and web-based • Automated problem delivery and grading • Initial development and applications in the fields of mathematics and physics • Currently in use at more than 60 colleges and universities (for maths)

in CS WeBWorK: Underneath • Problems are written in the Problem Generating macro language (PG) ‏ – Text, HTML, Latex, Perl • Underlying engine dedicated to dealing with mathematical formulae – x+1 = (x^2-1)/(x-1) = x+sin(x)^2+cos(x)^2 • Individualized and parameterized versions of problems

in CS WeBWorK: for Programming Fundamentals • Extension of WeBWorK for use in programming fundamentals - Pace and Cornell (since 2005) • True / false, short answer and multiple choice problems for Java, Python and SML • Evaluation of Java program fragments in real time by interfacing WeBWorK with JUnit [ www.junit.org ] – WeBWorK-JAG = WeBWorK + J ava A uto- G rader • http://atlantis.seidenberg.pace.edu/webwork2

in CS A Global Experiment • Global Perceptions on the Use of WeBWorK as an Online Tutor for Computer Science – Students and Professors – Perceived value and use (class size, integration) – Implications for uptake (US-centric) • 5 Country Experiment – 10 Java/C problems (common questions) – Time-boxed, unlimited attempts • Questionnaires – Students: Background info, technology access, time spent coding per week, opinions of use, preference for question types, etc – Professors: Background info, teaching style, class sizes, homework, time spent on tasks, contact hours, technology use in classroom, value of question types, etc

in CS 10 Royal University 8 Mahidol University 8 University of Delhi 10 Cornell College of Phnom Penh 8 University of Thies

in CS Problem Sets

in CS Problems

in CS Multiple-Choice Questions

in CS Fill-in-the-Blanks Questions

in CS You Can Build up Interesting / Typical Questions

in CS Randomize / Parameterize

in CS JAG - For Open Coding

in CS Typical Feedback - Correct

in CS Typical Feedback - Incorrect

in CS How Did They Do?

in CS … & With Respect to the Others?

in CS Results • Not designed to compare abilities! • Very few students got all the questions correct • Average number of attempts per question was 1 to 5 (except US where > 20) – Prior exposure – Driven to achieve 100% • Background data • Logistics - Internet

in CS Student Perceptions - Qs • Like best? Instant feedback • Like least? Clarity of questions/language • Suggestions? Live help-desk links • Most valuable type of questions? Open coding (MC in Senegal) • Easiest type of question to answer are least valuable • Why use? Practice • Use again? 72.5%

in CS Student Perceptions - Free-Form • “WebWorK is good but needs the user to have the Internet . So if we don't have internet we can't do it.” [Cambodia.] • “Though it was my first experience at WeBWorK, I found it quite simple to use and really encouraging to remind me about Java, as I haven't did any java practice for the last 3-4 months. Its interface was nice.” [India.] • “I liked the concept of WeBWorK. It is a useful tool for testing students’ level at programming.” [Senegal.] • “WeBWorK make it convenient to do the test by automatically checking the answer and responding back to the user.” [Thai.] • “It was a fun experience to go through the questions and find the answers. I enjoyed reviewing, and verifying that I know how to write code.” [US.] Implications? For uptake and future work?

in CS Professor Perceptions - Qs • All seek help/relief from compiling/marking homework • Prepared to invest time to learn -- how much? • Like most? Instant feedback to students so learn from mistakes • Most valuable question type? Open coding

in CS Professor Perceptions - Free-Form • “I think the problem should be clear - should [be] basic Java and OOP concept in Java. Internet was too slow to have all students complete the exercises.” [Cambodia.] • “I will be happy to see how it will test a large programming assignment with multiple classes.” [India.] • “Internet use which expands collaborations .” [Senegal.] • “…limited programming library ; expand problem library…” [US.] Implications? For uptake and future work?

in CS Conclusions • Supports an overloaded professor and can tide students over in-between classes • These systems open up opportunities for professors and students around the globe to transfer and share knowledge, and extend practice, but some local adaptations are necessary (Intranet, question formulation, integration) • Development of a novel pedagogy, encouraging students to contribute their own questions to the WeBWorK library, introducing them to crucial practices of software engineering • Aim to create a community of contributors to monitor quality, share work and extend the WeBWorK library

in CS Acknowledgements • NSF CCLI AI Grants “ Collaborative Research: Adapting and Extending WeBWorK for Use in the Computer Science Curriculum” #0511385 and #0511391 • Students: – The many computer science students in the US, Cambodia, India, Senegal and Thailand – Jacqueline Baldwin, Nathan Baur (JUnit extension) – Sophal Chiv (input of problems and help desk) – Eileen Crupi, Tabitha Estrellado (input of problems) – Allyson Ortiz, Veronica Portas (existing systems) – Yue Ma (user acceptance testing)