timing is everything what we can learn from survey
play

Timing is Everything: What we can learn from survey procrastinators - PowerPoint PPT Presentation

Mar 03, 2024 •160 likes •535 views

Timing is Everything: What we can learn from survey procrastinators Presented by: Lauren M. Conoscenti, Ph.D. Tufts University Office of Institutional Research & Evaluation (OIR&E) NEAIR Annual Conference, Newport, Rhode Island

  1. Timing is Everything: What we can learn from “survey procrastinators” Presented by: Lauren M. Conoscenti, Ph.D. Tufts University Office of Institutional Research & Evaluation (OIR&E) NEAIR Annual Conference, Newport, Rhode Island November 9-12, 2013

  2. Web-Based Surveys Tufts, like many institutions, relies heavily on online survey platforms to collect data. Online survey platforms have many advantages over paper-and-pencil surveys. Inexpensive Easy to use Improved distribution Bells-and-whistles Office of Institutional Research, November 13

  3. Web-Based Surveys A major advantage: busy respondents can complete surveys at their convenience. More time = better data? Comfy students = better data? Tech-savvy = better data? A major disadvantage: Low response rates. Office of Institutional Research, November 13

  4. Response Rates Response rates are going down… everywhere. (e.g. Jans & Roman, 2007) Low response rates are ok if sample is representative … …but most aren’t. Nonresponse Error: When survey responders differ from nonresponders in key ways, leading us to draw erroneous conclusions. Office of Institutional Research, November 13

  5. Why Don’t They Participate? Have we traded data quality in favor of ease and price? Why don’t students take web surveys? (Stay tuned, Tuesday at 10:30am) Office of Institutional Research, November 13

  6. Why Don’t They Participate? Online non- response might be because… Unread email routed to spam folder Student temporarily too busy (e.g. exams) Student forgot Office of Institutional Research, November 13

  7. Why Don’t They Participate? Online non- response might be because… Unread email routed to spam folder Student temporarily too busy (e.g. exams) Student forgot …Passive nonresponders. We assume they want to take the survey. Office of Institutional Research, November 13

  8. Why Don’t They Participate? Online non- response might be because… Student doesn’t want to participate …Active nonresponders We hope they are a small fraction of our sample! Office of Institutional Research, November 13

  9. Survey Reminders We send reminders to intervene in passive nonresponding … …and maybe a little active nonresponding, too. We assume people who complete a survey after the reminder are similar to those who completed it before the reminder. Is that a valid assumption? Office of Institutional Research, November 13

  10. Late Responders Non-responders are different from responders Male (McCabe, et al., 2002; Porter & Whitcomb, 2005b) Nonwhite (Couper, Traugott, & Lamias, 2001) Lower GPA (Porter, et al., 2004; Woosley, 2005) Could late responders – or “procrastinators” be different, too? Office of Institutional Research, November 13

  11. Late Responders Past research on this topic has found differences… But most is based on mail surveys… …and findings are mixed. Some evidence that late responders display more problem behavior. Office of Institutional Research, November 13

  12. Late Responders Who is a survey procrastinator? Most survey responses are submitted within 12-24 hours of an email. Survey A: 89.5% within 24 hours of email Survey B: 82.8% Survey C: 87.6% Survey D: 87% Office of Institutional Research, November 13

  13. Late Responders We send reminders to help the passive nonresponders Procrastinators: respondents who wait until a reminder is issued to start a survey. Office of Institutional Research, November 13

  14. Research Questions Are procrastinators different from “regular” responders? Demographic, academic, and survey variables How does the survey incentive impact procrastination (and the procrastinators)? Office of Institutional Research, November 13

  15. Research Method Analyzed data from two undergraduate surveys at Tufts Tufts University… Private Research University – Very high activity Entering class size ~1300 Competitive admissions 4-year Liberal Arts & Engineering undergraduate schools Office of Institutional Research, November 13

  16. The Surveys Sophomore Survey Administered sophomore spring 2012 Survey: Highly incentivized Initial email & 2 reminders Advising, majors, student life, services, civic engagement 2013 Survey Low incentiveInitial email & 6 reminders Added “flourishing scale”, removed advising questions Office of Institutional Research, November 13

  17. The Students 2012: 1073 Liberal Arts sophomores 93.3% Response Rate (LA class size: 1151) 54.5% female Not different from population GPA not significantly different 2013: 718 Liberal Arts sophomores 64% Response Rate (LA class size: 1123) 61% female Different from population Mean GPA higher than nonresponders Office of Institutional Research, November 13

  18. Percent of Class Nonresponding 100% 80% 2013 (Low Incentive) 60% 36.1% 40% 2012 (High Incentive) 20% 6.7% 0% Start R1 R2 R3 R4 R5 R6 End Office of Institutional Research, November 13

  19. Percent of Procrastinators 100% 80% 60% 40% 2013 (Low Incentive) 20% 2012 (High Incentive) 0% Start R1 R2 R3 R4 R5 R6 End Office of Institutional Research, November 13

  20. Results 2012 (High Incentive) Started before a reminder issued: 58.7% Procrastinators (41.3%): More males ( t = 4.52, p <.01) More nonwhite students ( t = 3.80, p <.01) Lower GPA ( t = 7.35, p <.001) Office of Institutional Research, November 13

  21. Results 2012 (High Incentive) Procrastinators: Reported more difficulty choosing a major ( t = 2.06, p <.05) Less likely to make an appointment to see academic advisor… ( t = 2.22, p <.05) …and more likely to drop in at the last minute. ( t = 6.81, p <.01) …and more likely to feel the time spent with their advisor was inadequate. ( t = 2.21, p <.05) Office of Institutional Research, November 13

  22. Results 2012 (High Incentive) Procrastinators: Less likely to participate in community service ( t = 2.99, p <.01) Less likely to feel comfortable asking faculty for help Rec letters ( t = 3.13, p <.01) Academic advising ( t = 2.13, p <.01) Office of Institutional Research, November 13

  23. Results 2012 (High Incentive) Procrastinators: Less likely to choose Tufts again ( t = 2.02, p <.05) Took less time to complete survey ( t = 2.53, p <.01) Proxy of how engaged they were? Office of Institutional Research, November 13

  24. Results 2013 (Low Incentive) Started before first reminder issued: 48.7% Six reminders to get the other 51.3%... Started before second reminder issued: 67% Explore procrastination in two ways: After one reminder or after two …but they were the same. (Stats presented will reflect one reminder) Office of Institutional Research, November 13

  25. Results 2013 (Low Incentive) Procrastinators: Less likely to engage in academic discussions outside of class ( t = 2.11, p <.05) Less likely to feel comfortable asking faculty for help Rec letter ( t = 2.17, p <.05) Additional educational opportunities ( t = 2.14, p <.05) Advising ( t = 1.94, p <.05) Office of Institutional Research, November 13

  26. Results 2013 (Low Incentive) Procrastinators: Less likely to participate in student organizations and community service ( t = 2.05, p <.05; t = 1.99, p <.05) Feel less able to contribute to the campus community ( t = 2.07, p <.05) Office of Institutional Research, November 13

  27. Results 2013 (Low Incentive) Procrastinators: Less satisfied with sense of community on campus ( t = 2.89, p <.01) Perceived prejudice against students from different socioeconomic backgrounds as a campus problem ( t = 3.07, p <.01) Office of Institutional Research, November 13

  28. Results 2013 (Low Incentive) Procrastinators: Less time to complete survey ( t = 1.91, p <.05) Less likely to have left a final comment ( t = 2.11, p <.05) (“Please use this space below to provide any additional comments about your Tufts experience.”) Office of Institutional Research, November 13

  29. Results 2013 (Low Incentive) No differences: Likelihood of choosing Tufts again GPA % Male % nonwhite Flourishing scale Office of Institutional Research, November 13

  30. 2012 vs. 2013 Early Late Early Late Non- Category Responders Responders Responders Responders responders 2012 2012 2013 2013 2013 GPA 3.48 3.31 3.45 3.43 3.35 % Male 39% 52% 37% 41% 57% % 24% 35% 34% 32% 31% Nonwhite Office of Institutional Research, November 13

  31. Percent of Class Nonresponding 100% 80% 2013 (Low Incentive) 60% 36.1% 40% 20% 6.7% 2012 (High Incentive) 0% Start R1 R2 R3 R4 R5 R6 End Office of Institutional Research, November 13

  32. Take Home Points Procrastinators were different from non- procrastinators More academically adrift Less engaged on campus More dissatisfied Those who waited for a reminder spent less time on the survey Office of Institutional Research, November 13

  33. Take Home Points Procrastinators different from non- responders Students who procrastinate in high- incentive situations participate because they have to. They otherwise wouldn’t be doing the survey. Office of Institutional Research, November 13

  34. Take Home Points Reminders are good! Send them. Dissatisfied and less engaged students procrastinate – wait for them! Plan your survey administration carefully. Incentives? When to send reminders? How many? Office of Institutional Research, November 13

Recommend

Tim ing is Everything: W hat w e can learn from survey procrastinators Presented by:

Tim ing is Everything: W hat w e can learn from survey procrastinators Presented by:

Tim ing is Everything: W hat w e can learn from survey procrastinators Presented by: Lauren M. Conoscenti, Ph.D. Tufts University Office of Institutional Research &amp; Evaluation (OIR&amp;E) AIR Annual Forum, Long Beach, California May

680 views • 30 slides
New methods for controlling timing channels Andrew Myers Cornell University (with Danfeng

New methods for controlling timing channels Andrew Myers Cornell University (with Danfeng

New methods for controlling timing channels Andrew Myers Cornell University (with Danfeng Zhang, Aslan Askarov) Timing channels e adversary can learn (a lot) from timing measurements. Known to exist Hard to detect Hard to prevent

639 views • 40 slides
2011 Citizen Survey Survey Administration Review of Feedback Future Action Plans February 27,

2011 Citizen Survey Survey Administration Review of Feedback Future Action Plans February 27,

2011 Citizen Survey Survey Administration Review of Feedback Future Action Plans February 27, 2012 www.cityofws.org 1 Prior Survey Methods Telephone Surveys Conducted 2006, 2007 and 2008 Structure and timing of survey limited

489 views • 14 slides
Origins of the project building condition survey Plan for funding and timing of the

Origins of the project building condition survey Plan for funding and timing of the

Address elements of the project over next several Board of Education workshops Origins of the project building condition survey Plan for funding and timing of the project Nov 1st Dec 6th High school and middle school maintenance

560 views • 36 slides
Surveying the Survey. What can we Learn about the Effect of Monetary Policy on Inflation

Surveying the Survey. What can we Learn about the Effect of Monetary Policy on Inflation

Surveying the Survey. What can we Learn about the Effect of Monetary Policy on Inflation Expectations? Michael Pedersen Central Bank of Chile XXV Reunin de la Red de Investigadores de Bancos Centrales CEMLA, virtual, 28-30 October 2020 * The

1.42k views • 66 slides
Learn Blackboard Learn Learn with others Learn in your own time, pace, space Learn through

Learn Blackboard Learn Learn with others Learn in your own time, pace, space Learn through

Learn Blackboard Learn Learn with others Learn in your own time, pace, space Learn through designing for teaching Learn with others 1. Black Swan Conferences Hear The bigger picture Higher Education Blackboard UWA Experience

478 views • 16 slides
Timing Analysis Timing Path Groups and Types Timing paths are grouped into path groups

Timing Analysis Timing Path Groups and Types Timing paths are grouped into path groups

Introduction to Digital VLSI Logic Synthesis Timing Analysis Timing Path Groups and Types Timing paths are grouped into path groups according to the clock associated with the endpoint of the path. There is a default path group that

541 views • 18 slides
FORMAL MODELING AND VERIFICATION FOR TIMING PREDICTABILITY Mathieu Jan, Mihail Asavoae, Belgacem

FORMAL MODELING AND VERIFICATION FOR TIMING PREDICTABILITY Mathieu Jan, Mihail Asavoae, Belgacem

FORMAL MODELING AND VERIFICATION FOR TIMING PREDICTABILITY Mathieu Jan, Mihail Asavoae, Belgacem Ben Hedia ERTS 2020 Toulouse Outline of the talk Motivations and goals Timing anomalies in Worst-Case Timing Reasoning An example of a timing

286 views • 16 slides
Top 5 Timing Closure Techniques Greg Daughtry Correct Timing Constraints Analyze Before

Top 5 Timing Closure Techniques Greg Daughtry Correct Timing Constraints Analyze Before

Top 5 Timing Closure Techniques Greg Daughtry Correct Timing Constraints Analyze Before Doing Implementation Strategies and Directives Congestion and Complexity Advanced Physical Optimization Create Good Timing Constraints

659 views • 31 slides
X-ray ray Timing and Timing and Polarizati Polarization on mi missi ssion &amp; &amp; inst

X-ray ray Timing and Timing and Polarizati Polarization on mi missi ssion &amp; &amp; inst

X-ray ray Timing and Timing and Polarizati Polarization on mi missi ssion &amp; &amp; inst strumen mentation DONG Y Yongw ngwei Center fo r for r Parti rticle A Astroph trophysics Ins nstitute o of High Ene igh Energy P gy

437 views • 15 slides
Timing and Coordination Essential Knowledge 2.E.2 and 2.E.3 Timing and Coordination Timing

Timing and Coordination Essential Knowledge 2.E.2 and 2.E.3 Timing and Coordination Timing

Timing and Coordination Essential Knowledge 2.E.2 and 2.E.3 Timing and Coordination Timing and coordination of physiological events are regulated by multiple mechanisms ( 2.E.2 ) Timing and coordination of physiological events are

598 views • 37 slides
A Highly Compressed Timing Macro-modeling Algorithm for Hierarchical and Incremental Timing

A Highly Compressed Timing Macro-modeling Algorithm for Hierarchical and Incremental Timing

A Highly Compressed Timing Macro-modeling Algorithm for Hierarchical and Incremental Timing Analysis Tin-Yin Lai, and Martin D. F. Wong March. 16, 2018 Outline Introduction Timing Macro-modeling Problem Formulation Previous

397 views • 23 slides
Digital Design Discussion: RTL Storage Components Shift Register Timing Register File Timing

Digital Design Discussion: RTL Storage Components Shift Register Timing Register File Timing

Principles Of Digital Design Discussion: RTL Storage Components Shift Register Timing Register File Timing RAM Up/Down Counter FIFO Queue Shift Register Timing Problem: Complete the timing diagram for the shift register shown. Assume

302 views • 12 slides
Timing Library Format (TLF) Advanced VLSI Design CMPE 414 Timing Library Format (TLF) TLF is an

Timing Library Format (TLF) Advanced VLSI Design CMPE 414 Timing Library Format (TLF) TLF is an

Timing Library Format (TLF) Advanced VLSI Design CMPE 414 Timing Library Format (TLF) TLF is an ASCII representation of the timing and power parameters associated with any cell in a particular semiconductor technology The timing and power

308 views • 15 slides
Agenda Compare 4 alternative timing mechanisms (long and short the S&amp;P 500)

Agenda Compare 4 alternative timing mechanisms (long and short the S&amp;P 500)

Agenda Compare 4 alternative timing mechanisms (long and short the S&amp;P 500) VectorVest confirmed ups and downs EMA / RT combo Timing Cube Ride the Waves Show the results of using the best timing method with a pair

804 views • 44 slides
1 Clock skew optimization Another approach for sequential timing optimization

1 Clock skew optimization Another approach for sequential timing optimization

Sequential Timing Optimization Long path timing constraints Data must not reach destination FF too late d max (i,j) s i + d(i,j) + T setup s j + P i j s s i i sj d(i,j) T setup Short path timing constraints FF should not

86 views • 7 slides
Learn more Do more Be more Learn more Do more Be more UNITY Learn more Do

Learn more Do more Be more Learn more Do more Be more UNITY Learn more Do

Learn more Do more Be more Learn more Do more Be more UNITY Learn more Do more Be more Key Staff linked to Year 7 Mrs Angela Haynes Mrs Louisa Smith Head of Year 7 Year 7 PSM Ms Leyla Mrs Julia Emmel Bilsborough

736 views • 27 slides
Liberty Timing File (LIB) Advanced VLSI Design CMPE 641 Liberty Timing File The .lib file is an

Liberty Timing File (LIB) Advanced VLSI Design CMPE 641 Liberty Timing File The .lib file is an

Liberty Timing File (LIB) Advanced VLSI Design CMPE 641 Liberty Timing File The .lib file is an ASCII representation of the timing and power parameters associated with any cell in a particular semiconductor technology The timing and power

298 views • 14 slides
A Compact and Accurate Timing Macro Model for Efficient Hierarchical Timing Analysis Pei-Yu Lee ,

A Compact and Accurate Timing Macro Model for Efficient Hierarchical Timing Analysis Pei-Yu Lee ,

A Compact and Accurate Timing Macro Model for Efficient Hierarchical Timing Analysis Pei-Yu Lee , Iris Hui-Ru Jiang, Ting-You Yang National Chiao Tung University Outline Introduction Problem Formulation Proposed Algorithm

807 views • 36 slides
Member Survey Results 2016 Membership Current # of Members 245 Survey Response The LCCC

Member Survey Results 2016 Membership Current # of Members 245 Survey Response The LCCC

Member Survey Results 2016 Membership Current # of Members 245 Survey Response The LCCC Board conducted a survey of its membership to learn more about the membership 245 members, 87 Responded=36% response rate Increased by 46

283 views • 15 slides
Relative Timing Driven Multi-Synchronous Design: Enabling Order-of-Magnitude Energy Reduction

Relative Timing Driven Multi-Synchronous Design: Enabling Order-of-Magnitude Energy Reduction

Relative Timing Driven Multi-Synchronous Design: Enabling Order-of-Magnitude Energy Reduction Kenneth S. Stevens University of Utah Granite Mountain Technologies 27 March 2013 UofU and GMT 1 Learn from Prof. Kajitana Think differently

844 views • 36 slides
Discussion: Remote Timing Attacks are Practical 600.624 2/11/05 Outline Why are timing

Discussion: Remote Timing Attacks are Practical 600.624 2/11/05 Outline Why are timing

Discussion: Remote Timing Attacks are Practical 600.624 2/11/05 Outline Why are timing attacks important? Clarifications Zero-One Gap / Neighborhood Size etc. Problems Questions Extensions Contribution Discussion

645 views • 26 slides
RESPONSIBLE INVESTMENT: NZ CONSUMER SURVEY 2018 A report exploring New Zealanders attitudes

RESPONSIBLE INVESTMENT: NZ CONSUMER SURVEY 2018 A report exploring New Zealanders attitudes

RESPONSIBLE INVESTMENT: NZ CONSUMER SURVEY 2018 A report exploring New Zealanders attitudes towards responsible investment Methodology +/-% . METHOD TIMING MARGIN OF ERROR 1,000 New Zealanders aged 18+ The survey took place from The

538 views • 11 slides
X-ray spectral timing: methods and interpretation Phil Uttley Why spectral-timing?

X-ray spectral timing: methods and interpretation Phil Uttley Why spectral-timing?

X-ray spectral timing: methods and interpretation Phil Uttley Why spectral-timing? Spectra are complex, hard to disentangle and GX 339-4 hard state interpretations are often degenerate model-rich (we have physical

424 views • 17 slides

More recommend