Design Lessons From Binary Fission A Crowd Sourced Game for Precondition Discovery Kate Compton, Heather Logas, Joseph C. Osborn, Chandranil Chakrabortti, Kelsey Coffman, Daniel Fava, Dylan Lederle-Ensign, Zhongpeng Lin, Jo Mazeika, Afshin Mobramaein, Johnathan Pagnutti, Husacar Sanchez, Jim Whitehead, Brenda Laurel University of California, Santa Cruz John Murray SRI International Augmented Design Laboratory // Center for Games and Playable Media / http://games.soe.ucsc.edu/adl // http://games.soe.ucsc.edu
Design Lessons From Binary Fission • Designing for citizen science • Formal program verification • Turning players into computers (but happy ones!) • Helping the computer help the player • Players supporting players This material is based upon work supported by the United States Air Force Research Lab- oratory (AFRL) and the Defense Advanced • Results and conclusions Research Projects Agency under Contract No. FA8750-12-C-0225. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of AFRL or DARPA.
Design Lessons From Binary Fission Designing for citizen science • Players are good at being better “computers” than computers • Better pattern-induction, better 3D spatial mapping, better “intuition” Khatib, Firas, Seth Cooper, Michael D Tyka, Kefan Xu, Ilya Makedon, Zoran Popovi ć , David Baker, and Foldit Players. 2011. “Algorithm discovery by protein folding game players.” Proceedings of the National Academy of Sciences 108 (47): 18949–18953.
Design Lessons From Binary Fission Designing for citizen science • Players are good at being better “computers” than computers • Better pattern-induction, better 3D spatial mapping, better “intuition” Reed, Jason, et al. "An exploratory factor analysis of motivations for participating in Zooniverse, a collection of virtual citizen science projects." System Sciences (HICSS), 2013 46th Hawaii International Conference on. IEEE, 2013.
Design Lessons From Binary Fission Designing for citizen science • Players are good at being better “computers” than computers • Better pattern-induction, better 3D spatial mapping, better “intuition” Iacovides, Ioanna, et al. "Do games attract or sustain engagement in citizen science?: a study of volunteer motivations." CHI'13 Extended Abstracts on Human Factors in Computing Systems. ACM, 2013.
Design Lessons From Binary Fission Designing for citizen science • DARPA Verigames Project • Crowd Sourced Formal Verification • Formal program verification is slow and expensive • …but it’s very useful! (the promise of secures and bug-free programs) • Can we repeat the success of Foldit, Zooniverse and EyeWire? • Dean, Drew, Sean Gaurino, Leonard Eusebi, Andrew Keplinger, Tim Pavlik, Ronald Watro, Aaron Cammarata, et al. 2015. “Lessons Learned in Game Development for Crowd- sourced Software Formal Verification.” In 2015 USENIX Summit on Gaming, Games, and Gamification in Security Education (3GSE 15). Washington, D.C.: USENIX Association, August.
Design Lessons From Binary Fission Formal program verification • Is x always even? var x = 0; • Will t = y(init) at the end of the var y = 20; var z = 1; loop? var t = 0; • Is x always less than y? Is z? while (x < y) { z += x%(y%6 + 1); • What about for different starting x += z%3; values of x, y and z? y—; t++; • What can we say about this loop? }
Design Lessons From Binary Fission Formal program verification var x = 0; • Fava, Daniel, et al. "Gamifying program var y = 20; analysis." Logic for Programming, Artificial Intelligence, and Reasoning. Springer Berlin var z = 1; Heidelberg, 2015. var t = 0; • Fava, Daniel, et al. "Crowdsourcing program while (x < y) { preconditions via a classification game." z += x%(y%6 + 1); Proceedings of the 38th International Conference on Software Engineering. ACM, x += z%3; 2016. y—; t++; • Require function invariants, so lets get some } from the players
Design Lessons From Binary Fission The Verigames project http://www.verigames.com/
Design Lessons From Binary Fission The Verigames project (Xylem)
Design Lessons From Binary Fission The Verigames project (Xylem) Don’t make your player feel like a bad computer, Make them feel like an awesome cyborg!
Design Lessons From Binary Fission Basic design of Binary Fission • Hiding the data, hide the math • The “Dragonbox” technique • Dealing with many sets of data at a time (generating sample sets of data to test • complexity early on! )
Design Lessons From Binary Fission Basic design of Binary Fission Brief Demo!
Design Lessons From Binary Fission Basic design of Binary Fission • There are data that were generated by the function (“true” data) • and data that weren’t (“false” data) • We represent these in blue and gold “quarks” • Each “filter” splits the quarks in a different way • Can you build a tree of filters to split the “good” data from the “bad” data, effectively?
Design Lessons From Binary Fission Optimizing for the right results • Scoring: optimize for sorting well, but efficiently • In”wild” data, not every problem may be perfectly sortable! (avoid “100%” UI, this game may be • frustrating to completionists)
Design Lessons From Binary Fission Turning players into computers • Don’t waste human-time doing what algorithms are good at If we can brute-force a solution, do that! • (unless you have a grant that says you need to use crowd-sourcing) • What aren’t algorithms good at? “intuiting” patterns? • Hypothesis: give players the “right” data, at the right rate, and they will come up with from what data? • intuitive understandings of underlying patterns (magic intuition)
Design Lessons From Binary Fission Turning players into computers • What we can do with algorithms • Make potential “basic” filters • x > y • x*19 - 4 < y%10 • Calculate their efficiency at sorting any given set of data • What we can’t do* • build an optimal tree
Design Lessons From Binary Fission Helping the computer help the player Given filters, how good are they for • the current set of quarks? Shopping in “filter space” • Players as the decider •
Design Lessons From Binary Fission Helping the computer help the player (magic intuition) Automate more information! • What *can* we calculate? What *can* we show? •
Design Lessons From Binary Fission Helping the computer help the player * …. so is this actually better than brute-force or AI? • • Can AI build a better tree with an AI than human users can? • FANTASTIC. • Give them a “robot arm” autosolver, and let them figure out when to override it. • Maybe it’ll be useful for more complex problems that can make use of human+bot power awesome cyborg!
Design Lessons From Binary Fission Helping the computer help the player • ….by helping humans help each other • Build a system where helping others is easy , obvious , and supported with automation?
Design Lessons From Binary Fission Players supporting players • Players like helping other players • Players like seeing that other players are playing (empty restaurant syndrome) • Players like competing with each other Sometimes competing to be the most helpful! • • Scale matters. • Feedback loops happen.
Design Lessons From Binary Fission Results and conclusions • The crowd managed to find some novel, legible and useful program invariants. • Social features never took off (we were an empty restaurant)
Design Lessons From Binary Fission Results and conclusions • Was there enough information to create the “magic intuition”? • Power users wanted more views of the math, more tools
Design Lessons From Binary Fission Results and conclusions • Was there enough information to create the “magic intuition”? • Power users wanted more views of the math, more tools
Design Lessons From Binary Fission Results and conclusions • Did we have the right theme? • What is the role of story and theme in science games? • if it gives you an actionable “hypothesis” (SimCity effect*) • if it gives you the right audience for your gameplay *Noah Wardrip-Fruin, Expressive Processing
In conclusion Automate Augment (awesome cyborg) Iterate!!!
Design Lessons From Binary Fission Questions? • Kate Compton, kcompton@soe.ucsc.edu • Jim Whitehead, ejw@cs.ucsc.edu • John Murray, SRI jxm@sri.com In conclusion Automate Augment (awesome cyborg) Iterate!!!
Recommend
More recommend
