The yt Project: Growing & Engaging a Community of Practice Matthew Turk Columbia University
What is yt?
astro-ph/1011.3514 astro-ph/1112.4482 yt-project.org
yt is a platform for analysis and visualization of astrophysical* simulations
install script: Full dependency stack Source code Development environment GUI Sample data
There are many simulation codes.
There are many simulation codes, but there’s only one sky.
data structures, methods, assumptions, IO formats, units, variable names, ...
Fully-Supported Semi-Supported In-Progress Enzo Chombo Cactus FLASH Athena Hydra Orion ART Gadget Nyx RAMSES Uniform Data GDF Piernik
yt is designed to address physical, not computational, entities and questions.
Orthogonal Rays 1D Non-orthogonal Rays Slices Oblique Slices 2D Projections Spheres Rectangular Prisms Disks/Cylinders Inclined Boxes 3D Clumps Extracted Regions Ellipsoids
Scripting interface, HTML5 GUI, in-situ processing, transparent multi-level parallelism...
yt is supposed to get out of the way.
from yt.mods import * pf = load(’galaxy0030/galaxy0030’) p = SlicePlot(pf, 2, ’Density’, ’c’, (200,’kpc’)) p.save(’Galaxy’)
10 -12 g/cc
C.B. Hummels et al. 6
Canned Analysis Tasks
Absorption Spectrum Coordinate Transformations Halo Finding Mass Functions Merger Trees Halo Profiling Level Sets Light Cones Light Rays Time Series Star Analysis Two-Point Analysis
“Community”?
Traditional View “Users” “Developers”
Most Scientific Software “Users” “Developers”
Community of Practice “Devusers”
“Developers”
“Developers” Inspection and verification
“Developers” Inspection and verification Tracking modifications
“Developers” Inspection and verification Tracking modifications Sharing information
“Developers” Inspection and verification Tracking modifications Sharing information Adding functionality
“Developers” Inspection and verification Tracking modifications Sharing information Adding functionality All are necessary characteristics of the scientific process as a whole.
“Users”
“Users” Uncritical acceptance of code...?
“Users” Uncritical acceptance of code...? “These are people we give the code to that don’t care how it works.”
Developing as a Team
Challenges
Reward Structure
de facto and de jure
de facto and de jure • Utilization of developed tools • Respect from community • Project involvements • Invitations to speak
de facto and de jure • Funding • Publications • Citation count • Influence
Traditional astrophysics does not favor tool builders.
Chores
Chores Documentation, testing, outreach, infrastructure.
Tasks not fully-aligned with reward structure goals present great motivational challenges.
Co-opetition
Co-opetition ( ) • Funding • Publications • Citation count • Influence
How developer community engagement -- as seen in cohesion, excitement, energy -- is affected by funded improvements is unclear.
Strategies
The Art of Community by Bacon Producing Open Source Software by Fogel Team Geek by Fitzpatrick & Collins-Sussman Organizing Simulation Code Collectives by Sundberg Scientific Software Production by Howison & Herbsleb
You must design the community you want.
Technical & Social
Technical & Social
SVN hg Users & Repos Users & Repos Repository Users & Repos Users & Repos Users & Repos Users & Users Repos
Forky development: very low barrier to entry; everything comes in the box.
Testing: answer as well as integration tests get run every 30 minutes.
Code review: eyes on (nearly) every changeset mentoring new developers
The path upstream must be obvious.
• Happy application • Itch-scratching • Submission • Code review & mentoring • Participation
Non-core contributed code: a place for sharing scripts, data and images
Communication
Communication All business is conducted openly.
Communication Immediate
2 Communication Low-Latency
Communication High-Latency
Technical & Social
H R T
Humility R T
Humility Respect T
Humility Respect Trust
Humility Respect Trust (Fitzpatrick & Collins-Sussman)
The most important thing I try to remember is that I want to foster a community of peers, not of elites.
“ I’ve noticed something is acting strangely with ... “
“ I’ve noticed something is acting strangely with ... “ “ You’re probably doing it wrong. “
“ I’ve noticed something is acting strangely with ... “ “ Can you tell us how you would “ expect it to act?
“ I think that there might be a bug in the way ... “
“ I think that there might be a bug in the way ... “ “ It’s like that for very good reasons. “ Don’t touch it.
“ I think that there might be a bug in the way ... “ “ It behaves the way it does “ because ...
Thoughtfulness, inquiry, confidence, letting go.
This culture must be seeded directly.
This culture must be seeded directly, because culture will self-propagate.
letting go
Pride rather than ownership
Projects passing between people
Too much control can smother.
will Too much control can smother.
Successes
Developed by working astrophysicists.
Tom Abel Jean-Claude Passy David Collins Thomas Robitaille Brian Crosby Anna Rosen Andrew Cunningham Anthony Scopatz Nathan Goldbaum Devin Silvia Cameron Hummels Sam Skillman Ji-hoon Kim Stephen Skory Steffen Klemer Britton Smith Kacper Kowalik Geoffrey So Michael Kuhlen Casey Stark Eve Lee Elizabeth Tasker Chris Malone Matthew Turk Chris Moody Rick Wagner Andrew Myers John Wise Jeff Oishi John ZuHone
Tom Abel Jean-Claude Passy David Collins Thomas Robitaille Brian Crosby Anna Rosen Andrew Cunningham Anthony Scopatz Nathan Goldbaum Devin Silvia Cameron Hummels Sam Skillman Ji-hoon Kim Stephen Skory Steffen Klemer Britton Smith Kacper Kowalik Geoffrey So Michael Kuhlen Casey Stark Eve Lee Elizabeth Tasker Chris Malone Matthew Turk Chris Moody Rick Wagner Andrew Myers John Wise Jeff Oishi John ZuHone
Recommend
More recommend
