Planning & Managing Migrations Its for the birds. Har har. - PowerPoint PPT Presentation

Planning & Managing Migrations It’s for the birds. Har har. Aimee Degnan / aimee@hook42.com answers@hook42.com answers@hook42.com

Expectation Setting This is the first run of this presentation. It is being shaped for DrupalCon. Is text heavy because sketches are WIP. There is a lot to talk about. This is fast. Your input and questions are encouraged, although I may field them later. answers@hook42.com answers@hook42.com

Me Aimee Degnan, CEO 1996 – Enterprise Web Tech & CMS 2006 – PMP, Stanford Advanced PM 2008 – Drupal 2010 – Agile: Scrum Master, Product Owner aimee@hook42.com @aimeeraed www.hook42.com @hook42inc answers@hook42.com answers@hook42.com

Hook 42 Who are we? A boutique software development firm. Located in the San Francisco Bay Area. 4+ years, 12 employees Focus areas: site rescue, migration / upgrades, multilingual, SEO, responsive design, digital strategy, business process improvements answers@hook42.com answers@hook42.com

Who are you? Product Project Manager? Manager? Developer? Executive? All?!?! 5 answers@hook42.com

When and why to migrate? Software end of life. Mergers and acquisitions. Fixing source is worse than migrating. Infrastructure cleanup. Rebranding. answers@hook42.com answers@hook42.com

Who has migrated a site? In one word, describe it. J 7 answers@hook42.com answers@hook42.com

Types of migrations 1. Straight. One-to-one data + functionality. 2. Old à new architecture. 3. Multiple sources à single destination. (i18n) 4. Single source à multiple destinations. (i18n) 8 answers@hook42.com answers@hook42.com

Frequency of migration Single Pass (D6 à D8 Magic Button) Incremental Hybrid Some content is single Some content is incremental Some content is new / imported 9 answers@hook42.com answers@hook42.com

Size & scale of migration Small amount of content (manual) Enough content to code a migration. So much content!! It takes so long!! How many files?! Hybrid migration types. 10 answers@hook42.com answers@hook42.com

Technical platforms Drupal to Drupal Drupal to static archive Flat to Drupal Custom DB to Drupal Other CMS to Drupal 11 answers@hook42.com answers@hook42.com

Migration & infrastructure Pantheon to Pantheon (can’t mv files!) Acquia (file / SSH structure, branches) Local hosts vs. remote / shared hosts Network delays Shared authentication / source DBs 12 answers@hook42.com answers@hook42.com

Specialized team skills Migration Engineer Migration Manager Migrator (not just a button pusher) Data Tester 13 answers@hook42.com answers@hook42.com

Team considerations Projects are long Migration may be after-hours Work is incredibly detail oriented Careful, deliberate, detailed note taking Can be intense 14 answers@hook42.com answers@hook42.com

Role-specific considerations Business owners Account managers Project managers Migration engineers Developers Site builders Themers 15 answers@hook42.com answers@hook42.com

Migration terminology Migration pass = code + running code to migrate some content Plugins It is migration code that maps and modifies data. 3 types: process, source, destination Stub Drupal Migrate: Rollback (full-site or single pass) Import Update Status Feedback 16 answers@hook42.com answers@hook42.com

Thorough planning and vigilant management leads to success. And the numbers prove it. 17 answers@hook42.com answers@hook42.com

Make it easier on your team. Simplify where you can. 18 answers@hook42.com answers@hook42.com

Devil is in the details. 19 answers@hook42.com answers@hook42.com

Spreadsheets! No cell left behind. 20 answers@hook42.com answers@hook42.com

No, really. Spreadsheets. Migrations have a lot of moving parts. 21 answers@hook42.com answers@hook42.com

answers@hook42.com

Order of Operations is key 23 answers@hook42.com answers@hook42.com

Phases of a migration project 1. Pre-project education 2. Audit for migration 3. Discovery phase 4. Architect the new site 5. Migration mapping 6. Development phase 7. Pre-production migration passes 8. Testing and migration audit 9. Go-live!!!!!!!!!!!!!!! 10. Post-launch validation 24 answers@hook42.com answers@hook42.com

Pre-project education Goals: Set expectations of project activities Clarify the importance and impact of requirements freeze Identify phases statements of work, if a large enough migration Migration takes: Time Specialization Requirements lockdown Project fitness Transparency 25 answers@hook42.com answers@hook42.com

Audit for migration Goals: Surface the As-Is details of the site Begin understanding data Familiarity with site functionality 26 answers@hook42.com answers@hook42.com

Audit for migration Artifacts: Risks register & mitigation thoughts Content audit (structure, data types, size, UGC) Functionality audit Data health audit Infrastructure audit (for access) Functionality specific audits: SEO, Accessibility, User/Roles Source URL lists (patterns, special pages) Links to representative content. They will be used for the whole project. 27 answers@hook42.com answers@hook42.com

Audit for migration Lessons learned: Very few developers know how to audit for migration. Takes longer than you’d expect, even with tools. Auditing twice is costly. Do it right the first time. No cell left behind! Blank is not an answer. Keep your artifacts and info in one place. Prevent: “Oh, I didn’t think about that.” 28 answers@hook42.com answers@hook42.com

Spreadsheets! No cell left behind. 29 answers@hook42.com answers@hook42.com

Future state discovery phase Goals: Define functionality and improvements Prioritize features, with data in mind Capture data expectation requirements Artifacts: Feature list Glossary with AKAs Are all representative content links addressed? 30 answers@hook42.com answers@hook42.com

Spreadsheets! No cell left behind. 31 answers@hook42.com answers@hook42.com

Architect destination site Goals: Define new content structures Define infrastructure w/ migration considerations Artifacts: Site architecture spreadsheet (CT, Taxo, Forms) URL pattern planning Lessons learned: Every entity is a migration pass! (pssst. Paragraphs) Media entity is two passes (files + entity) You have to architect everything. Really. You can’t let your site builders just build stuff. Ever. 32 answers@hook42.com answers@hook42.com

Spreadsheets! No cell left behind. 33 answers@hook42.com answers@hook42.com

Migration mapping Goals: Document migration expectations for the business Provide the requirements to Migration Engineers A testing matrix for post-migration data audit 34 answers@hook42.com answers@hook42.com

Migration mapping Artifacts: Migration mapping (by entity type, field, subfield, value) Source fields à Destination fields + expected behaviors Taxonomy term / Select list à term mapping 35 answers@hook42.com answers@hook42.com

Migration mapping Lessons learned: Often done in parallel with new architecture Mind your types, field length, formats, dates, and filters! Select lists à taxonomy / Term à term Blobs à structured content take extra programming 36 answers@hook42.com answers@hook42.com

Spreadsheets! No cell left behind. 37 answers@hook42.com answers@hook42.com

� � Development - FIXME Site architecture Testing � migration mapping Bug Fixes � infrastructure site building Incremental Passes � migration passes theming answers@hook42.com answers@hook42.com

Development phase Goals: Get ‘er done. Artifacts: Migration order w/ dependencies, stubs Migration code New site + infrastructure Detailed site rollback process and time Go-Live checklist: full list of migrations, duration, expected behavior 39 answers@hook42.com answers@hook42.com

Development phase Considerations: Site building must be complete before migrations start.* Migration pass dependency plan Develop migration code Track commands, durations per pass Don’t over engineer. You are only doing this once. 40 answers@hook42.com answers@hook42.com

Development phase Lessons learned: Max joins on MySQL DB is 61 Don’t over-engineer. Your code will go away when the migration is done. Documentation is your friend. 41 answers@hook42.com answers@hook42.com

Spreadsheets! No cell left behind. 42 answers@hook42.com answers@hook42.com

Pre-prod migration phase Goals: Keep running migrations Debug and test data Populate the bulk of data Artifacts: Use the migration pass worksheet Track time, success / failure, issues Lessons learned: This is needed by all teams. 43 answers@hook42.com answers@hook42.com

Testing & migration audit phase Goals: Testing new site architecture with migrated data Layout, functionality, etc. Artifacts: Browser testing SEO testing / redirects Performance tuning Lessons learned: You need this. Give this phase a good time. There is the “Moment of Truth” when the dev server is like prod. 44 answers@hook42.com answers@hook42.com

Spreadsheets! No cell left behind. 45 answers@hook42.com answers@hook42.com