1. Large Drupal Site Builds SXSW Interactive, 2015
2. Who the Heck are We? Rob Ristroph Technical Architect at Acquia, a digital experience company based in Boston, MA and with remote team members worldwide. Based in Austin, TX! ATX Hacker Space Member Drupal/Open Source contributor
3. Who the Heck are We? Jenn Sramek Sr. Engagement Manager at Acquia, based in the San Francisco Bay Area Manage large-scale Drupal projects for organizations and companies that are often new to Drupal Formerly Dir. of Operations at CivicActions Board President for the Haight Ashbury Food Program
4. Who the Heck are We? Alex Urevick- Ackelsberg Co-Founder/CEO of Zivtech Extensive large Drupal project experience Wears many hats Nick Lewis Site Architect at Zivtech Extensive Drupal site architecture experience Sock Puppet Master
5. Who the Heck are You? Drupal experience? Large-scale web? Open Source? Have an active/current large-scale project? Planning one? Waterfall/Agile? Local/from other states/countries? Anything else interesting?
6. Logistics Connectivity Power/Wireless Biology Restrooms Food/Drink Overall Schedule 9:30 am - 1:30 pm Shared Notes and Resources https://pad.lullabot. com/p/Large_Drupal_Sites_Workshop_SXSW_2015
7. Topic Overview Project Management - whats different about big projects ? Technical: Development workflow Custom APIs and the ecosystem of API consumers Scaling and performance at the enterprise level - configuring cache, finding bottlenecks Large-scale data migration Monitoring Social: Open Source and corporate culture Training Larger development team dynamics - culture of quality Multiple teams and vendors Tell us what to focus on
8. Defining Large Projects SXSW Interactive, 2015
9. What is Large? Customer has a lot of money ? Site has a lot of traffic or data ? Project build runs for a long time ? Whatever is pushing past your comfort / experience zone in some way.
10. What is Large More than one website Platform and distribution builds Another team uses your platform or distribution Multiple stakeholders Company vs. Divisions or Franchisees Technical vs. Marketing departments APIs and System interactions mail delivery SSO authentication, including roles and other metadata Web metrics Workflows, editorial processes Significant migration of data
11. What is Large Content Staging Workflow on steroids / between servers Inflexible legal requirements Solutions Content staging server and clone Site Preview System (SPS) Deploy suite of modules Challenging, Interesting and Fun Inter-disciplinary development Complex feature development Cutting-edge interaction design
12. What is Large The most common single feature of a large project is the interaction with more team members and more stakeholders.
13. Working with Large Teams Working with large teams is the most likely source of project failure... If you are new to big projects, interacting with and managing big teams is probably the most unfamiliar area and the greatest area of risk Even large organizations can fail at large project implementation repeatedly All the same technical issues that show up in large projects can show up in smaller projects, but it is harder to maintain clarity, quality code, expectations, and scope when dealing with large teams
14. Working with Large Teams You start with a client / customer Add a design / strategy vendor Add another developer team on the customer side Add other outside developer team(s), with sub-contractor relationship Add multiple other vendors, with no sub-contractor relationships Separate projects with out-of-sync timelines All still working on the same project with ultimately the same goal for success...
15. Working with Large Teams Timeline differences Scope differences or gaps Resource gaps Process differences Deliverables differences or gaps that may be significant Coordinating vacations/time off Internal team mentorship, developer ladders, or other process for fully on-boarding new team members (including client)
16. Working with Large Teams Multiple parties make DevOps essential Practices that just provide efficiency at the scale of one team become absolutely necessary: Pull request workflows Peer code review Standardized development environments Automated testing Tech demo ?
17. Working with Large Teams The most rewarding parts of big projects Personal growth and learning Exposure to other teams is like having several jobs without switching bosses Part of a larger community
18. Getting Large Projects Pick the projects, dont let them pick you Initiate contact with particular clients and projects that interest you Talk with your whole team when going after something large Long term projects should be something people find fulfilling Winning by the lowest bid is a red flag Spend much more time vetting/ do formal Discovery Be picky; build padding into quotes
19. Picking a Vendor Is the vendor interested enough to be committed to a long project ? Unrealistically low bids are a red flag Budget for discovery, and pay for it If the vendor has not done projects of your size before: Allocate more to planning Smallest MVP possible A partnered bid may mitigate some risk If you fall behind, cut scope instead of adding resources
20. Overall Process and Project Management SXSW Interactive, 2015
21. Project (Program) Management Overall... The basics become essential JIRA or another tool Discipline in using that tool Communications and Report Generation is key, and may involve X-facing veation Reporting is part of the effort of a Project or Program Manager More project managers on different teams, coordination Often multiple JIRA instances and trackers in play On big projects, project managers can really shine and make an obvious difference, and technical people who hate it are more likely to be able to avoid it (not necessarily a good idea ?)
22. Discovery for Large Scale Drupal Curiosity is a Skill...
23. Discovery for Large Scale Drupal Discovery is really a very long list of questions to answer before you start development: - Create shared understanding of the work to be done and its priority, and start technical requirements docs - Align everyone on the process that will be used for development, quality assurance, signoff/acceptance - Surface any risks or potential issues that will require mitigation planning - Document any schedule or resource considerations - Right-size the effort of development
24. Discovery for Large Scale Drupal Technical Requirements: - Content types and fields - User roles/permissions - Content and site management workflows - Integrated service requirements (CDN, analytics, single-sign-on, Translation Management Services, payment processing, feeds,) - Migration details (size and complexity of database, content type/field mapping, consistency of data) Non-Technical Requirements: - Resources including third-parties, dev team(s), schedules, dependencies, internal technical resources, training needs - Internal timelines and schedule constraints (vacations, major internal events, conferences, product releases) - Security/Governance workflow and approval process (incl. clearance for those with data access and secure systems for data transmission and storage) - Access to associated systems and tools
25. Planning for Large Scale Drupal - Timeline - Dependencies - Risks and Mitigation Planning - Resources (considering attrition of internal and external teams) - Technical/Architectural planning (including POCs/rapid prototypes) - Code management - Security management - Plan for addressing technical debt - Governance (assignment and acceptance) - Maintenance plan, development cycles, post-launch - Pre-launch/Post-launch planning
26. Planning for Large Scale Drupal Involves LOTS of technical-specific planning (by the architect). A site architect may do as much technical planning as the PM does project planning (code management, maintenance, release planning, etc). Documenting risks protects everyone, and the project Interim maintenance and remediation Documentation needs may differ for large projects
27. Now that all the pieces are in place... Waterfall? OR Agile?
28. Working with the MVP
29. Tools for Large Scale Development SXSW Interactive, 2015
30. Tools for Large Scale Development Plan for tools to scale - know what happens to them when the project is over Ticketing system Document repository Code Management tools UX/Design Management tools Communications Channel(s) Training tools
31. Tools for Large Scale Development JIRA/Rally/others - plan for tools at scale and know what happens to them when the project is over. Code management tools - Github, BitBucket (more integrated with JIRA for one PR per JIRA ticket) Process more than product Onboarding dev documentation. UX/Design management tools - differ by firm, many use BaseCamp Living document for coding/best practices/standard practices
32. Development Managing multi-team interactions Deployment tools and procedures will take up a lot of time Delays between code and UAT Deployment and DevOps can have huge (%30 ?) impacts on efficiency Re-work is the biggest cost
34. Iterative Development Developers cannot JUST develop, have to document how they developed. There is a much larger team depending on clarity. Can effect technology choices Iterative can differ at the enterprise level There can be more than one effective Agile team Include maintenance rounds in iteration Plan for longer lags between dev and signoff Get sprint signoff outside of JIRA or other ticket system
35. Performance Planning Plan for addressing performance issues early Collect realistic requirements with context Benchmark legacy site for reference Test early How do you review contributed modules/code for performance? Not intuition - always check What tools to use to benchmark? Make sure they match your expected traffic How do you populate the site with enough content to test? If benchmarking can be automated, just repeat as project proceeds
36. Scaling and Performance Scaling often matters more than Performance Editorial interactions, cache updating are more important Collect performance, scaling, time-to-refresh requirements just like any other, and test for them Optimizing Drupal for performance is a specific skill that requires Drupal experience But if you can automatically test it can just like any other debuggin Hard to designate the performance issues to one person or team; again interactions between teams are essential
37. Scaling and Performance Scaling may mean more small sites not one big site See multisite debates Enforcing Code quality and standards Peer Reviews Views Cache Bully and similar Automated testing Automated code sniffer Automated testing is another thing you may not get to work on except on large projects.
38. Deployment For each deployment of code, it will be important to have a consistent and documented process: Preparing for Deployment: What do you need to do to get ready? Includes reviewing all merged pull requests, updating release notes, and preparing and validating any needed deployment steps. QA Steps: What do you need to do to prepare for QA? Includes tagging the release, cloning UAT database to QA environment, smoke and unit tests. Acceptance: What do you need to do to gain client acceptance? Includes preparing a demo, regression testing, and preparing the client for UAT testing.
39. Large-scale data migration and content management SXSW Interactive, 2015
40. Data Migration Review and get agreement on the process Consider data security Clean the data Do not scope it until you have mapped it Make sure you know what is all is! Dump and Map it! (content type by content type, field by field)
41. Data Migration Re-Scoping migration after mapping Incremental migration Differential migration Special considerations for beta sites (where there may be two versions of a live site) Plan for deprecated systems Plan for wrapping up with manual migration (copy/paste or rewrite/edit)
42. Migrate Module A complex migration can be a large portion of development effort Plan for continuous migration Run a scheduled migration until canonical DB switchover Iterate like any other development Make sure client can devote resources to checking content throughout the process
43. Content Staging SXSW Interactive, 2015
44. Content Staging Often discussed as part of Discovery, because many enterprise companies have structures that utilize a content staging environment. With Drupal, this is most often not needed or can be fairly complex. Content staging, if needed, may be one of the interesting problems to solve that inspires your team to take on the project.
45. Content Staging Draft / published Workflows Workflows with SPS - complexities multiply Staging server, simple DB copy - limiting Staging server, with workflow, and deploy module If editorial staff get involved late, this is often a late-in-project scope explosion.
46. Training SXSW Interactive, 2015
47. Site Owner / Builder Training Talk about it during Discovery!!! Training needs to be built into the schedule Project Managers can become de-facto trainers Include teaching clients to do UAT (as requisite for end- of-project training) Leverage available resources (Drupalize.me, Lynda.com, Drupal.org, BuildAModule and others) for training Plan to train the trainer
48. Training Model Start with defined needs Decide on the format of training (in-person, training materials on separate site or tool, formal training) Quantify trainees and role-specific tasks required of them in order to plan and estimate Plan to train at least some roles incrementally (often some training will be needed for UAT ) Leverage QA and UAT testing steps for training materials
49. Handoff and Transitions SXSW Interactive, 2015
50. End Goals and Transition Big projects will involve transitioning in some way: To other vendors To internal teams From active development to maintenance mode Re-acquiring the project later (in another phase) Documentation and consistent development practices are essential
51. End Goals and Transition What happens next? Discuss at the beginning of the project what the end of it looks like. Maintainability - avoid risk to support obligations Technical debt - document it Who will manage code after the final launch? Process and tools changes?
52. SXSW Interactive, 2015 Giving Back to the Open Source Community
53. Takin it to the Community Contribution makes Drupal go! Discuss the value of and plans for contribution during Discovery (even if you do not plan to contribute code right away) Making contributable code a design principle Reduce future risk / support costs Celebrate contributions (and give time to devs to do it as part of the project schedule and effort)
54. Takin it to the Community How to Contribute back to the community: Report Bugs! Contribute a patch if you fix the issue Design for Contribution Document how the module or code works Employ best practices Get a peer review
55. Takin it to the Community Some examples of sponsored modules: - https://www.drupal.org/project/media - https://www.drupal.org/project/i18n - https://www.drupal.org/project/commerce
56. Takin it to the Community Now Available! Attributions for contributions on Drupal. org! - https://www.drupal.org/node/2451283
57. SXSW Interactive, 2015 Questions for Us? Or for Each Other?
58. SXSW Interactive, 2015 Resources
59. Resources Shared Resources and Notes document https://pad.lullabot. com/p/Large_Drupal_Sites_Workshop_SXSW_2015
60. Contact Us Rob Ristroph - @robgr - firstname.lastname@example.org Jenn Sramek - @ideaseed - email@example.com
61. (we would love your) feedback, please... sxsw.feedogo.com/fdbk.do?sid=IAP42024
62. Questions Answered 1. What issues should receive special attention early in life time of a large project, that need to be set up in the first few weeks to establish best practices and culture through the longer life of a big project? 2. How do you keep a larger team of developers working in close collaboration, especially as bigger projects are more likely to have developers spread across time-zones and even different companies through multiple vendors and sub-contractors? 3. How can I assure that project and team can handle people leaving and joining the team on a regular basis, given that large Drupal projects can survive long enough to experience developer turnover, and often require specialists to join the team for a shorter phase (such as a migration expert at the start)? 4. What special efforts might be required to make sure that enhancements to Drupal and other open source projects can get outside of the organization and back to the parent project? 5. In terms of the "ecosystem" of a large enterprise drupal project, what special needs will surprise someone who has only worked on smaller projects -- APIs and Documentation that need long term maintenance, automated testing, large numbers of stakeholders and resulting approval bureaucracy? 6. What things are related to doing large projects that someone who had only done small/medium ones would not know? Partner JIRA vs. Own JIRA - Tools differences? other diffs?
63. Iterative Development On large projects, devs spend a lot of time figuring out where a ticket is and whether or not it can be moved forward. Another portion is if you do not document technical debt as it accumulates, you cannot do maintenance sprints very efficiently, nor can you argue for a maintenance sprint effectively. People can be hesitant to keeping a list of technical debt. By documenting it, you protect yourself. Make it part of your process to document when you make technical decisions that create technical debt or find technical debt on code review, and get signoff as part of sprint planning. Estimate work only after the details of the story have been approved (so there is not scope creep on each item as scope is added that client expects the same estimate still applies).