Agile Baby Steps

By and large, most software in the world is made to a spec enshrined into immutability, then interpreted differently by various parts of the teams building it, and leading finally to a loose coupling of incompatible pieces which form a whole that doesn’t serve the needs of the people who it was built for, followed by delays until it does. This is part of the reason why most software is awful, and this is called “waterfall” development.

Having experienced the software development cycle fall short of expectations a few times, we’ve decided to look at a more Agile software development process, perhaps with some skepticism since its rhetoric tends to sound a bit like Scientology or Crossfit, but with code. Much like any ideology, Agile is split into several camps including Scrum, Kanban, and eXtreme Programming. Kanban was inspired by manufacturing processes and being able to change the design of a thing if a problem is discovered, Scrum involves a full-time “Scrum Master” and eXtreme Programming is known for it’s focus on pair programming. After reading Jonathan Rasumusson’s The Agile Samurai, we decided to start adopting pieces of Agile methodologies into our latest web project.

Previously in this blog you we’ve introduced our mobile development and transforming kiosks into the greater ASK program. On the other side of the conversations happening on our mobile app is our audience engagement team, who will need to field questions from several people at once being outnumbered by curious visitors while bringing up relevant information on what artwork is near the user and more. This is software for a job which hasn’t existed in the past, exactly. Unlike building, say, a blog where its use cases are well established and understood, we needed to be able have a solid and clear understanding of what this dashboard would do.

We adopted Agile user stories so that our four-strong web developer team would ultimately land on the same page. Using a textbook Agile technique, we wrote stories down on physical sticky notes using a strong cryptographic cipher known as my handwriting. Bringing in our stakeholder, designer, and entire team into discussion around these stories helped build a coherent narrative which invoked more of the right questions and helped push decisions along quickly. This, along with standup meetings helped keep our vision consistent with our code. Unlike the waterfall model, the project was subject to change as we discovered new problems to make changes so that we would build a system that would survive reality.

Although sticky notes enforced a consistent narrative, the pieces needed to be built were still siloed in individual developers’ todo lists. This turned out to be problematic since it disjoined work from stories and scattered work into tasks across everything rather than tackling a story at a time. Our first in-house alpha launch fell short of its release date partially due to the underestimation of the amount of front-end work that needed to be done—this was also our first use of a JavaScript front-end framework. Since tasks were not attached to stories, it wasn’t easy to tell, from a project management perspective, which particular stories are affected or even whether tasks belonged in a story and should not have been worked on. Because of that, there was no new knowledge to gain by looking at the sticky notes and thus, everyone forgot about them. We also happened to be using free sticky notes gifted to us by a vendor we’ve worked with, so with the science of wall adhesive still not quite perfected a few stories literally fell through the cracks and weren’t rediscovered until we rearranged the furniture in the office.

After that first release, we migrated the stories into Trello, which is structured so that each “card” represents a story with checklists attached. With that structure, it’s a lot easier to see the time needed for a story grow from its original estimate as tasks are added. Once it reaches a rough “too many tasks” peak, it can then be split into multiple stories with a new time score added in, triggering a reality check on the overall project scope.

Another aspect of Agile we adopted was test-driven development. The dashboard is made up of several layers of code and data, with an SlimPHP API at the core, A basic FuelPHP site as its mantle, and a large and very busy AngularJS application at its surface. If a bug should be discovered, it’s easier to find in a test for just that part of the stack while it’s being built than to build everything and go on a time-consuming multi-layer bug hut. Writing tests hasn’t eliminated bugs, but it has reduced them to mostly display or protocol/connectivity-related issues. In our case, we used phpunit for the SlimPHP and FuelPHP layers, and Jasmine for the Angular layers. Since we’re using Jenkins as our continuous deployment system, we were able to make sure code was only pushed live when tests passed.

Even with unit and integration tests passing, there is no better bloodhound for bugs than a human operator. Before any story can be moved to “Done,” it must go through “Verification” where I and sometimes others recreate the question and answer action that the story satisfies, and only move it to done if nothing is missing. This catches a lot of “it works on my machine” type issues, which can then be tossed back to be revisited and verified again. After verification, we then simulate a test with other department staff in-gallery where the whole flow of the visitor experience is tested holistically.

We’re not only inventing a new type of application with our dashboard, but also a new type of workflow for a new type of job to cover it. The moving target of the scope as we tread into new territory is bound to change. Even though we’re not canon orthodox Agile (can’t always pair program!), we’re moving forward with much more agility than we’ve been historically.