Alistair Cockburn was in Helsinki some weeks ago to run a workshop on agile software development for the AGILE ITEA project and since they were running short of participants from within the project, us externals got a chance to get in as well.

In general, I enjoyed the workshop a lot although Alistair seemed to be too harsh on XP in particular. Lots of familiar faces of course from Nokia and F-Secure, for example, but also a lot of interesting discussions.

Some of the observations made during the workshop were that some saw agile from a pure software development or project perspective while others saw it as more of a property of an organization. A lot of this variation was obviously due to people coming from varying backgrounds and varying roles. Similarly, while Alistair clearly separates "agile" (able to respond to change) and "adaptive" (agile + able to adapt itself), the audience (including me) considered the two to be largely synonymous. I suspect some of that is because the adaptiveness is largely a given these days even though the first descriptions of XP, for example, may not have included it explicitly.

Along these lines, Alistair also posed the suggestion that XP and Scrum are "Shu" processes while Crystal (his own agile method) is a "Ha" process. This of course refers to Shu Ha Ri, the progress of one's skill level from imitating known-good techniques to adapting the techniques to making no difference between techniques. The argument was that XP and Scrum define concrete practices to follow while Crystal suggests ways to adapt a process to the situation. Honestly, I found this a bit far-fetched since XP and Scrum also recommend very explicitly these days to adapt the process once you've got the experience to know how to adapt it.

Some specific ideas we went through included the use of video tape as documentation (in the context of the bandwidth and temperature of various communication media), the slope and the cliff, and a certain reality TV show as an example of setting yourself up for the next game. (this all makes more sense if you've read Alistair's writings in the past)

It was a good workshop. I hope you're coming back before XP2006, Alistair ;)

Just wanted to post a quote from Keith Ray on the XP mailing list:

If you recognize your place of employment from this rant, you're not alone. I've seen a lot of that, coming from a global consulting giant working for some of the largest corporations in Finland. My advice (and not just if you want to do XP) would be to consider whether you're proud of what you do and whether you would like to be, try to make management see the problem, and bring in a management consultant who actually knows something about delivering quality software (I realize that excludes most of the so-called "management consultants"...). Who knows, maybe it's not all lost yet.

With the risk of getting biled for a lousy conference report, I'll try and spill some of my XP2005 experience into bits and bytes. I haven't written about all of the sessions I attended, nor have I managed to capture everything that went on. Goes without saying, you need to have been there in order to get the full effect.

Coding Dojo

Laurent Bossavit and Emmanuel Gaillot ran a workshop around their concept of "Coding Dojo". Laurent and Emmanuel had started this weekly meetup in Paris, France where a small group of software developers interested in developing their programming technique while having fun would gather into a place somewhere--the dojo--to witness someone driving through a "kata", a choreography of developing a given piece of software if you will. Laurent and Emmanuel talked about developing one's understanding of three aspects of writing code: method, form, insight.

This concept of having a presenter code something from scratch on a projector in a rehearsed order of tests (test-first was one of the few rules they had laid out for presenting a kata) with a rehearsed implementation of each test and a rehearsed narrative talk to go with what's shown on the projector. The audience's part in this is to listen. The goal of the presenter is to get through the kata with everyone on board. Ideally, no member of the audience should at any time feel the need to ask "what the heck are you doing?", or "why are you doing that?", etc. The audience is also responsible for asking questions as soon as they fall off the wagon. Fast feedback, remember... This explicit focus on keeping everyone "on the bus" leads to a very simple suggestion for modifying your behavior when presenting a kata: don't assume, explain!

Laurent and Emmanuel also introduced a new term for a class of ping-pong development: "randori". Randori is basically "exploratory kata" where the whole group participates in carrying out the choreography. Except that there's no predefined choreography--you just go with your intuition. In practice, the group decides what they're going to implement (typically someone has written a set of acceptance tests before the meeting) and someone writes the first (unit) test. At that point, that someone will also make the first test pass and write a new, failing test before passing on the keyboard. The group will then go into a rhythm of getting the keyboard with a failing test they need to implement, implementing the test, and writing a new failing test for whoever's next in line. People would applaud upon each and every green bar. It's ping pong development. Nothing new, not nearly as enlightening as a regular kata can be, but loads of fun.

An interesting observation was made during the randori session we did in the workshop. The time spent without a green bar seemed to correlate strongly with the amount of chatter in the room. Spending more than 2 minutes on a given test and you could count on people starting to move their attention from the screen into something else. That's actually a pretty good indicator for many other kinds of situations as well. Keep an eye on chatter.

Regarding the community aspect of the coding dojo, the group in Paris had come up with a very nice routine for creating continuity. In order to keep the dojo going, they would start each meeting by agreeing on the next meeting. Similarly, they would not do a retrospective on the day's kata right there and then but would delay it until the next meeting. In other words, the agenda for each session would look roughly like the following:

1. Agree on a date for the next meeting
2. Discuss the previous week's kata
3. Vote today's kata/presenter
4. Run the kata
5. Go home

I haven't yet tried anything like the dojo myself but it would be interesting to see if that kind of a thing would fly here in Finland as well.

Weekly Refactoring Sessions

Jeff Nielsen of Digital Focus mentioned in a corridor discussion that they're doing weekly refactoring sessions with his team. They're essentially keeping a list of refactoring targets in an internal wiki, get together once a week to vote on what they're going to do from the list, and then go ahead and do it. This sort of a routine can be very useful if it works out--there's a lot of social issues to look out for, mostly to do with egos and perceived individual code ownership. I've seen plenty of teams keep lists. Most of them haven't acted on them, though. I was glad to hear that Jeff's team has.

Coming of Age

Jutta Eckstein's keynote titled "Agility - Coming of Age" brought up a bunch of perspectives on the (agile) software development world today that I hadn't really thought of until Jutta said it out loud. For one, the projects adopting agile methods are no longer the ones in severe crisis with nothing to lose. Projects are started anew using an agile method because it has seen to work and work well. Similarly, the projects adopting agile methods are no longer the simple, small, non-essential funds type of projects. Companies are betting their whole business on projects run with agile methods and being successful in doing so. On the same note, the projects are also not necessarily co-located--distributed agile software development seems to be a hot topic for many companies with global presence. Further evidence of agile methods approaching mainstream is that the German government has recently revamped their famous V Model for software development into "V Model XT". The "XT" stands for extreme tailoring and, apparently, it's not that far away from Scrum or XP--provided that you tailor your process in that direction.

Jutta's mention of distributed agile development raised some discussion on the type and volume of communication necessary to make distributed agile development a non-oxymoron. Obviously people's opinions and experiences were somewhat varied. Daily phone calls, netmeeting, whole-team video conference calls, daily hand-offs between continents with the whole team present... Whatever good techniques were mentioned, none of them are silver bullets. Using any of these tools does not guarantee a thing about the effectiveness of your distributed project team. Point. Blank. It's all about the people--your particular people.

The Elusive Business Value

John Favaro's keynote on the "elusive business value" was simply an excellent talk. In essence, John wanted to say that IT is myopic, nearsighted. IT in itself doesn't do much good in terms of revenue and profits. IT's influence on the company's success is indirect and John had come to the conclusion that many of us software professionals seem to forget that too often. As a true management consultant, John presented a diagram with four squares. The X-axis would represent the "goodness" of IT, and the Y-axis would represent the "goodness" of management. The lower left quartal would represent 0% increase in profit. The lower right quartal, with good IT but average management, would amount to some 2% increase in profit. The upper left quartal, with average IT but good management, would already bring a 400% improvement over the influence good IT alone has with an estimated 8% increase in profit. It's no surprise that the biggest increase in profit (20%) was in the upper right quartal with good IT and good management combined. These numbers are obviously rather meaningless (John had picked them up from some sort of a statistical analysis, I believe, but still) but the point behind the numbers is strong. IT alone won't do s*it for your investors. IT's role should be to support the strategies developed by management.

I made a note about looking more into value-based management after John's talk. Speaking of value-based management, John also noted that "XP has great decision-making processes that support value-based management" which is actually true for many other agile processes as well, including Scrum.

How to sell XP?

I also attended a workshop on "selling <your favorite agile method>". Some of the points brought up during the workshop included the gap between the customer and the buyer and, of course, the neverending annoyance called contracts. In short, and also according to my personal experience, the customers love agile. They love it. They might actually marry it if the church would let them. The problem is in convincing the buyer.

Some might argue that "selling XP" is not a problem--nor should be--and point out that you can succeed in being agile within a fixed-price, fixed-scope contract. That's actually what Reaktor has been doing a lot and with success. Actually, the scope often becomes unfixed rather quickly as the customers realize that their requirements specification was far from a perfect expression of their true needs and wants. Timo Lukumaa, one of our Scrum masters, will actually talk a bit about this in his talk.

When it comes to selling anything, we must be aware who we're talking to. That seems obvious but take a look around and observe how often people go into a "sales" situation with a wrong slant. For example, if you're going to try and convince someone to go with an agile process, you most definitely want to pick your focus according to the personal interests of the audience. Management cares about visibility--focus on the constant delivery, bringing problems on the table early, the savings from avoiding late bugfixes ($1500 a pop, according to some IBM/Gartner/whatever analyst report I once read), and so on. Customers care about functionality they themselves need--talk about how they're in control and free to change their mind. Developers care about quality software and personal safety--talk about the rhythm of delivery creating confidence, talk about the collective ownership, talk about people helping each other, talk about the techniques that help them produce high quality with less effort. Talk about them having the say on estimates. Talk about the customer listening to them. Remember who you're talking to.

The Sommerville keynote

You're probably familiar with "the Sommerville book", i.e. that textbook on software engineering that's going on its 10th edition or something and is being read by tens of thousands of undergraduates around the globe every semester. That Ian Sommerville did the opening keynote at XP2005, believe it or not.

Now here's the scoop: Sommerville believes Extreme Programming can deliver speed, quality, and lower cost. There you go.

His keynote, however, focused on the things he considers lacking in XP. For example, stressing that software projects are really socio-technical endeavours, he argued that systemic requirements often don't have a real customer, many users don't really want to be part of software development, and that customers and users are a bad source for requirements. I mostly disagree with that last one, but I do see his point.

Sommerville also suggested the creation of "why" documentation over "what" documentation (fully agreed), that we must define system boundaries (I think I disagree here), that we need to tackle architectural design explicitly, and that in general the literature needs to tell the whole story before XP can really become mainstream. Not surprisingly, the suggestion about architectural design raised some discussion among the audience...

Sommerville also talked about why we're still using waterfall even though practically everyone has said it's suboptimal and rarely works out smoothly. To quote him, "[waterfall] kind of works in sort of a way." That's so very true. People buying software projects are more afraid of getting yelled at for being different than for a constant stream of late projects with a constant budget overrun and a constant rate of churn. This is an aspect of the IT business I believe would improve significantly if people in charge of buying IT would be held accountable and given the power to do the right thing.

Training

I also dipped into the agile education and training panel on Monday. The panelists (Angela Martin, Steven Fraser, Rachel Davies, Mike Holcombe, Rick Mugridge, Duncan Pierce, Tom Poppendieck, Giancarlo Succi) were pretty evenly divided between industry and academia.

Some of the topics discussed included follow-ups. The academia has it easy. Myself, doing a lot of short training sessions with no chance for a follow-up, was very much interested in hearing other consultants' tips and tricks on who they're dealing with it. Rachel mentioned her contracts often being something like "three days a week for a couple of months", which sounded really good. In general, it's kind of obvious that long-term mentoring is more effective than a blitz training session. That's exactly why I've started suggesting multiple short sessions spread out instead of a single, intense two-day training, for example. One problem with traditional classroom trainings is that there's no think time. People need to relate what they're hearing and seeing into something their familiar with. Often times, the trainer cannot provide that experience productively.

The same problem is with measuring your own effectiveness. The follow-ups are also a chance for the trainer to get concrete feedback on how well he has managed to transfer ideas. One can do a lot of useful reflection based on purely self-observed details but it's no substitute for coming back a week, a couple of weeks later and see how the trainees have changed their behavior and routine.

Another issue mentioned was the "real world code" problem--how to balance between the "hello world" and the "seven-layer EJB architecture with integrations to a proprietary content management system, an SMS gateway, and what not." Unfortunately, and not surprisingly, nobody had a silver bullet to present. I've personally used a combination of simple examples to illustrate the technique, and more complex examples (focus on a single "difficult" technology) to illustrate feasibility in the real world. "It kind of works in sort of a way"...

There was also some discussion about letting students coach other students. Rick Mugridge is doing something like that in Auckland, and Alex (?) is doing something like that in Brazil. This is again one of those things that one can only wonder why didn't we start doing this earlier. It just makes sense. I suspect the biggest problems are, as in most cases when it comes to academic education, in how students are graded.

Tom Poppendieck also asked if anyone's training students to be customers before training them to be software developers. That really seemed to make people think--I could sense that a lot of people in the room were getting that "oh, hadn't thought of that" feeling.

When it comes to training (and this is what I had scribbled down as a "summary" when people were walking out from the training panel), there's no substitute for doing.

Kent Beck's ending keynote

Kent Beck was back this year to attend a panel on leadership and to make the ending keynote for the conference. In line with his recent theme, the keynote was titled "Up another notch" and focused on accountability (which is not about setting blame!). Tim Bacon had already posted a pretty good report of Kent's keynote so I'll focus on a couple of things Tim left out.

Kent mentioned the need for support. If it's you against the world, the chances are you won't come out as the winner. Everyone needs support. That's a given. The context in which Kent talked about support, however, was in "doing the right thing". It's easy to do the right thing if the rest of your team/department/company is also doing the same right thing. When they're not, we need support. We've all bit the bullet and survived a project where someone on the team has done more harm than good, and we've all had those moments when we've felt like that other guy is just too stupid to see what a wonderful idea we have. That's only human.

The insight that Kent gave me in relation to support was that the support doesn't have to be from your workplace. That really opened up a whole new dimension for me as I looked back a couple of years. Looking at all the discussions I've had about work related stuff with people who in many cases don't know Java from COBOL, I realize that those very discussions have helped me both vent my frustrations as well as guide my thoughts towards actions to take. I'm pretty sure I would've made a lot of worse decisions than I have if I hadn't had that support.

Kent also mentioned a couple of real life examples of how accountability means rendering an account of your actions. Google's "I feel lucky" is one such example. Simply by having that feature, that button next to the search field, Google is making itself visibly accountable for the quality of their service--the first hit must be the most relevant one. Another example Kent mentioned was how Agitar makes their own software metrics publicly visible. Although I can't find them anywhere on their website, the idea certainly is valid (although I'm not so sure about whether "metrics" is the best type of information to expose...) and worth considering if you want your company to produce quality software--make it clear that there's no room for bad quality.

Ok. That concludes my report from XP2005 (which was already two months ago...). I hope you got something out of it--I certainly did. I'm already looking forward to XP2006 next summer in Finland.

Starting with a couple of pointers into the blogosphere...

• Glen B. Alleman posted a briefing of a $3.3 Billion program which looks quite much like a briefing given by an agile coach to a regular software project.
• James Bach's thoughts on making intermittent problems reproducable was a nice read.
• Robert Watkins blogged about a nice-looking approach to maintaining test data in simple YAML files. I should probably try that some time.
• Jason Yip shrinked the manifesto into four suggestions. While I still kind of prefer the longer version, Jason's dumbed down version certainly speaks to me.
• Kevin Rutherford stresses the importance of asking the three questions. I'd like to stress *not asking* and instead suggest that it's better if everyone *tells* the answers to those questions proactively. It's already easy enough for teams to forget that they're not reporting to the Scrum master but to each other. I'm afraid focusing too much on asking, this small but significant difference will be forgotten.

Oh, and I recently got a domain for our Agile Finland seminars from DreamHost.com. They pack 8 gigs of diskspace and Ruby on Rails for $20 a month. Worth considering, I'd say. Oh, and if you'd like to support our new community site while signing up with Dreamhost, please consider signing up through the above link--that'll get me a referral bonus which comes off from the hosting costs. Oh, and don't forget to register for the seminar itself which takes place on September 7th in Helsinki, Finland!