Achieving Consensus – It’s so funny, how we don’t talk anymore

Walk into any office and there can usually be found a surplus of opinion and a dearth of consensus and agreement. While the crowd may be wise, as individuals we all know we’re right and damn any fool who disagrees with us.

A quick perusal of the comments section of any, even moderately controversial post, and you’ll see an echo chamber of hundreds of people carefully explaining why each other’s opinion is completely and hopelessly wrong.

The rarest phrase on the internet, is – “That’s a compelling argument, you’ve completely changed my mind”

In fact, the comments sections of most blogs are now so poisonous that the creators of the content, and the majority of its readers, don’t even bother to “Read below the line”.  Worse, people with interesting and unusual opinions are intimidated into not broadcasting them in the first place.

I’d like to say we can all agree that this is a terrible state of affairs, but as I’ve explained, that’s merely an invitation to be pointed at 100 other blog posts explaining why I’m wrong.

So all we can say is that opinions are cheap, and getting consensus is the hard part.  For a contemporary example – the recent Paris climate change talks are both simultaneously The World’s Greatest Diplomatic Achievement, and A Squalid Retrenchment.

Picking battles and getting permission

I wanted to write this post, not because I have the answer to generating constructive discussion on the internet (I wish).  But because I think teams and individuals (including myself) struggle on the projects we work on every day – not for want of ability but for want of consensus and clarity on what we want to achieve.

Because we all instinctively understand the difficulty and energy required to get a group of people to agree on just about anything – the path of least resistance is to avoid the discussion in the first place. Indeed by having the discussion you expose yourself to the risk that your argument will not carry the day, and you’ll be forced down a road you’d prefer not to be on. So it’s easy to understand why tough decisions and “honest” conversations are avoided and “difficult” people worked around.

The old cliché “Pick your battles” or my personal favourite “Better to ask forgiveness than get permission” are undoubted truisms, but they come with a cost.  By avoiding the battles and not getting the permission, in other words not getting consensus, you can find yourself isolated when you hit the inevitable bumps in the road.  As you’ve neatly provided those excluded from the decision with a convenient scape-goat.  eg – “Well if he’d asked me, of course I’d have told him that was a terrible idea.”

Or as Sun-Tzu put it more pithily:

“Victorious warriors win first and then go to war, while defeated warriors go to war first and then seek to win” 

Are you thinking what I’m thinking?

I’m certainly not advocating that every decision needs to be ran past the team, but decisions that will impact more than one or two people or that will have a significant impact on what is delivered,  and when it will be delivered, should be discussed as a group.

Keeping the team and the business involved in the decision making process will cost you time and you’ll inevitably have to slaughter some of your personal sacred cows, to reach an approach everyone can agree on.  However, the prize is worth the pain.

That prize is a clearly articulated plan that both the team and business believes in and much more importantly – are invested in.  Reason being – because you’ve arrived at the plan as a group, the group should want it to succeed as they all have some some skin in the game.

The drive and pressure to “Get stuff done” and the small windows many agile methodologies allow for meetings and planning between sprints means it’s easy to skip getting the team bought into the plan and approach. This is a mistake.

Rather than thinking of getting consensus as an exercise in cost/pain/stress think of it as an extremely valuable deliverable and vital part of leading a team (unfortunately the cost/pain/stress are still there 😉

In the end it’s just all about talking, and remember to heed Cliff’s warning…

Advertisement

Agile – Dealing with complexity by avoiding chimeras

In my last post there was a certain amount of “Son, I’ve seen more large projects, than you’ve had hot dinners” attitude.  So in this blog post I offer my thoughts on dealing with complexity and large projects.

Defining complexity

The problem with complexity is that it resists easy definition, but you know it when you see it, or more likely you know it when you’re in a middle of massive deathmarch project with no end in sight.  However, for the purposes of this post I shall define it along two axes: time and chimera component.

• Time –  Any project or task you think will take your team or organisation more than 3 months to complete can be considered complex.
• Chimera Component – Like the many-headed beast of legend, the greater number of features/requirements/scope – the more complex the project.  Keep adding features (or heads) and the project will eventually eat you.

Of course it’s the chimera component that is the true test of complexity.  The building  industry implicitly understands that it’s a bad idea to build a house that is also a shopping mall, or an office block that is also a bridge.  Likewise the car industry realises it’s a bad idea to create a car that’s also a boat, and if you do create the car-bo, it’ll be expensive and be a rubbish car and a rubbish boat.

Yet in the software industry we create these types of chimeras all the time.  The #1 offender for signing off chimera projects is the government, who you would imagine, after years of IT fiascos NHS, ID cards etc, would think – maybe we should reign the scope in a bit.  According to the Independent the British taxpayer has squandered £26Bn on failed IT projects since 2000 (and to think we’re worried about banker bonueses).

Big business runs government a close second, but the failures are less widely publicised.  In fact a recent Computer Weekly article says that large IT projects are 20 TIMES more likely to end in catastrophe than any other type of project.  All disgraceful and embarrassing statistics for all of us working in the industry.

Even a stopped clock tells the right time twice a day

However, from time to time we manage to deliver a success.  For example, I am in awe of London’s Oyster Card system which was largely delivered on time and on budget, contains 1.5 million lines of code, and successfully deals with over 7 million commuter journeys per day.

So what made Oyster a success and NHS a disaster?  I’m not pretending I’ve worked on or managed one of these ultra projects, but here’s my views on pragmatic steps you can take to try to avoid calamity.

1 – Turnover is vanity, profit is sanity

First a quick anecdote.  Back in 2002 the company I was working for was reeling from the dotcom bubble bursting and the silver haired boss of the American parent company was dispatched to layoff some of us minions.  He gave the usual speech with the usual BS about how this was more painful for him than us etc.  However, he made one point that has stuck with me to this day and was one that I hadn’t really considered before

All businesses exist to turn a profit

It looks obvious when written down, but while I was worrying about the minutiae of my Delphi project (those weren’t the days), I hadn’t thought of the wider context of – has this project got any chance of turning a profit.  Given the layoffs – the answer was clearly no.

However, many working in IT act as if they have all the time in the world to work on vanity projects, specious methodologies and dubious features without any understanding that they are based in the real world of capitalism and profit & loss.

So first analyse your project through the lens of profitability.  Does the timescale, and price for the project look realistic, and if not cull those features that appear less useful to the user and shorten the delivery and scope.

Item: “[NHS System] intended as a complete “big-bang” replacement for the many and varied existing EHR systems”

2 – Phased delivery of working USEFUL software

Agile methodologies all have phased delivery built  in.  At the end of each iteration you must deliver working software.   However, working software is not necessarily useful software.  If you have kicked off multiple features simultaneously you may deliver 10 features that are all 5% complete and are in actual fact no use to anybody.

So using step 1 you’ve hopefully identified the features you think are profitable ones, now only start work on one or two of those so you should get the team to deliver it at more like 60% completeness.  So the user can assess it straight away, feedback on it etc.  You should then endeavour to get that feature to 100% completeness in the next iteration.

Item: ” a crucial decision [in Oyster’s success] was to revise the original plan of a “big bang” introduction in favour of a phased roll-out”

3 – Three month mini projects

Split the project into a series of 3 month mini-projects, each 3 month deliverable should aim to deliver part of the project but containing only complete features.  The customer can see progress and potentially has a walkaway point, or can reprioritise featues based on progress.

Important – Assess critically if the project still makes sense, is still on track to make a profit.  Assess if the features sitting at the top of the backlog are still high priority.  Has the tech changed, has the environment changed, is there still buy-in from the customer etc.  Has the market changed, eg Ipad platform now a priority – so ditch Flash etc.  Take action and kill features or parts of the project that no-longer make sense or threaten to derail the project.

Item: “Large projects nearly always fail unless broken into components that can be delivered step by step”

4 -YAGNI and DTSTTCPW must be your religion

YAGNI and DTSTTCPW are a much better guide than the latest fads within agile.

Everyone says it, but you must keep the team tightly disciplined on scope creep.  It’s so very easy to slip into Gold-Plating features that don’t need it and worrying about optimisation early.  It’s surprising the number of projects I’ve been on that have a massive Oracle DB backend and are probably only doing a few 1000 transactions per day  As Eric Ries says deliver the minimum viable product.

Item: “process has greater emphasis than outcome and that’s not going to get a project over the line”

Conclusion

So while we can all agree that small is beautiful when it comes to IT projects the rub is that if you want to make some serious money, you’ll inevitably have to sign-up to some form of complex project.  Ie a 3month + project with a large chimera component.

However, if you manage complexity with an eye towards profitablility, delivering working features (not just working software) that have survived the Occam’s razor of YAGNI and DTSTTCPW and you aim to have an actual product every 3 months you can do a lot to mitigate complexity.

Of course we’re our own worst enemies.  If we’re asked to tender for the next NHS system, instead of questioning the scope, we’ll put in a low bid and hope the government is stupid enough to sign it off.  When we walk away £10bn richer for a failed delivery, we know we won’t be punished for our failures.  Equally people putting £10bn contracts out to tender should by now know a lot better and stop putting such massive scope into them.  But that’s another story…

I do feel agile (applied without all the ceremony and fluff) is the best solution we’ve got to delivering working software into the hands of end-users quickly and to a high-degree of quality.  We can cut the heads off the chimera one by one, and win.

Agile punks – Go write an app

Last week I ranted posted about how some members of the agile community are engaging in pointless debates about complexity theory, kanban maestros & scrum blockades. I pointed to a post by Liz Keogh as an archetype of the genre (as I said to Liz, there’s plenty of other authors I could have used).  Liz asked for some feedback and was generous enough to agree that I had a point.

If we try to ignore the first commenter, who hilariously and spectacularly, misses the point by wondering if the issue is a “complex problem, possibly even chaotic”.  I think Liz has drawn all the wrong conclusions, about the state of the industry.

It is the best of times, it is the worst of times

In my 14 years in the industry I’m confident that there has never been a better time to be a programmer or to work in IT.  But thanks to a community of well intentioned “agilists” many in the industry are flagellating themselves into believing themselves hopeless programmers.

I live in Scotland and there’s an old joke about the definition of a Scottish Calvinist:

“Someone who is paralysed by the fear that someone, somewhere, is enjoying themselves”

Similarly the definition of a modern programmer could be:

“Someone who is paralysed by the fear that someone, somewhere, is doing BDD better than they are”

Many programmers are so obsessed with writing better unit tests, understanding the problem better, worrying about their kanban wall, working through Martin Fowlers pattern library, trying to be “masters” not “journeymen” that they’ve forgotten the basic joy of cranking out some code and solving a users problem, and are paralysed by the fear that they’re doing it wrong or poorly.

Ironically, it’s never been easier to write some code and get it downloaded and used.  Individual devs can create and upload apps to the App-store for a potential audience of millions.  Cloud computing means the home coder can create apps that can scale massively on demand for an initial outlay of a less than £20 quid.  While you’ve been worrying about your BDD adverbs the iFart dev just made $40,000.

It’s not that improving your skills isn’t important, it just shouldn’t become all consuming and derail shipping software.  Apart from my blog posts, there is no such thing as perfection, and guess what, you learn from your mistakes. And we certainly shouldn’t be putting up barriers to welcome all and sundry into the industry, it sure beats working at Tesco.  If you are a “master” your one mission in life should be to make it easier not harder for “journeymen” to improve (BTW I loathe this master/journeyman patronising nonsense, it just succeeds in making programmers more paranoid).

Are you as punk as Ward?

I like to think of Ward Cunningham, Kent Beck, et al as the first punks in our industry.  Just like the original punk rock was a reaction to the complex and multi-instrumented prog music as epitomised by bands like Yes and Emerson Lake and Palmer.  XP was a reaction against the “analysis paralysis” caused by, as they saw it, the complex and formal software development methodologies of the day.  By turning things like TDD, Small releases, CI up to 11 they managed to break the deadlock and start rocking out the code.

XP was the forefather to agile, and sadly since those heady simple days of XP, we’ve slowly been adding the complexity back in – until I overheard someone say recently “I’m not sure I understand the definition of a story, properly” as if that were important.  We’re now in a state of “over-analysis paralysis”.

Anarchy in the IDE

I’m not the only one thinking like this, Jeff Atwood recently tweeted:

“all this process stuff at #ordev is boring. At Stack Exchange, we have one process: kicking ass and chewing gum. And we’re all out of gum.”

Zed Shaw sends up the agile community far better than I can using a tad more colourful language at Programming Motherf***ker

So I propose we need to bring back some of that original punk rock spirit.  There was a famous punk t-shirt which said “Here’s 3 chords, now form a band”

Similarly in programming we could do with something similar:

1. Here’s an IF statement
2. Here’s a loop
3. Here’s an editor
4. Now write an app

Or as Johnny Rotten said “Don’t accept the old order. Get rid of it.”

Agile Manifesto – The First Amendment

Riddle me this:

Q – What’s the #1 difference that an agile approach gives you versus old-skool software development methodologies (hint clue’s in the name)

A – Quickly or “agilely” getting working code into your end-users hands

It’s this answer that an awful lot of people miss, and it’s why I propose the first amendment to the Agile Manifesto:

“Any agile process should strive to get working software into the hands of end-users as quickly as possible.”

Let me now explain my reasoning:

There is an awful lot of guff talked about agile, for example, I’ve held my head in my hands after reading some recent blog posts discussing Cynefin and complexity thinking.

This sort of pontificating rubbish is what gives the consulting industry a bad name.  The main elements of developing software are the same as they’ve ever been from the Waterfall method on down – Requirement gathering, Design, Implementation, Verification (Testing) and Maintenance.  Agile (in any of its forms) isn’t a silver bullet and doesn’t magic away any of these elements, it’s just a different way of mixing these basic ingredients to build the same cake.

To stretch the baking metaphor – proponents of agile will tell you that with a Waterfall methodology you might end up with a Dundee cake, when the users really needed an Eccles Cake.  This is exactly right, but the fundamental point is you’re only going to know the users wanted an Eccles Cake if you show them the first slice of Dundee cake, and they say “I’m sorry but you have misunderstood my requirement”.

So what? The point is that only the end-user can know if the product works for them.  However, often the end-user never sees the product until the end of a 12 month development phase, much like the Waterfall method.  What gives?

This problem is common in web-development.  You win a nice fat contract to build out a site for your customer (lets call them AcmeCorp).  The dev-team apply their shiny agile methodology and ship working code to AcmeCorp every few weeks.  AcmeCorp looks at it and supplies comments etc, you ask for clarifications, get more requirements and so on and so forth.

12 months later AcmeCorp launches their shiny new site, but it doesn’t get the traffic they expected, or users don’t like the features, or it doesn’t appeal to the target demographic.  Spot the problem?

You might as well have managed the project Waterfall style, you did 12 sprints/iterations/leans/kanbans/cynefines, but from the End-User point of view, it was a big-bang delivery.  You’ve been doing CAT (Customer Acceptance Testing) when you should have been doing UAT.  Arguably there may have been some benefits about getting feedback from your Customer, but if you are not getting feedback from End-Users as early as possible in the project, you’re asking for major trouble.

Conclusion

Agile teams must strive to get feedback from the end-users, not just the customer, as early in the project as possible.  As the feature you and the customer are convinced is the killer-app, may well be something your end-users don’t care about, or don’t understand how to use effectively.

UAT is the name of the game, and Agile teams should impress upon their customers the importance of putting the product under the noses of the real end-users as early as is feasible.