More prep for Agile on The Beach: Reductio ad absurdem?

4 July, 2011 (20:20) | 7N Limited - UK news | By: Roger Marlow

Divide and conquer. It is one of the best tools in the box for solving problems, especially on software projects. Simplify the complex, get a toe-hold of insight into the incomprehensible, handle the unmanageable. All by breaking things down into their component parts. If it is still too unwieldy then decompose those parts into their sub-parts. Keep on going until you have something you can represent with an integer, a date, a reference, a label.  With the parts modelled we then have a host of containers for rounding them up: sets, arrays, lists, trees, stacks. With all our components modelled and reassembled we have our solution, our system. Yes, a problem modelled is a problem solved. Complexity tamed. How clever we are.

The problem with this kind of problem solving is that it concentrates on the parts. And it does not resist valuing those parts more than the whole. Indeed producing the parts by finding the fault lines through which we can cleave the problem is highly prized. It “cracks” the problem. The vocabulary, the spin, is telling us that the whole is the problem, whereas the parts are the solution. The means by which we find those parts turn problems into solutions. How valuable analysis is. Seeing any other way is difficult when you spend your formative years encouraged and trained by your parents and teachers in these techniques, you study and idolise the masters of the techniques and go on to earn your living through their mastery and application in a huge range of life’s trades and activities. And it is understandable then that it is the default way that we try to understand the world about us: by reducing it to its parts and valuing those parts above the whole. And if you are the kind of person who earns your living through application of those techniques then the world around you may well be the reality of software projects.

You won’t be surprised to know that there is a word for this kind of approach: reductionism. What might be more surprising is that it in philosophical circles it is generally considered a derogatory term. That is because the value, the magic, so often belongs in the whole, not the parts, especially when it comes to dealing with people, with life. Once we decompose it, the magic is gone. Take for example the simple pleasure of a joke:

“Standing in the park, I was wondering why a frisbee looks larger the closer it gets…then it hit me”

Thanks to Stewart Francis for this gem. So where is the humour? Which word or sub-phrase made us smile? Decomposing it only detracts from the humour. Indeed the humour seems to come from the whole sentence’s connection to an even greater whole, that of our collective experience, extending amongst us and into our shared past. Quite the opposite of decomposition. If you want to understand why that’s a funny line, or come up with a similarly funny line, decomposition is a problem not a solution.

What has this got to do with software projects? Reductionism may be a dirty word in comedy but it’s a universal solvent on software projects. Or is it? Is it reductionism that has beset these projects with plans, contracts, tools, legacy systems and siloed roles? I recently visited a large software organisation where management seemed to have looked at every conceivable role then analysed, categorised and labelled it. People were isolated into functional groups that were separately managed. All activity was codified in processes. Communication was reduced to the exchange documents via emails between anonymous email “front door” accounts. The managers claimed there was nothing that couldn’t be measured. And their measurements were telling them that their projects were taking longer and longer and costs were spiralling. Everything that was produced was unimaginative and regular. Several of the staff were on performance improvement plans that demanded, amongst other things, that they make estimates accurate to 10%. They had taken the magic of several hundred conscious, creative, free willed people collaborating and creating, looked through that to focus on the shadow cast on the wall behind, the objects, the processes, the categories and their labels, then turned that into reality. And that is the problem with reductionism in software projects.

This is not to say that analysis is bunk. What I am saying is that we need to complete the circle. Going from the messy, complicated, fuzzy, fluid real world to a crisp, clean, elegant, static model is only half the journey. We need to make the return trip, bring what the model taught us back to where it can inform the real world. Not in a container, or a joined-up policy, that is just more modelling. Nor an aping of the model. We need to bring the learning back to the world of people, relationships, creativity, empathy, intuition and emotion. Too often we are beguiled by our model, or the tool that holds it, prizing them above the reality from which they are derived. Even worse we try to make reality become the model, confine our activities to the process and let the tools operate us rather than the other way around.

In the Agile Manifesto we are presented with a series of couplets: individuals and interactions over processes and tools; working software over comprehensive documentation; customer collaboration over contract negotiation; responding to change over following a plan. At the start of each couplet is something in the real world. At the end of the couplet is a model, an artifact of reductionism. So Agile is valuing what happens in reality over what we can determine from reductionism. I couldn’t agree more.

Perhaps everything that Agile teaches us reduces to just one word: anti-reductionism. The irony is that this itself is reductionism. So if we are to take our own medicine we must not focus solely on the word and its abstract meaning but bring what it tells us back to reality and see if it can inform us in what happens there. Bear it in mind on your next software project.

Preparing for Agile On The Beach 2011

13 June, 2011 (16:16) | 7N Limited - UK news | By: Roger Marlow

I am fortunate to have a speaking slot at the excellent Agile On The Beach conference in Cornwall this September. I will be talking about the peculiar divergence in the way two sorts of problems are solved on software projects. I have been thinking about this for a while but I now need to turn it into something more concrete. As I do so I thought I would share my preparations.

I say ‘divergence’ but that is really just a diplomatic way of expressing my exasperation that one class of problem seems to be left entirely to chance while there is an all consuming obsession with the other class.  And what is more I believe that the deprecated class puts projects at greater risk.

The two classes are, as always, people problems and technology problems. It is a well worn argument and indeed one I have chipped in on in the past in this blog and elsewhere. So is there anything really new to say? Regarding solving it I fear not, but regarding understanding it better I think there is.

In reading around the subject I have come across two extraordinary authors, Iain McGilchrist and Ray Tallis. Neither work in software (they are both doctors) and they have diametrically opposed views but I have learnt a lot from both of them. McGilchrist is a Consultant Psychiatrist and teacher of English at Oxford now living on the Isle of Skye. In his book “The Master and his Emissary” he asserts that the divided nature of the brain and the difference in what each hemisphere contributes has shaped our world. He argues that the left hemisphere deals with abstraction, reduction to components and analysis, logic, classification and dealing with non-living objects. Meanwhile the right hemisphere provides broad context, intuition, empathy and a sense of the here and now. He suggests that ideas originate in the right hemisphere and are passed to the left for analysis and elaboration before being passed back to the right for contextualisation and re-presentation. For example putting ideas into words requires the left hemisphere as the right cannot articulate speech. The point of McGilchrist’s book is that the left hemisphere is becoming dominant and this is leading to all sorts of problematic repercussions in culture, politics and society. To use his metaphor the emissary (left hemisphere) is usurping its master (the right hemisphere).

Although McGilchrist is writing about history and culture not software development the parallels are striking. Abstraction, that essential tool for software development, should remain firmly in the toolbox when it comes to people issues. This left brain thinking, as McGilchrist would have it, takes us away from the specific situation and the here and now with its feelings and emotions. It replaces all that with models, cold facts, simplifications, objects, causes and effects. Useful for software development, alienating for people. If by getting better at the technology of software development we continually reinforce left brain thinking does this diminish the right brain contribution? Perhaps so because in McGilchrist’s scheme the right brain is adept at relating with people. To remind ourselves why this is a problem, to paraphrase Monty Python’s sketch “What have the Romans ever done for us?”, regarding people in software, apart from: commissioning it; selling, buying and paying for it; designing, building, testing and managing it; using and training others to build, test, manage, commission and use it; and supporting, improving and maintaining it; people hardly have any relevance to software at all!

So one thing that I want to bring out in my talk in September is that in all of the debate on software methodology, Agile or otherwise, intuition and empathy along with many other values in relationships make little or no appearance. Therefore is it any wonder that we are still not handling people well in software?

And what of my second author, Ray Tallis? Like McGilchrist he is a doctor, a retired Professor of Geriatric Medicine and a clinical neuroscience researcher. In reading around the subject I came across his book ‘Aping Mankind‘. He starts by making the astonishing claim that the brain is not responsible for consciousness. Piece by piece he dismantles pretty much everything I have ever understood, as a lay person, to be the workings of the brain and its relationship with our conscious lives. It is a courageous, brilliant and devastating critique of neuroscience and evolutionary theory as applied to human consciousness, behaviour, culture and society. Amongst other things he argues that locating aspects of behaviour in ‘compartments’ in the brain, usually accompanied by an fMRI scan, is the modern equivalent of Phrenology: “feeling your bumps” or explaining behaviour from the shape of parts of the skull.  And to my horror he cites McGilchrist and his “The Master and his Emissary” as its most extreme proponent!

Back to the drawing board for my talk? Well not entirely. I am still convinced that there are common patterns of thinking and behaving on software projects that we have developed, perhaps over developed, to deal with technology but that have exacerbated the problems people encounter when working together. While their ultimate origin is hotly debated by greater minds than mine, how to practically deal with them is still something to which we can contribute. While they are not currently well articulated I hope to indicate what they might look like in the next blog posting.

7N’s inaugural social event

31 October, 2008 (15:18) | 7N Limited - UK news | By: Pierson Broome

Soiree 1.0 was the inaugural 7N Limited UK event and we were delighted to welcome 40+ consultants to the Institute of Directors’ Atium facility at New Broad Street House in the City of London - see ‘7N Events’ page for more details.

Working code, failed project. Agile can do better.

25 October, 2008 (18:27) | Uncategorized | By: Roger Marlow

Let’s be honest, some Agile projects fail. Some projects produce working software but the business cancels anyway. See Mitch Lacey’s courageous talk on infoq.com for an example. Even superstar teams are not immune; there is even debate about the outcome of the seminal C3 project. I have experienced similar projects which seem to win the battle but lose the war, or others that deliver the software but IT and the business may never speak to each other again. There is still no silver bullet and agilists make no such claims. Martin Fowler in his bliki entry about C3 says “its cancellation proves that XP is no guarantee of success”. Quite right.

But that shouldn’t be the end of the story. In my opinion there is a substantial class of failed projects where the failure is dismissed because it is attributed to “politics”. What are we doing to improve that situation? The tacit acceptance of failures due to politics, or at least their dismissal as being outside the influence of IT and being somehow a result of unfair external factors reminds me of the attitude to the failure of waterfall projects before we knew there was something better. Back then it seemed absurd to argue with the logic of the process and when problems arose they were explained by transgressions from the logical path and therefore not a problem of the process itself. Efforts to avoid straying from the process exacerbated the problem, albeit unknowingly at the time.

The realisation that led to the new agile processes was that waterfall’s logically attractive process was susceptible to human factors. The new methods were less prone to their human operators’ foibles and to a certain extent even exploited them. For example the project could get a fillip every two weeks at the end of iteration, not just once before the final deadline. Iterations acknowledged the need for manageable increments in system complexity prior to integration and a stable working environment. Story cards were notably tactile. Ten years on, the failure of some agile projects in politically difficult environments is surely pointing to susceptibility of process to human factors, although admittedly more intractable ones.  To state my premise clearly, I believe that a class of projects fail even though they use agile methods because there are even more adverse human factors than the current methods address or exploit. If that is the case then I see no reason why we cannot make further innovations to improve project success rates once we have understood what those human factors are.

What is of concern is that the majority of innovation in the agile space is expended in other areas: on tools, technology and the incorporation of Lean to eliminate waste in the process; very little of which addresses human factors. And the tacit acceptance of wider project failure due to behavioural, cultural and political issues, often characterised by the derogatory use of the words “politics” and “culture”, is leading to a situation where we don’t think it is our problem to be solved. The community seems to be saying: we have the answer, the process is logically correct, politics is someone else’s problem that prevents them from taking advantage of our process. And that reminds me of the attitude to failing waterfall projects before everyone realised there was an alternative. Furthermore, the continued polishing and embellishment of the tools and techniques in the agile kit bag while this goes on is akin to the embellishment that increasingly encumbered waterfall processes.

Returning to the premise, figuring out the precise nature of the human factors will be important. Broad categories such as politics and culture are too coarse to be useful. We need to become, as Daniel Goleman would put it, emotionally literate. We need to be conversant in the language of emotion, feeling and behaviour of the individual and the group. I expect a goodly portion of the readership to now roll their eyes heavenward, and that is precisely the dismissive reaction I want to reverse. These issues are causing projects to fail and not only are we not even conversant in the language to talk about it but we don’t think it is a language we should learn. But just like fixing a bug in software, precisely describing the problem leads to a much a better chance of understanding and therefore providing a solution.

The first step to improving project success rates therefore would be to equip ourselves with the vocabulary of emotion, feeling and behaviour. I would expect it to come from fields such as psychology, sociology and politics, in which case so would a cannon of work studying the attendant issues and how we might address them. So in some respects the problems may already be solved in those fields. The work for us in the software development community is to connect that learning and insight back to our existing agile software practices. We already see agile coaches reading books on NLP. In the context of this posting that is an encouraging sign. But we have to go much further. NLP is a start and the broader field of psychology should be a happy hunting ground too. But if NLP and psychology have insights to problems of bullying, fear, hubris, self-preservation, tribalism, group-think and every other behaviour and emotion that we experience on projects, then we have to connect that knowledge back to new innovations in agile practices which are specific, named and ultimately adopted by the whole development community. Developers need to be as conversant with it as the coaches.

Every agile developer can reel off practices such as pairing, refactoring and continuous integration. If we continue the journey of Agile development, what will be joining that list to deal with all the aspects of projects that failed because of what we currently glibly refer to as politics? We know how to write working software - how to win the battles - let’s continue the journey and figure out how to win the war.

The Contractor’s Viewpoint

15 October, 2008 (11:43) | 7N Limited - UK news | By: Pierson Broome

Recently, we were talking with 7N Collective member and independent contractor Ben Hughes about his experiences of ‘going it alone’ and we felt that his comments might make interesting reading for anyone thinking about doing the same – over to Ben…

Being a Contractor has many upsides.  Independence; real choice over the work you do; financial freedom.  However it’s not all a bed of roses.  In uncertain markets such as we find ourselves in today, companies tighten their belts, putting Contractors in the firing line for redundancy.There’s often a ‘them-and-us’ attitude to contractors in the workplace, resulting in some people never really delivering on their full potential.  Also, there’s the minefield of financial regulations, with self-managed limited companies, umbrella companies, managed service companies - all with their own unique interpretations of the subtleties of tax law, adding to the risk of going it alone.

However, one of the biggest deprivations of being a contractor is the isolation.  Your friends are just IM characters, or voices on the end of the telephone.  The banter in the office is between you and your cat- if you’re lucky, maybe your dog.  Your home becomes a prison but for a few networking meetings, and the odd stumble onto a distant friend on Facebook.

7N is different in that the guys are doing their best to address these issues by creating a real community around the talent pool that’s being assembled.  Sure, there are likely to be some that like the solidarity that Contracting brings, but some - such as myself - thrive on people.  Variety, social and otherwise can inspire great creativity, bringing fresh ideas about how we can create the same sense of community you felt when you worked on the Dream Team (for those that have, you know what I mean) whilst at the same time harnessing the entrepreneurship that makes you leave the comfort of a regular salary for a life less ordinary.

So, if you’d be interested in meeting up at events such as 7N’s Soiree evenings to talk with like-minded folk who are also part of the 7N network, and hear someone else’s entirely different view on their life, I’d urge you to consider and preferably contact 7N – theirs is a refreshingly different way of doing business.”

Ben’s views are certainly not unique – we hear the same point of view time and time again. And 7N’s desire to create a social aspect to contracting is resonating loudly. We’d be delighted to hear from other people in a similar position – and also to help anywhere we can. Do please get in touch.

A closer look at the Cheaper Talent Hypothesis

14 April, 2008 (10:25) | 7N Limited - UK news | By: Roger Marlow

In Martin Fowler’s bliki post he asks: are more expensive programmers actually cheaper? His hypothesis, which he calls the Cheaper Talent Hypothesis, is that talented programmers cost more but if that talent gives you sufficient extra productivity then the benefits outweigh the costs. As Martin says, you cannot prove it but it is a view which is widely accepted. He uses his hypothesis to justify the higher rates charged by ThoughtWorks:

In case anyone hasn’t noticed this hypothesis is a key part of our philosophy at ThoughtWorks. [...] We believe we actually end up cheaper for our clients, even though our rates were higher.

Now I am big fan of Martin and I used to work for ThoughtWorks too, but there is a problem with using this hypothesis to justify high rates because although the fees charged by consultancies pay for talented programmers they also pay for the cost of running the consultancy company. And the extent to which the consultancy itself contributes to your productivity is not part of the Cheaper Talent Hypothesis.

The question then is how much of the rate is not for talent? If you buy from a consultancy you might like to ask them, but don’t expect a straight answer because it is usually a closely guarded secret. Even so, it is not hard to estimate with openly available data:

Advertised salaries for programming jobs at consultancies: £40k - £85k, average: £63k. Rates charged to clients by consultancies for those programmers: £600 - £850 per day, average: £725. Assuming the consultancy keeps a programmer charged to clients for 45 weeks of the year, they receive average fees of £163k. Implied cost of running the consultancy per programmer: £100k.

So we estimate that on average for every £100 paid by a client to a consultancy for a programmer only £39 goes to their salary. Assuming £39 is above average but justified by the hypothesis, then what is justifying the other £61? No wonder Martin goes on to say:

Of course, we do have difficulty persuading many clients [that we actually end up cheaper for our clients, even though our rates were higher].

Clients might be more easily persuaded if talented programmers were only available from consultancies, but they are not. Or perhaps 61% is the smallest margin that a consultancy can run on, but it isn’t (ours runs at 25%, others have managed even less). Perhaps by belonging to the consultancy the already talented programmer becomes even more productive, enough to cover nearly twice their own costs. But that doesn’t happen either, or at least not to that extent. And of course the main premise of the hypothesis is that it is the talent of the programmer that really makes the difference, not the consultancy that employs them.

What about teams? As Martin rightly points out talented teams can be smaller and therefore cheaper. You will likely get better quality code, an earlier release, lower cycle time and even a better product. I agree with Martin that it is a highly compelling argument for hiring a talented team, even if you have to pay more. Yes, pay above average to a talented programmer, pay that premium for every member of a talented team, even pay a consultancy to find, hire, retain and manage them, it can be worth it. But don’t pay more than you have to, especially for the consultancy fee. If you can get the same talented team, but for less, then it doesn’t matter whether or not you can measure productivity or quality, you will simply be saving money.

Martin goes on to say that the Cheaper Talent Hypothesis is far off being accepted by the software industry as a whole, as evidenced by the difference between premiums for contracting fees/salaries and productivity, he says he accepts the situation but doesn’t like it. There is no measure of productivity but he uses a factor of 2 for example. But if a programmer received the average £725 per day that their consultancy receives their salary would be over £163,000. That is more than seven times the salary being advertised for junior developers on Monster today. The cost premium is there, but it is going to the consultancies, not their talented programmers. That is what we don’t like and we don’t accept either. By running a consultancy at much less than 60% margin the savings can be given to programmers as higher salaries or to clients as lower fees, or hopefully both. And in case anyone hasn’t noticed that is a key part of our philosophy at 7N.

7N delighted with initial consultant-community response

3 April, 2008 (10:26) | 7N Limited - UK news | By: Pierson Broome

logo2.jpg

Having only begun operations in February 2008, 7N has been overwhelmed with the positive response company members are receiving to the concept and business model structure.

After just 8 weeks of operations, the 7N UK network already has more than 25 independent consultants and contractors, all of whom have said that they welcome the introduction of a consultancy created to cater for the needs of the Agile (and beyond) community.  Having an organisation focused upon seeking out the types of ‘cool’ work that such community members thrive on and being treated in a fair, honest and individual way seems to resonate loudly with many people and it is these talented individuals who are pledging their backing to 7N in the UK.

The 7N model doesn’t restrict network members through exclusivity or competition clauses, rather it simply encourages independents to want to work with a company which in turn understands the needs of the individual yet can also contribute to the discussions surrounding contemporary IT and business themes.

Pierson Broome, UK Consultant Manager at 7N:

“Having spoken to many, many independent contractors at numerous events, social and business-focused, it appears that 7N’s arrival fills a distinct gap in the UK contractor market.  The response that we are receiving is fantastic, with a high number of contractors asking 7N to represent them in their work endeavours.”

In addition, even after such a short period of operation 7N is on the verge of signing its first customer contracts during April, placing a number of independents in differing roles at a variety of organisations.

7N launches in the UK

25 March, 2008 (14:52) | 7N Limited - UK news | By: Pierson Broome

7N Logo

The successful consulting concept of 7N A/S Denmark has now launched in the UK market.

7N matches the IT development, coaching, testing and consulting needs of customers with the highest quality independent contractors. 7N’s rich network of consultants ensures the very best practitioners and theorists from contemporary fields from within the world of IT, not just people following a trend, but those actually defining the future of technology.

Unlike so many agencies which are content to simply match perceived requirements with names and data-tags from a vast database of CV’s, 7N takes time to:

  • understand the customers’ real requirements - and not just at a superficial technological level
  • Build a strong, lasting relationship with the customer
  • get to know - personally - all of the consultants in the 7N network
  • match all consultants with only those positions and contracts that truly excite them
  • ensure that the negotiation, billing and commission contract is wholly transparent to all parties.

7N’s network contains experts in the fields of Agile, Lean, PRINCE2, APACS payments…. the list goes on and on. What ties almost all of these individuals together is the desire to contribute to business success through the use of innovative, contemporary technologies and concepts.

The path to 7N’s first UK contract has been extremely swift - from a February start-up, the first consultants are being placed with customers in less than 8 weeks. Response to the 7N concept as been extremely encouraging, with many new consultants joining the network every week thus far.

If you are a consultant who wants (or needs…) far more than your usual agency is able to deliver or a prospective customer who simply wants access to the best independents in IT, contact 7N:

  • Telephone: 0870-766 7715
  • Fax: 0870-766 7725
  • e-Mail: uk@7n.com

We very much look forward to hearing from you.