back to article Agile consultant behind UK's disastrous Common Platform Programme steps down

The head of the agile company instrumental in the UK Ministry of Justice's disastrous Common Platform Programme has stepped down. His departure follows a Register exclusive exposing the project's failings. Jeremy Renwick, chief exec of Agilesphere, held the role of Programme Manager/Agile Coach from February 2015 until April …

Page:

  1. Anonymous Coward
    Anonymous Coward

    "It is a sad tale of missed opportunity. A senior civil servant with no experience of running a complex IT programme; an overbearing Government Digital Service with misplaced philosophical goals..."

    ..And bears, they shit in the woods.

    1. John Smith 19 Gold badge
      Unhappy

      "on a time and materials basis with no hard deliverables or deadlines "

      Isn't that what con-sultantancies and con-tractors dream of?

  2. Voland's right hand Silver badge

    Agile and government do not mix

    Using agile to deliver a government deliverable cannot work.

    Procurement, finance, contract selection and deselection, etc are as waterfall as waterfall gets. Add to that the inevitable pre-election "what we have delivered" fixed dates and you get an environment where Agile has no logical function.

    You can use agile as much as you like internally, but the actual shipment of stuff to gov (of any shape and form) and gov's side of planning the programme is not agile territory. It cannot be.

    So the person to fall on its sword should not be the Agile consultancy, that should be the mandarin which decided to go all-buzzword compliant and run it as agile.

    1. Lee D Silver badge

      Re: Agile and government do not mix

      You used "agile", "deliverable", "waterfall" and "mandarin" (in the context of bureaucrat) in the space of one comment.

      And you complain about others being buzzword-compliant?

      1. RyokuMas
        Happy

        Re: Agile and government do not mix

        House!

        /buzzwordBingo

        1. Locky

          Re: Agile and government do not mix

          Damn,

          I had "sprint" and "Did you snapshot all the servers last night? We've had an unexpected result and need to roll back" left on my card...

    2. Anonymous Coward
      Anonymous Coward

      Re: Agile and government do not mix

      Using agile to deliver to government can work, and does. I've done several Agile projects for government and they've all been successful.

      Even with a legislative deadline to meet, Agile is beneficial. It might look like a legislative deadline means fixed schedule and fixed scope, but in reality it doesn't mean fixed scope. The legislation will tell you what you have to do, but not how. With Agile the customer has constant visibility of progress, and it should be clear to them whether you're on course to meet their goals, or whether scope needs to be reined in and they need to settle for a simpler solution.

      With waterfall, the customer's taking your word for it that everything is on course. They hope you're being honest and realistic with them. You hope they'll be pragmatic with you when you tell them requirements need to be cut (good luck with that if you committed to a fixed cost, fixed scope waterfall contract).

      This failure probably isn't an agile vs waterfall thing. If they get 30 months into an "agile" project and all they've delivered is a glorified appointment booking system, clearly they haven't been doing agile very well. I expect if the same team had tried to do it as a waterfall project they'd have failed at least as badly.

      1. Matt Bryant Silver badge
        Stop

        Re: AC Re: Agile and government do not mix

        "....With waterfall, the customer's taking your word for it that everything is on course...." Male bovine genetalia. With proper waterfall you get a comprehensive reporting system that keeps the customer well-informed. What happens is this - the customer produces a wish-list of airy-fairy nonsense as a requirements doc; the experienced waterfallists say "no, doesn't pass the feasibility study, go away and map out your requirements properly if you want the project to succeed", but the hipsters say "hey, no probs, we'll just go agile (and make it up as we go along)"; management ignore the waterfallists and accept the pipe dream that agile can solve everything; project fails; management finally bite the bullet, tank the hipsters, and ask the experienced waterfallists to try and rescue something from the wreckage. Seen it in government projects plenty of times.

        1. Sir Runcible Spoon
          Joke

          Re: AC Agile and government do not mix

          I've just had the oddest experience of fully agreeing with everything MAtt has put into a post on El Reg.

          Time to buy a lucky-dip methinks :P

        2. Anonymous Coward
          Anonymous Coward

          Re: AC Agile and government do not mix

          "With proper waterfall you get a comprehensive reporting system that keeps the customer well-informed."

          Who produces the reports?

          Unless they're receiving actual functionality that they can use and test themselves, they're taking on trust that your progress reports are true.

          1. Matt Bryant Silver badge

            Re: AC Re: AC Agile and government do not mix

            "Who produces the reports?...." That depends on how savvy the customer is. If they are smart they will have one of their staff included in the team to make sure what is reported actually matches real progress.

            "....Unless they're receiving actual functionality that they can use and test themselves, they're taking on trust that your progress reports are true." LOL, it's called a court case. As it was once put to me by an experienced PM, lying to the customer is bad, but giving them written evidence of your lie that could be used in court, well that's just downright stupid. The Government has lots of lawyers, time and money to pursue those that are stupid. Any consultant worth his money will make sure any report doesn't put a rope round his own neck by falsifying results. Consultants not worth their money....?

        3. Anonymous Coward
          Anonymous Coward

          Re: AC Agile and government do not mix

          But 'proper waterfall' is an even rarer beast than 'proper agile'. The fact is that government projects are hard, and the people undertaking them generally woefully ill-equipped to do so.

    3. <BLINK/>

      Re: Agile and government do not mix

      So true, speaking as someone who previously did system engineering for a public facing Gov organisation. Turns out most of the techies were doing Agile-like within their teams even before the high and mighty picked up on those trendy words. These are the same people who would not allow us to take up a discounted offer from a software supplier and make us go through procurement and end up paying more than even the standard price (as they went with a third party to supply us the same licence) and cause a delay of 6 months on a project.

    4. TechnoSceptic

      Re: Agile and government do not mix

      Well, they have to, thanks to Lord Maude & pals, regardless of whether it makes sense or whether anyone knows what they are doing, else they don't get past the Guard Dogs & Sentries: https://www.gov.uk/service-manual/agile-delivery/agile-government-services-introduction

    5. Anonymous Coward
      Anonymous Coward

      Re: Agile and government do not mix

      Unfortunately using waterfall approaches to deliver something to the public sector doesn't work either.

  3. Anonymous Coward
    FAIL

    Agile doesn't work

    If you're a half-decent developer you're already more than enough 'agile' without the need to graft a religion on to what you do; if you're not half-decent, then no amount of consultants and twat-speak will make you better. Instead, try using your brain, or try a different career.

    1. 1Rafayal

      Re: Agile doesn't work

      Sounds like you dont know what Agile actually is. It goes beyond the work of one single developer.

      The whole point of Agile is to deliver small amounts of work on a very regular basis, it doesnt have anything to do with the individual talents of any one person - much in the same way as every project methodology doesnt rely on this.

      1. Anonymous Coward
        Anonymous Coward

        1Rafayal - "it doesnt have anything to do with the individual talents of any one person"

        How can that possibly be true? Good software has everything to do with individual talents. A lot of software is in the state it's in today because people like you think everything can be commoditised if you throw enough consultants at it.

        1. 1Rafayal

          Re: 1Rafayal - "it doesnt have anything to do with the individual talents of any one person"

          Looks like we have some AC's who have never worked in a team of others.

          1. Anonymous Coward
            Anonymous Coward

            Re: 1Rafayal - "it doesnt have anything to do with the individual talents of any one person"

            Looks like we have an 'Agile Evangelist' who doesn't like their religion being questioned...

        2. Anonymous Coward
          Anonymous Coward

          Re: 1Rafayal - "it doesnt have anything to do with the individual talents of any one person"

          I think the point being made is that agile is a project methodology, not a programming methodology.

      2. Doctor Syntax Silver badge

        Re: Agile doesn't work

        "The whole point of Agile is to deliver small amounts of work on a very regular basis"

        My experience is that there's a fairly substantial minimum of work that must be delivered to be of any use at all. If your first delivery is, say, a simple order entry screen and a database behind it there's no use anyone entering any data because there's no way of using it because the warehouse needs to be able to get its instructions from the system.

        And if the next delivery is to print a despatch note it's still no use. Maybe the third delivery will be a picking list print so it's usable.

        Then when the invoicing is added someone realises that the data collected is inadequate. Then go back to the beginning, revise the database and order entry.

        A few iterations of this nonsense and you get confronted with an angry DBA who wants to know why you have to keep buggering about with database reorgs and can't you get a simple thing like designing a database right the first time.

        1. Mike Timbers

          Re: Agile doesn't work

          That first delivery *is* useful because although it can't be used it can be tested and accepted. A sprint - especially at the beginning - doesn't have to be shippable, it just has to be testable. It may take multiple sprints to get to the minimum viable product but each sprint along the way should deliver incremental benefits - an interface, a database table, a screen layout with accepted UX.

          And as for the DBA, just buy him a pint and he'll shut up!

          1. eldakka

            Re: Agile doesn't work

            @Mike Timbers

            I work in Operations/system management.

            Our experience with Agile projects is similar to the DBA reference.

            We've had to rebuild 10 servers across multiple environments 2 times (in addition to the original build) because the Agile team has gone off half-cocked, had servers built for them for how they like to work, then come to us and said "oh, and when we go into production, you have to manage/support these systems". They used their own deployment system, filesystem layouts and so on.

            So our first 'rebuild' (2nd build total) was to set the systems up in the standard we had that just slotted into our existing deployment methodologies and support/monitoring frameworks. And of course, since they were iterating through these sprints, detailed requirements were always changing, from environment-to-environment (dev, testing, integration, production), and since they had drop deadlines we had to, effectively, hand tune each server, so no 2 servers were exactly alike.

            So after we got a couple drops into production, so we'd finally been able to nail down exactly their final (well, to date!) configuration requirements, we are rebuilding again with a consistent build/configuration across their servers, so that all are built the same way. So that's the 2nd rebuild, for 3 builds total.

            The issues with your "just buy him a beer" are:

            1) We don't have the staff resources to do that much repeat work for 1 project when there are dozens happening, we just don't have the manpower -irrespective of the ACTUAL budget- to perform that much work dedicated to 1 project. Doesn't matter how much money they give us, we have an allocated staffing level and we are not allowed to exceed (hire more staff) that irrespective of how much money a project team will throw at us (a projects budget is transient - capital, whereas hiring staff is a permanent employee, an ongoing operating expense, so the project money might fund 2 more staff for 1 year, but who's going to cover those staffs wages when the project is completed? Therefore we can't hire based on project funding).

            2) They expect us to do work for them on-the-fly, to immediately respond to their emails/calls/IMs since they are 'agile' and need work done for them to complete this weeks sprint. This basically assumes we've got nothing else to do other than sit their waiting for their calls to do work. Whereas we may be

            busy doing work for a project that submitted it's work request 3 weeks ago and have no time to work on their shit - until they cry and whine that we are the reason they can't make their sprint/release objectives and manage to find someone senior enough to either tell us to do their work as higher priority or increasingly often as my manager's manager is starting to do - tell em to fuck off and submit a proper work request and get in the queue behind everyone else

            The problem is, Agile seems to require a massive increase in staffing level to support their iterative - i.e. read redoing the same work multiple times (a least in the infrastructure/middleware/backend sides) - and quick turn-around - i.e. expect you to be able to drop anything else you are doing and do work for them - approach because they haven't provided an end-state DB/hardware/support/operations/monitoring/capacity requirements that we can plan around and just build once and mostly forget about.

            So buying the DBA (insert here other backend-type staff, infrastructure, server support, etc ) a beer is going to do fuck all for all the extra work and stress the agile teams are throwing around.

            1. Mike Timbers

              Re: Agile doesn't work

              @edlakka, sorry but it's not agile that's at fault in your situation.

              your agile teams didn't work with you at the beginning to agree the server build.

              or

              your agile teams could have developed their software, their table layouts etc. then worked with your teams to implement the service once the solution was functionally complete

              or

              your agile teams could have built their software to not have dependencies on your builds

              or

              your agile teams could have deployed onto cloud platforms and supported them themselves

              Agile isn't at fault here because agile doesn't specify any method beyond iteration and constant checking with the stakeholder as the iterations progress.

              You're describing a failure of process agreement, maturity and delivery.

            2. Anonymous Coward
              Anonymous Coward

              Re: Agile doesn't work

              This government project has failed precisely because of ITIL jockeys like you in very senior posts. People who do not realise what cloud and agility REALLY means. YOU should 1) be able to provision a box in minutes. Clearly you can't. 3 rebuilds? Are you fucking joking? It should have been closer every single mother fucking deployment. 2) Your box configuration should PARTICIPATE in the entire pipeline flow. I'm certain this is not the case either with your static shitty infrastructure.

    2. Primus Secundus Tertius

      Re: Agile doesn't work

      I spent my career wondering how to turn science and engineering graduates (whom one would expect to think logically) into competent programmers. That is, produce programs which handle correct cases correctly, and error cases with error handling that does not crash. Programs which were tested to see whether they fail, not whether they barely scrape through a cursory test or merely compile.

      Those skills cannot be imparted by management.

      I never did find the answer.

      1. BebopWeBop
        Headmaster

        Re: Agile doesn't work

        One of the reasons that our experience has been that decent mathematicians make extremely good coders and designers (in real time systems anyway) - got to work to stop them getting bored :-) but the documentation is good if sparse!

    3. Anonymous Coward
      Anonymous Coward

      Re: Agile doesn't work

      From what I've seen Agile is insecure and full of holes and often promoted by comedians and slipshods. Fine for a short lived task but fails when reliability, privacy and security are concerned.

      It takes discipline and training to implement effectively.

      1. 1Rafayal

        Re: Agile doesn't work

        How is Agile less secure than a waterfall methodology?

        Or any other project management methodology?

        1. Anonymous Coward
          Anonymous Coward

          Re: Agile doesn't work

          Because it allows cock ups like the scheme this article is about and the reason why the emperor has been called out for having no clothes.

          I have seen many areas where secure thinking is overruled in the name of delivery leaving the support function to pick up the pieces. With decent design and control this situation is avoidable, but in the interests of cost reduction this seems to go by the wayside. Agile allows people to forget tasks while they move (or are moved) onto other tasks and more glory.

          It's not a personal thing as waterfall can either be like a trickle from the meltwater that is management or can result in a whirlpool that everyone gets sucked into.

          1. 1Rafayal

            Re: Agile doesn't work

            Agile isnt the reason this project failed.

            It failed because its a government IT project and would have failed regardless of the project methodology used.

            Your comments on security make no sense what so ever.

            1. Anonymous Coward
              Anonymous Coward

              Re: Agile doesn't work

              I'm a taxpayer, voter and citizen and I expect my data always to be secure.

              By allowing this project to fail are you saying the government doesn't take security seriously?

              Security should be the first and foremost consideration of any project whether government or not. I've found this approach ensures projects are more likely to achieve their objectives if this approach is taken.

          2. Doctor Syntax Silver badge

            Re: Agile doesn't work

            "trickle from the meltwater that is management"

            That trickle from management - I don't think it's meltwater.

          3. Mike Timbers

            Re: Agile doesn't work

            Agile does work and saying it doesn't is ignoring vast numbers of successful agile projects. It's like saying that waterfall doesn't work because of all the failed waterfall projects (and let's be honest, there are some fantastic examples of government waterfall project disasters).

            This scheme seems like it was doomed no matter how it was delivered because the whole point of agile is that it delivers iteratively and continuously. If there isn't a step forward for more than a couple of sprints, a good agile project would be asking why and doing something about it. It sounds like this project simply didn't follow any governance. That's not a methodology failure; it's a governance failure.

          4. The Commenter formally known as Matt

            Re: Agile doesn't work

            > Agile doesn't work Because it allows cock ups like the scheme this article is about and the reason why the emperor has been called out for having no clothes.

            um no. Thats not agile allowing cockups like this, its management/stakeholders.

            The projects been running for 3 years. With agile (assuming 2 week sprints) that means management have had 78 demos/releases showing a complete lack of work being completed, and they ignored them all.

            Waterfall would wait for 3 years and then give one release showing a lack of goodness.

      2. The Commenter formally known as Matt
        WTF?

        Re: Agile doesn't work

        No framework will give you the skills you are talking about.

        If you don't test fro reliability, privacy and security then they won't be there, Agile or Waterfall.

    4. Jonathon Green
      Holmes

      Re: Agile doesn't work

      No project methodology works.

      33 years of experience suggests to me that well motivated teams of talented developers working with good management will produce good quality results in a timely fashion while demotivated[1] teams of mediocre[1] developers working with mediocre management will produce poor quality results late.

      In both cases it will be in spite of the methodology adopted rather than because of it...

      [1] Trust me, in the absence of competent management if they aren't at the beginning of the project they will be by 6 months in...

      1. Anonymous Coward
        Anonymous Coward

        Re: Agile doesn't work

        @JG: Hammer, meet nail head. Bang on, thanks for coming.

  4. zebthecat

    Sounds as if they weren't actually doing Agile at all but merely spouting the management bullshit that can surround it.

    The projects I have been a part of that did use Agile were all about delivering working software first and foremost. From what i can remember that is pretty much top priority in the original manifesto.

    1. Anonymous Coward
      Anonymous Coward

      Yes, no design, no documentation, minimal testing. Just working software. Yeah, right.

      The problem is that although all developers think they can do that, very few are actually capable of it. The inevitable result is a large quantity of buggy poo, with a few decent bits here and there that the competent developers worked on.

      1. Anonymous Coward
        Anonymous Coward

        Yeah, so an agile team comprised of shit developers will deliver crap code just like a waterfall team composed of shit developers will. The question is whether a competent team work better under agile or waterfall. Given the correct environment (ie. proper backlog management) an agile team of competent developers will out perform the same developers doing waterfall. This is because agile done properly allows developers the freedom to manage their own effort.

        So many posters on here effectively saying "I can't be trusted to organise myself"....

      2. Vic

        Yes, no design, no documentation, minimal testing. Just working software. Yeah, right.

        That isn't Agile!

        Agile simply asks you to prioritise working software over complete documentation. Nowhere does it say that the documentation should be absent, nor does it place anything other than working software above documentation.

        It should be patently obvious to all that failing to go through a conventional design process never produces anything of value; it becomes development-by-evolution, and almost all evolutionary experiments become extinct.

        An old colleague of mine had a wonderful phrase: "A week's worth of keyboard-bashing can sometimes preclude the need for an hour's thought". That pretty much describes the approach of so many people who claim to be "Agile" (and clearly aren't).

        Vic.

    2. Anonymous Coward
      Anonymous Coward

      "Sounds as if they weren't actually doing Agile at all but merely spouting the management bullshit that can surround it."

      P much.

      I've done formal Agile processes in government (usually some form of scrum) and it goes one of two ways.

      1) You're working directly for a small, operational team who just want shit done and you're able to deliver on time every couple of weeks. Nothing is perfect but everyone is happy because real things are being delivered, until central IT take over and it all turns to shit as they graft your delivery process onto their corporate standard. From then on you spend more time exporting JIRA reports to their standard project reporting spreadsheet template than you do producing useful stuff.

      2) You're working on a "strategic initiative" or "digital transformation" with dozens of consultants where no one has any fucking idea what is to be delivered or what the overall goal is and there are dozens of disparate projects organised into programmes competing for funds. Senior leadership spend their days being bamboozled by the supplier and firefighting one thing at a time. GDS swan into the project once a quarter to tell you you're all shit and should use Ruby for everything. Nothing serious gets delivered but everyone gets paid because it's all T&M because government has *no idea* how to structure a contract to support iterative delivery.

      It's not fun, but neither is it unique to agile or unique to government.

      1. Anonymous Coward
        Anonymous Coward

        Agile fine until it hits the real world?

        "1) You're working directly for a small, operational team who just want shit done and you're able to deliver on time every couple of weeks. Nothing is perfect but everyone is happy because real things are being delivered, until central IT take over and it all turns to shit as they graft your delivery process onto their corporate standard. From then on you spend more time exporting JIRA reports to their standard project reporting spreadsheet template than you do producing useful stuff."

        Not to be cynical but it sounds like everything is fine while the development team asses its own progress and outputs but as soon as the software is used and assessed by someone outside the bubble of self congratulation reality bites but instead of accepting what was delivered did not meet the customers needs the customer gets blamed for not drinking the coolade.

        1. Anonymous Coward
          Anonymous Coward

          Re: Agile fine until it hits the real world?

          When the customer is sat a handful of feet away from you using the stuff you're building and shipping once every week or two it's really rather hard to make the argument that the dev team is "assessing its own progress" or that the product "did not meet the customers needs". I would further propose, based on experience, that the delivery teams and end users are far better placed to report progress than a professional project manager whose sole purpose in life is to change the colour of boxes on his ever-sliding Gantt chart.

          This is actually at the root what most Wagile frameworks get wrong - particularly in government. Someone takes on the title of "Product Owner" but they're not empowered to make decisions or knowledgeable enough to represent their peer stakeholders. What you'll usually end up with in the civil service is some mid-level fast-streamer from the IT delivery organisation acting as a classic business analyst. You end up without the iron-clad requirements and specifications of proper waterfall and without the empowered decision maker required by agile processes.

          What you need is a decision maker with enough clout to bring round a consensus. Doesn't matter if the consensus is wrong - you fix it in the next iteration. It does matter if you spend 2 years and £200m dithering over requirements to the point nothing ever gets built, fooling yourself that it's ok because "it's agile", during which time the carousel of fast streamers and ever-changing senior senior civil servants has been refreshed four times over.

          Guess which one happened here.

          1. Robert Grant

            Re: Agile fine until it hits the real world?

            Yeah agree - powerless (and probably 95% absent) product owner makes a huge difference.

            To all the waterfallists: you'd be surprised how similar Agile and Waterfall are in achieving results, if both are done properly. The main difference is Agile will deliver what you need now, whereas Waterfall will deliver what you thought you needed two years ago, and now have to lawyer up to argue about what features have or haven't been delivered.

            1. Roland6 Silver badge

              Re: Agile fine until it hits the real world?

              >The main difference is Agile will deliver what you need now, whereas Waterfall will deliver what you thought you needed two years ago

              Not heard of change control?

              From my experience, the primary difference between Agile and waterfall is in the contracted deliverables and the ownership of risk. With waterfall, I the supplier commit to delivering a complete system that satisfies the requirements within a defined timeframe. With Agile, I the supplier commit to delivering some functionality through a number of development iterations within a time period.

              1. Anonymous Coward
                Anonymous Coward

                Re: Agile fine until it hits the real world?

                "Not heard of change control?"

                Change control: the act of trying to wrestle reality into being what you decided it should be two years ago, instead of evolving and adapting to reality as it is now.

                "From my experience, the primary difference between Agile and waterfall is in the contracted deliverables and the ownership of risk. With waterfall, I the supplier commit to delivering a complete system that satisfies the requirements within a defined timeframe. With Agile, I the supplier commit to delivering some functionality through a number of development iterations within a time period."

                You are comparing apples and oranges. The waterfall situation you describe is about delivering a defined outcome. The agile situation you describe is just T&M delivery of services, not delivery of a defined outcome. Agile can be used for both, but agile has a far better chance of delivering to expectations because of far better risk management abilities - unless, of course, management only pay lip service to agile without actually being agile.

    3. Doctor Syntax Silver badge

      "The projects I have been a part of that did use Agile were all about delivering working software first and foremost."

      At which the users gaze in desperation because there's no documentation on how to use it.

      And the developers successor s throw it away and write something that they hope does the same thing because there's no documentation for them either.

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon