“Empower people to act responsibly and they generally will. Exercise empathy and people will trust you. Give people the information they need and they will reciprocate. Agile development was an attempt at doing these things better, but it has been reduced to a collection of code words and empty rituals by managers who don’t realize that it is actually their philosophy about the purpose and nature of management itself that needs to change, and not merely their choice of techniques.”
“It’s well known that creative people lose their creativity if asked to explain themselves while they are working. It’s the same with software. Programmers often have to work in an environment of one-sided transparency. These Agile systems, so often misapplied, demand that they provide humiliating visibility into their time and work, despite a lack of reciprocity. Instead of working on actual, long-term projects that a person could get excited about, they’re relegated to working on atomized, feature-level “user stories” and often disallowed to work on improvements that can’t be related to short-term, immediate business needs (often delivered from on-high). This misguided but common variant of Agile eliminates the concept of ownership and treats programmers as interchangable, commoditized components.”
“I am proud to be one of the 17 founders/authors of the The Agile Manifesto back in 2001. I think it provided a jolt of energy, hope of a better way of doing things, of creating software and making the world work better. It was a pivotal turning point.
But in the 14 years since then, we‘ve lost our way. The word “agile” has become sloganized; meaningless at best, jingoist at worst. We have large swaths of people doing “flaccid agile,” a half-hearted attempt at following a few select software development practices, poorly. We have scads of vocal agile zealots—as per the definition that a zealot is one who redoubles their effort after they’ve forgotten their aim.
And worst of all, agile methods themselves have not been agile. Now there‘s an irony for you.
How did we get into this mess?
...
It’s all too common these days to see arguments on Twitter or mailing lists with these rules-bound zealots arguing that ”you’re not agile” because you aren’t following the rules to their satisfaction.
What happened to the idea of inspect and adapt? What happened to the idea of introducing new practices, of evolving our practices to suit the challenges at hand?”
Self-organization means empowerment. Empowerment is not the rigid adherence to a process that you cannot change.
Fig 1. True agility
In this post we'll look at practices that work against self-organization, against empowerment, and may be, hopefully, ways to avoid it. We'll look at Scrum, it being the most comon flavour of AgileTM.
Common Dysfunctions of Scrum
I'm going to look at Scrum because it is the most common form. But these dysfunctions can be applicable in part to other flavours of AgileTM I think.
Scrum has a Product Owner and a Scrum Master and your experience of Scrum hinges on how these two roles choose to act. Additionally there might be more actors who affect the Scrum implementation such as higher management and project stakeholders.
Let's look at the Product Owner first. In our first post I identified effective and frequent communication as important for agility. It is necessary for an effective inspect and adapt cycle.
Dysfunctions of a Product Owner
The Product Owner is supposed to be a proxy for the client. The PO is responsible for communicating on behalf of the team to the client and vice versa. They are a kind of champion that focuses the development team on the needs of the client and the business value that needs to be delivered.
Dysfunction #1 - Fake Product Owner
The first common dysfunction of this role is when the PO is a Product Owner in name only. This can be because in reality:
they do not know or understand the real business domain
have no relationship with the real client
Choosing the wrong person as the PO, or forcing the role when no good choice exists is dysfunctional and leads to problems. The team knows that the PO speaks for no-one, does not understand the domain and ultimately adds nothing to the process.
This becomes a real issue if the charade continues. The PO is supposed to be speaking with the client, enabling effective communication, taking some of the burden from the dev team and prioritizing user stories. But none of that is possible if the PO doesn't know the domain and/or is not speaking with the client.
If you see this dysfunctional practice then call it out and suggest a change to the process. That might be suggesting a different PO or doing away with the role and finding a different way of managing communications with the client.
Dysfunction #2 - Lack of Prioritization of Non-Functional Stories
Software development is a highly technical field and there are many purely technical aspects to the work like:
good modular design
separation of concerns
clean code
good abstractions
performance characteristics
technical points related to chosen technologies
refactoring
automated testing
security
and many more
The PO is responsible for prioritizing the user stories, but is most likely not a software developer and never has been. This can lead to technical stories being under-prioritized. Also, by making the team fight for each non-functional story, it discourages the developers from even trying to negotiate for them.
Note: If you can avoid the need for technical stories then the better. Writing tests should be part of writing code, I'm not a TDD guy (more of a test during/soon after development), but I always argue that writing tests is part of writing code and not a separate task. Also certain levels of refactoring should be included as a part of writing code. Do you ask for permission to write an if statement? Of course not, and you don't need permission to write tests or carry out refactorings.
So the following things can go wrong:
it is hard to get the PO to prioritize technical stories. It always requires a struggle and often we get denied and discouraged.
time pressure and estimation pressure lead to more technical debt which leads to more technical tasks to fix the mess.
I don't see a problem with the PO signalling to the team what the priority is of the functional tasks. But I think that the team should have the final say on what gets worked on and when. If the team needs to focus on refactoring, paying off technical debt, working on some architectural component, then they should be trusted to do so.
Really it comes down to trust. If management and stakeholders don't trust the development team then self-organization is hard to achieve. To be truly agile, management and stakeholders need to trust that the team are best placed and best suited to making decisions about how to build the software.
Dysfunctions of a Scrum Master
Some Scrum Masters are professionals that know what they are doing, why they do it and are flexible. Others are not. Some can be quite religious about things and be quite divorced from reality. Some are project managers turned SM, some are developers turned SM. It's really the luck of the draw.
The first signs of dysfunction are:
a rigid interpretation of Agile
a rules based focus
a lack of adaptability, or some might say a need to control
a self-righteous attitude
The only way you counter those negative behaviours is to know what true agility is and what behaviours enable it. Then you need to convince the SM (and perhaps your boss) that change is part of agility.
Frame all practices, both existing and proposed, in terms of the goals of true agility:
long term, sustainable productivity
building the right software
being responsive to change
being able to deliver frequently
Framed like that, no practice can be argued as indispensable. There's more than one way to skin a cat.
So perhaps you want to drop the Daily Stand Up? You need to demonstrate that dropping it will make things better. Perhaps you are a close-knit team and already communicate well and often. Perhaps impediments that need the SM are few and far between? You can ask for help when the impediment comes up. Or perhaps you can sort out your own problems. It depends entirely on your context: the team, the organization and the project/product.
There is no "one way", it's all about context and you are smack bang in the middle of it. You decide. But you justify it in terms of real agility and continuous improvement.
If you want change then fight for it. Argue for a new practice or the removal of one, framed against the behaviours that lead to true agility. Don't let people put artificial boundaries on continuous improvement. Don't let others force practices on you that you know are dysfunctional.
Higher Management
Things can get even tougher when higher management back up AgileTM practioners. For one, you probably don't have a direct relationship with higher management, or if you do then there's still middle-management in between.
Management usually don't know what it takes to build software. Unless you're in an engineering led organization why would they? So they might be at the whim of Agile consultants, trusting that they can deliver what they've promised. AgileTM forced on dev teams from this level is not a great situation to be in.
Sometimes management might want to be flexible, but they also want to standardize. They want a single template for all development teams so business people and stakeholders know what to expect when dealing with teams. Standardization leads to efficiency right?
But taken too far, standardization inhibits agility. Standardization usually means standard rules for teams to follow. A focus on rules leads to the following of the rules for the rules sake. Sounds a lot like Scrum. This might be one reason why Scrum is so popular. You can implement this cookie cutter solution across all teams where everyone works the same way and become "Agile".
So perhaps we standardize on two week sprints with a demo at the end. We have a corporate powerpoint template for use in the demos. We standardize on a workflow for our project management system. We standardize on a set of meetings that each team attends. We standardize on forms of communication between development teams and the business. We standardize on the roles involved. We standardize on how we do software design and break work into user stories. We standardize on who prioritizes the work (and it isn't the developers).
Does that sound like micro-management anyone?
Developers at this point have little room for self-organization. It is when developers are so thoroughly controlled that sentiments like "[Agile/Scrum] treats programmers as interchangable, commoditized components" appear.
What can you do about it? I really don't know. All I can think of is what I've already said:
argue for change framed against behaviours that lead to real agility
call out behaviours that do not contribute to agility
call out dysfunctional behaviours
The Developers
Finally, we need to make sure we step up. First and foremost we need to exercise technical discipline. Without technical discipline we cannot put into practice good technical practices like test automation, clean code, separation of concerns etc. If we get the freedom we desire and do nothing with it then we're the problem.
Here are some recommendations:
be proactive and honest. Look at what can be improved, what dysfunctional behaviours can be eliminated, which roles are not needed.
invest time in developing good relationships with the business side and invest time in regular communication.
invest and prioritize time on software quality practices
use the project management system effectively. If you keep your tickets up to date (daily) then work can be tracked without the need for stand ups.
if you can't negotiate a "no estimate" process, then at least be honest regarding your estimates. Of course all estimates are wrong! I am not suggesting that estimates can be accurate, but there are pressures, often slight social pressures to purposefully under-estimate work. This is where the project management system can help. If you can track hours spent on previous work, and you can roughly compare tasks, then you can get a ball-park figure. If it looks like a lot then go with it, its probably more accurate than the socially acceptable estimate.
Under-estimation coupled with a time-boxed iteration schedule (sprints) is honestly one of the major contributors to a state of constant rush which then leads to corner cutting. If you care about sustainable productivity then corner cutting needs to be avoided at all costs!
True Self-Organization
There will always be some constraints on your team. But there are constraints and then there are rigid, rules based processes that relegate development teams to acting like factory workers, mindlessly following the assembly process.
Agility is about being responsive to contextual change. It means adaptability. It requires a motivated team! Micro-management is demotivating and demoralizing. Empowerment is empowering! Empowerment, trust, faith in your development teams creates an environment where creativity, drive, passion and autonomy thrive.
Only by being given the freedom to be truly self-organizing can you get into that state. In that state you can deliver far more value, far more innovation and be far happier doing it. So fight for it! I know I am.
Next we'll look at practices that lead to an unsustainable pace which leads to technical debt, a software quality crisis and the inevitable slow down.