Tim Bray: the Fortune 1,000 are bleeding money for lack of agile practices
What I’m writing here is the single most important take-away from my Sun years, and it fits in a sentence: The community of developers whose work you see on the Web, who probably don’t know what ADO or UML or JPA even stand for, deploy better systems at less cost in less time at lower risk than we see in the Enterprise. This is true even when you factor in the greater flexibility and velocity of startups. This is unacceptable. The Fortune 1,000 are bleeding money and missing huge opportunities to excel and compete.
The word “agile” is overloaded with multiple meanings, but long before it became a fad in the software world, it was a subject of study for academics who were trying to improve the efficiency of supply-chains at large firms. And that is basically what Tim Bray is talking about: the efficiency of supply-chains at large firms.
Regarding software development, I wrote about agile practices in Geography still matters: physical proximity is essential to agile practices.
Using agile practices in large organizations is a tough issue and it always has been. Large firms must, by their very nature, be about hierarchy, whereas agile practices depend on trust, and in some respect trust and hierarchy are opposites. For someone at the top of a large hierarchy to send an order down to the bottom, they are implicitly overriding whatever the lower level employees might think best – therefore, a lack of trust is implied in every order.
In the 1990s, DARPA invested some money in agile research, which lead to such books as that by Goranson: Agile Virtual Enterprises. Goranson looks back at the whaling industry of the mid-1800s and examines how 2 small towns in Massachusetts were able to capture 90% of the whaling industry, when most of the actual hunting was done in the South Pacific. He found that the patterns of hiring and team building had much in common with what start-ups aspire to nowadays – a tight-knit, fast-forming team of highly competent professionals held together by a commonly understood body of professional ethics and expectations. That environment sounds a lot like the startup scene that now generates new web companies.
The curious thing is how little has changed: small firms arising from a tight-knit, high trust culture continue to lead the way in innovation, whereas large firms continue to lag behind. But large firms can possibly afford to lag behind, they have other strengths, including the capital that they are charging monopoly rents on.
I think Tim Bray is terribly wrong when he suggests that the way forward is, basically, to just do stuff better:
Here’s a thought experiment: Suppose you asked one of the blue-suit solution providers to quote you on building Ravelry or Twitter or Basecamp. What would the costs be like? And how much confidence would you have in a good result? Consider the same questions for a new mobile-network billing system. ¶
The point is that that kind of thing simply cannot be built if you start with large formal specifications and fixed-price contracts and change-control procedures and so on. So if your enterprise wants the sort of outcomes we’re seeing on the Web (and a lot more should), you’re going to have to adopt some of the cultures and technologies that got them built.
It’s not going to be easy; Enterprise IT has spent decades growing a defensive culture based on the premise that you only get noticed when you screw up, so that must be avoided at all costs.
I’m not the only one thinking about how we can get Enterprise Systems unjammed and make them once again part of the solution, not part of the problem. It’s a good thing to be thinking about.
In my opinion, there are certain problems that come up at large scales that will never get fixed, because they are fundamental to that scale. Scale determines the context. Consider mass: I am not pulled toward an ant, a house or a mountain, but I am pulled toward planet Earth, because once you get enough mass in one place, you start to notice its gravity. I do not think that the gravity of planet Earth will ever be “fixed”, I think it is normal that the amount of mass that makes up the Earth should have the Earth’s gravity. Likewise, I do not think hierarchy (and its attendant risk avoidance) will ever be fixed in large organizations, because I think hierarchy is natural to large organizations.
However, things will get smaller. And that is the real solution. And it has been going on for a long time now. Since the recession of 1991, the Fortune 500 have shrunk significantly, both as a percentage of the GDP, and as a percentage of total workforce employed. And there are more small firms.
That is the fix that I see: we will not fix large organizations, but we will have less of them.