Waterfall Drives IT Engineer Over the Edge
Last week, I saw the exchange above. The names have been changed, but the cartoons basically show what happened.
There is only one problem here. The engineer in the cartoon has no idea how to really develop software.
He has been sold a bill of goods. Conned. Tricked. Hood-winked.
Like a dedicated member of a cult, he believes what he has been told; he believes there is only one way to develop software. The method he believes in is called the Waterfall.
Just as Karl Marx predicted that Communism is the inevitable next step after Capitalism, the Waterfall model of developing software dictates that all software development must follow a set path:
1. Requirements specification
2. Design
3. Code
4. Test and Validate
5. Maintenance
Sure…. history has proved Karl Marx absolutely wrong. Sure, even the guy who first came up with the Waterfall approach said it was “risky and invites failure”. And sure the Waterfall isn’t the only approach to developing software.
But, that hasn’t stopped IT department after IT department from sticking rigidly to the Waterfall approach.
Most IT departments stick to it so rigidly that they can not even understand when their clients are trying to work with them towards an iterative approach.
One alternative to the Waterfall is called the Agile Development.
There is a foundation, with an Agile Manifesto. You can read the a great introduction by Victor Szalvay, called An Introduction to Agile Software Development.
You might also want to check out a great book called Agile Management for Software Engineering: Applying the Theory of Constraints for Business Results Here’s a quote:
If software knowledge work is to remain in the rich, developed countries of the world and software engineers in America, Europe, and Japan are to maintain the high standard of living to which they have become accustomed, they must improve their competitiveness. There is a global market for software development, and the rise of communications systems such as the Internet, have made it all too easy to shrink the time and distance between a customer in North America and a vendor in India or China.
Jobs are at stake! Just as western manufacturing was threatened by the rise of Asia in the latter half of the 20th century, so too is the knowledge worker industry threatened by the rise of a well-educated, eager workforce who can do the same work for between one tenth and one quarter of the cost.
The answer isn’t that software developers must work harder if they want to keep their jobs. Software engineers aren’t the problem. The answer is that management techniques must improve and working practices must change in order to deliver more value, more often, in order to improve competitiveness.
…
The secret to economically viable software engineering is new working practices based on new management science. The Agile manager must construct an Agile learning organization of empowered knowledge workers. When this is achieved the results will be dramatic. Improvements of 4 times are easily achieved. 10 times is definitely possible. Imagine if your software engineering organization could do 5 times as much work in half the time it currently takes. What would it mean for you, your job, and your organization?
The Root Cause - A Belief that IT can’t talk to Business
Managers use the Waterfall approach because they think that it takes a long time to develop an IT application.
Managers also use the Waterfall approach because they think that their IT engineers can’t communicate with the business.
Both ideas are wrong.
If you really have an IT department that is incapable of talking to your business users, fire them all, and out source the whole department to India or China. However, it is unlikely that your IT engineers are incapable of talking to your end users. Instead, it is highly likely that they can and should be able to directly deal with at least your internal end users. That direct communication will help them to produce solutions that better fit the end user’s needs. In other words, direct communication radically reduces the risks of any IT development project.
The notion that IT solutions can not be instantly developed is also wrong.
New tools, such as Ruby on Rails, mean that most web based applications can at least be prototyped within a couple of days.
If you are a manager, a director, if you sit in the C-level suite, you need to learn just a little about Ruby on Rails. You need to learn about Ruby on Rails so you can set your expectations correctly.
Learning about Ruby on Rails only takes 5 minutes. For non-coders, some of that 5 minutes is dull.
However, you do not have to do much, other than watch.
5 min video of how a coder creates a web interface to search for photos on Flickr
In 5 minutes, you will see a developer put together a very cool, very slick and very powerful web application.
When your IT teams develop something for you, you should be expecting that kind of instant turn around.
Given that it only takes minutes to build working prototypes, it is obvious that IT engineers should be sitting with their end users, working quickly and iteratively towards a working prototype.
The crazy scene I witnessed last week should never happen with a modern capable Agile IT team.
I do not blame the poor developer who was forced to stick rigidly to “the reqs doc”, but I do blame his CTO.
The Waterfall is not the right way to build software. The Waterfall is evil! ;)
I’m a little surprised you haven’t got some flames from lovers of Waterfall - they seem to still exist in hordes. Despite my best efforts, I have been over-ridden by Waterfall on a current implementation project … a feast for consultants, and little value for the business.
Great piece. I like your writing style and the use of cartoons. The message is right on the mark, too.
Interesting you mention RoR just now, and in this context. I work at a 33,000 employee company with an IT department of some 1,300 people. We have an agile delivery group with around 60 people, down from its peak size of 70. Obviously, there are many more traditionalists than agilists in the organization. The most recent silliness came last week, when the lead enterprise architect sent out a memo demanding that all references to RoR information be deleted from the corporate intranet. RoR is “not supported” and we are not to mention it or make any information about it available internally, to anyone for any reason. (He had happened on some information about a proof of concept I had done last year that empirically demonstrated how RoR fits into our corporate technical infrastructure seamlessly.)
Direct communication between development teams and customers, and anything else that enhances communication and streamlines delivery, is seen as a sort of “threat” by many traditionalists. I suspect it’s because they worry the layers of documentation they generate as part of their jobs would no longer be necessary, and by extension…
Regarding Ric’s worry about flames…remember that traditionalists can only take advantage of flame opportunities that were “in plan” from the very beginning. As this post provides an emergent flame opportunity, traditionalists will have difficulty adapting.
A good question to ask is, why DIDN’T the requirements document include Daily Reports?
Is it because she (the business user) didn’t know she needed daily reports when the document was written?
Was the process used in writing the requirements document faulty because it did not discover the need for daily reports? (Doesn’t the Developer know that report generation is one of the areas where business users frequently feel that, at last, they have some understanding of how to make a system do something useful for them?)
Or, as Steve Baker write in the wiki entry “Agile CMMI - Oil and Water or Sugar and Spice Open Space Session” from the Agile 2006 Conference [http://agile2006.stikipad.com/public/show/Agile+CMMI+Open+Space]: “The heart of the conversation was on the perception that if a model forces us to write something down then we are not trusted (the more we write stuff down, the less we trust one another).”
My personal feeling is that this “waterfall” vs. “Agile” dichotomy is beginning to sound like a bit of a straw man argument to me now, given the increasing interest in use of blogs, wikis, and relationship management systems in the enterprise. My guess is that this is forcing a rethinking of the concept of “documentation” since documentation can now be more reflective of the conversations that actually take place between developers and users.
I agree with Dennis that the argument between Waterfall and Agile is becoming a mantra with no meaning. What we need to focus on is that the agile methodologies (which use the same steps that you describe for waterfall, albeit, within one user story) are nothing but a rearrangement and better understanding of how to execute projects when there is a high level of uncertainty in the requirements.
I would place a small question mark on your comment that IT people can talk to customers. In my experience, IT folks (who talk tech-speak) typically aren’t fluent in business-speak or user-speak and their ability to translate varies over a wide range. Those who can do this translation effortlessly are those that should be revered and used as a model for those who can’t. I do agree, however, that this is definitely an ideal goal.
I also agree that your writing style is great and I very much like your cartoons. Best regards!