Software Practices and Fairy Tales

Let me tell you a fairy tale

Once upon a time, not so long ago and not really that far away there was a Manager.
He was a good Manager, well liked by his subordinates and appreciated by his superiors, and he liked his job. But lately he had been charged with a new type of Project a Software Project.
This caused the Manager much anguish, for Software Projects where difficult beasts to master. They always seemed to take longer than expected, even when taking into account that they would take longer than expected, and even when completed the client never got what he actually wanted.
So the Manager went on top of a tall office building to gaze at the city and reflect; and lo and behold he saw a skyscraper being built, hundreds of people working in harmony toward one end.
"Why," he thought "can architects build a glorious skyscraper while I struggle to produce simple programs?"
And he was enlightened.
Architects, after all, did not gather together hundreds of workers and tell them, I want you to build me a skyscraper. There were months even years of preparation before even a single stone was moved. The same should be true for software:
First business analysts would analyse the problem to be solved along with the client, until a scope and solution was agreed upon.
Then software architects would create a model (a blue print) for the application that would implement the solution, and only then would programmers start coding.
Finally after testing everything is as it should the application would be deployed.
As so it was, and from that day forward software projects where always on time, and the Manager lived stress free ever after.

The Waterfall Model Was Actually Successful

The tale above describes the software development life cycle Waterfall methodology. It is now, in the age of the Internet and Agile methodologies, vogue to look down upon the Waterfall model as out of touch with reality and antiquated.

Some detractors of the Waterfall method claim it aims to turn software development into an assembly line where programmers mindlessly follow the architect blue print to churn out applications. In their view the Waterfall method fails because it fails to recognize software development is a creative and complex process, and any "blue print" no matter how carefully designed is going to differ to some degree from the actual implementation. This is a great disservice to the pioneers of software project management.
Waterfall is modeled after civil engineering and manufacturing, but it's goal is not to treat software development as an assembly line; rather it aims to impose the same rigor necessary to bring new products to market.
Building a skyscraper after all is not just a bunch of builders following an architect's master plan; behind any such enterprise there's an army of people whose job is to deal with all the inconsistencies between the plan and reality, and still deliver a skyscraper (hopefully without going over budget too much.)
The failure of the Waterfall and related methodologies in the Internet age is due to a change in circumstances. For projects running in Internet time it is more important to be able to deliver incremental functionality fast and cope with constantly changing requirements, than to predict the resources and time needed to deliver the ultimate product.
To claim any non Agile methodology is pure useless bureaucracy, imposed by control obsessed managers on programmers without regard to how software is really developed, is going too far.

Everything Has a Faerie Tale

My intention in this article is not to praise or condemn the Waterfall model or any other methodology since, but rather to address the "Fairy Tale" description that invariably accompanies methodologies and processes.

Software development practices are always evolving, and any new practice, whether it is a new design concept, methodology or even a new library is first described in using a "Fairy Tale".
Let's take ORMs for instance, according to the Fairy Tale description an ORM library makes writing applications using a relational database easy by mapping object oriented models into relational schemas in a database; but if they are so good how come untold hours are wasted trying to make the ORM do things which using raw database access and SQL would be trivial. Does this mean the ORM promise of data-oriented applications without SQL a, er, fairy tale? Well, in this case probably or maybe not object relational mapping is a complex problem.

When first analyzing requirements and organizing them as user stories or use cases, it is customary to start with the most common scenario without problems; the Main Success Scenario (MSS) in use case parlance. How to deal with all the problems and edge cases is left for later user stories; so it is not surprising "Fairy Tale" descriptions are used.
The thing to remember is they are just that, fairy tales, the best possible outcome when there are no incompatibilities. There are no panaceas in software development, practices may deliver on their Fairy Tale, but only if you take the time to learn how to apply them, when to apply them and when you are better off without them.