In larger enterprises, there is a strong and venerable tradition of trying to avoid functional redundancies. Especially senior management makes it a priority to ensure that, if possible, nothing is every done twice, or in two different ways, or by two different people or groups.
Cherished enterprise traditions and lore
In many companies, when a senior manager realizes that anywhere in the company, two similar or seemingly identical problems are being solved with two different tools, or by two different groups, they lurch into action like superheroes. The redundancy is perceived as a problem per se. The reasoning behind this is the following:
It’s a waste of resources to solve the same problem twice (and adapt the solutions twice too, as time goes by).
There has got to be one single best way to do it! Let’s find out which is best and eliminate the other one. (I really like this argument.)
Too many different procedures increase complexity in the organization. If we permit more than one procedure, an infinite number of processes will sprout and chaos will destroy the Earth.
Swiftly, meetings with everybody involved and their bosses and their bosses’ bosses are arranged. The culprit who has sinned against the “standard” process (standard = whichever came first) must repent and is let off the hook after agreeing to provide input to help improve the original process. When the dust has settled, our manager is happy because he has successfully eliminated a redundancy in the company’s processes and saved a small piece of the world from chaos.
To their credit…
I’m writing this slightly tongue-in-cheek not because the reasoning behind eliminating redundancies is wrong – most of it isn’t and in many cases it’s quite reasonable to proceed as described above. What makes me grin is the way the behaviour seems to have become automated: It’s an absolute no-brainer. Everybody who takes part in it could script the whole process, do it in their sleep and without a conscious thought. The outcome is clear from the beginning. And it makes people feel good, too – now why shouldn’t that be a reason to smile?
This is a software blog, right?
Yes, I just took the scenic route. Now, why can the above be a problem for software developers? It can be because the same procedure, with the same degree of automation, is applied to software architecture and design. Suppose the manager encountered above is a senior manager in software developing, overseeing a number of different products. Now he realizes that two project teams are using different archiving tools. This can’t be right, he thinks, and lurches into the well-known routine. I’ve seen this happen for absurdly small-grained development entities. I’ll give you an example.
A real-life example: A design decision and the consequences
Two huge enterprise applications (each built in hundreds of man-years) which are part of a same product suite use the number X. X is a single customer-specific number that is maintained in a customizing table and changes once per year. It is a well-visible and important factor in a number of calculations across the application suite, similar perhaps to a tax rate. Elimination of redundancy by management resulted in the design decision that the number would be stored in a cross-application table located in a cross-application software layer developed in a different development system. And the consequences?
Consequence 1: More meetings indefinitely
Quickly it was realized that both applications needed not only the number X. There are a number of parameters and customizing values which are closely related to the number X. So the table was subsequently extended by another field whenever one of the applications thought they needed something related to X. The shared table being common property, the teams need to agree upon every change. Whenever changing business processes or legislation forces one feed to make a change (quite frequently), they need to travel, meet with the other team and discuss it. The cost of cross-project, cross-application cooperation can be quite high: organizing the meeting, preparing the slides, travel time, the actual meeting, writing a protocol. And all of this before a single line of spec (not to mention code) has been written.
Consequence 2: Software logistics
Another unwanted side-effect are higher costs for software logistics. Because cross-application development takes place in a different development system or track, you need to get the code (or dictionary object) from where it is written to where it used.
Consequence 3: Tight release coupling
When two applications both define and consume a shared API, they are extremely tightly coupled. None can move without the other moving in perfect sync. Examples from release management hell: Suppose application A requests a change.
CA (cross-app development) needs to have the resources to make the change.
B needs to have the resources to adapt to the changed API.
A and B must wait for CA to complete its work, which results in a smaller time frame in which they must have spare resources.
If A and B run on the same production system, the change in A triggers a new production release of B and everything else that depends on CA.
Let’s have a closer look at release cycles.
A is rather new and puts out new releases quarterly.
B is older, has a low release frequency and puts out one major release per year.
In the worst case, the coupling through CA can lead to a battle about when the changed CA goes live.
If A looses, they have to wait for the next scheduled release of B.
If A wins, B is forced to put out a release quickly, which can be very difficult if their software is undergoing a major reconstruction or preparing for a major release.
Needless to say, assessing the complex situation and weighing the alternatives consumes a lot of time and money.
Consequence 4: Unclear semantics
Some of the columns of the shared table represent a semantic overlap of two business areas. Where this occurs, a closer look typically reveals slight differences. Tax calculation in A and B is almost identical, only in B there’s an exemption rule that A hasn’t got. Or A and B both use the “last day of contract” information, only for A there’s a rule that the effective end date is at least one calendar month after “begin of contract”. This kind of semantic overlap makes it difficult for developers to know how exactly to interpret anything in a shared API. If the documentation is not very, very concise, the trouble is that asking experts from A’s business domain does not provide a sufficient answer, because freak cases from B’s business domain might causes unexpected values, or cause unexpected behaviour, and vice versa. The true semantic overlap is always much smaller than it seems. A vague semantic overlap causes endless troubles.
Consequence 5: Difficult testing
In the worst case, the cross-application code can only be tested in a different system. Each iteration of the test cycle is expensive and takes a long time because of the costs and delay resulting from cross-system software logistics. (Also, the quality of the software is likely to suffer from poor testing.)
Consequence 6: No redundant development
The sixth consequence is the only one that ways actually intended. Instead of two customizing tables, only one was developed. Also, customers maintain the value of X only once (per customizing system) whenever it changes.
Weighing the costs for the organization and the software
Although reasonable when seen for themselves, the advantages of consequence 6 seem rather lousy when weighed against the costs and risks resulting from consequences 1 through 5. The costs and risks of tolerating a redundancy must always be weighed against the costs and risks of a coupling through centralization. For organizational processes, the cost of coupling is frequently lower and risks are not so high. Because business processes can be changed fairly quickly. Though it may not always seem so, organizational processes are still more agile and flexible than software. That is why centralizing functions is frequently a cost-saver in organizations. In software development, centralization is frequently a cost-driver. The high complexity of enterprise software makes it difficult to oversee the consequences of making changes in a part that affects others. Reducing the scope of changes is achieved with encapsulation, which typically increases redundancy as a necessary side-effect. So frequently in software development, redundancy is good for you.
Encapsulation as a design principle for the organization
Some of the benefits attributed to encapsulation and higher degrees of redundancy in software can also be reaped in the organization of the development process. Decoupling departments in an organization can be just as beneficial as decoupling software components, if the cost of working together is high. Suppose your central QA department is physically and conceptually far away: It costs time and money to travel, and time and nerves to connect on a conceptual level, until each side of the table understands the perspective of the other side. Also, your QA department may not be able to support your development project when you need it, so you have to wait for them or speed up your development to find a common time frame. These costs must be weighed against the advantages of a central QA department: standardized QA methodology across projects, synergies due to centralization, and so on. If in your particular case, the costs of extra-project cooperation outweigh the benefits, you might be better off with a more decentralized approach: Have a small QA team within the development project. Don’t go for the synergies across projects, but reap synergies within the projects. The same applies for software logistics, server administration, and so on. Maybe here too, redundancy is good for you.
