Skip to Content

Notes from the Ivory Tower Part 4: Redundancy is Good for You

In larger enterprises, there is a strong and venerable tradition of trying to avoid functional redundancies. Especially senior management makes it a priority to ensure that, if possible, nothing is every done twice, or in two different ways, or by two different people or groups.

Cherished enterprise traditions and lore

In many companies, when a senior manager realizes that anywhere in the company, two similar or seemingly identical problems are being solved with two different tools, or by two different groups, they lurch into action like superheroes. The redundancy is perceived as a problem per se. The reasoning behind this is the following:

  • It’s a waste of resources to solve the same problem twice (and adapt the solutions twice too, as time goes by).
  • There has got to be one single best way to do it! Let’s find out which is best and eliminate the other one. (I really like this argument.)
  • Too many different procedures increase complexity in the organization. If we permit more than one procedure, an infinite number of processes will sprout and chaos will destroy the Earth.

Swiftly, meetings with everybody involved and their bosses and their bosses’ bosses are arranged. The culprit who has sinned against the “standard” process (standard = whichever came first) must repent and is let off the hook after agreeing to provide input to help improve the original process. When the dust has settled, our manager is happy because he has successfully eliminated a redundancy in the company’s processes and saved a small piece of the world from chaos.

To their credit…

I’m writing this slightly tongue-in-cheek not because the reasoning behind eliminating redundancies is wrong – most of it isn’t and in many cases it’s quite reasonable to proceed as described above. What makes me grin is the way the behaviour seems to have become automated: It’s an absolute no-brainer. Everybody who takes part in it could script the whole process, do it in their sleep and without a conscious thought. The outcome is clear from the beginning. And it makes people feel good, too – now why shouldn’t that be a reason to smile?

This is a software blog, right?

Yes, I just took the scenic route. Now, why can the above be a problem for software developers? It can be because the same procedure, with the same degree of automation, is applied to software architecture and design. Suppose the manager encountered above is a senior manager in software developing, overseeing a number of different products. Now he realizes that two project teams are using different archiving tools. This can’t be right, he thinks, and lurches into the well-known routine.
I’ve seen this happen for absurdly small-grained development entities. I’ll give you an example.

A real-life example: A design decision and the consequences

Two huge enterprise applications (each built in hundreds of man-years) which are part of a same product suite use the number X. X is a single customer-specific number that is maintained in a customizing table and changes once per year. It is a well-visible and important factor in a number of calculations across the application suite, similar perhaps to a tax rate.
Elimination of redundancy by management resulted in the design decision that the number would be stored in a cross-application table located in a cross-application software layer developed in a different development system. And the consequences?

Consequence 1: More meetings indefinitely

Quickly it was realized that both applications needed not only the number X. There are a number of parameters and customizing values which are closely related to the number X. So the table was subsequently extended by another field whenever one of the applications thought they needed something related to X.
The shared table being common property, the teams need to agree upon every change. Whenever changing business processes or legislation forces one feed to make a change (quite frequently), they need to travel, meet with the other team and discuss it.
The cost of cross-project, cross-application cooperation can be quite high: organizing the meeting, preparing the slides, travel time, the actual meeting, writing a protocol. And all of this before a single line of spec (not to mention code) has been written.

Consequence 2: Software logistics

Another unwanted side-effect are higher costs for software logistics. Because cross-application development takes place in a different development system or track, you need to get the code (or dictionary object) from where it is written to where it used.

Consequence 3: Tight release coupling

When two applications both define and consume a shared API, they are extremely tightly coupled. None can move without the other moving in perfect sync.
Examples from release management hell:
Suppose application A requests a change.

  • CA (cross-app development) needs to have the resources to make the change.
  • B needs to have the resources to adapt to the changed API.
  • A and B must wait for CA to complete its work, which results in a smaller time frame in which they must have spare resources.
  • If A and B run on the same production system, the change in A triggers a new production release of B and everything else that depends on CA.

Let’s have a closer look at release cycles.

  • A is rather new and puts out new releases quarterly.
  • B is older, has a low release frequency and puts out one major release per year.
  • In the worst case, the coupling through CA can lead to a battle about when the changed CA goes live.
  • If A looses, they have to wait for the next scheduled release of B.
  • If A wins, B is forced to put out a release quickly, which can be very difficult if their software is undergoing a major reconstruction or preparing for a major release.

Needless to say, assessing the complex situation and weighing the alternatives consumes a lot of time and money.

Consequence 4: Unclear semantics

Some of the columns of the shared table represent a semantic overlap of two business areas. Where this occurs, a closer look typically reveals slight differences. Tax calculation in A and B is almost identical, only in B there’s an exemption rule that A hasn’t got. Or A and B both use the “last day of contract” information, only for A there’s a rule that the effective end date is at least one calendar month after “begin of contract”.
This kind of semantic overlap makes it difficult for developers to know how exactly to interpret anything in a shared API. If the documentation is not very, very concise, the trouble is that asking experts from A’s business domain does not provide a sufficient answer, because freak cases from B’s business domain might causes unexpected values, or cause unexpected behaviour, and vice versa.
The true semantic overlap is always much smaller than it seems. A vague semantic overlap causes endless troubles.

Consequence 5: Difficult testing

In the worst case, the cross-application code can only be tested in a different system. Each iteration of the test cycle is expensive and takes a long time because of the costs and delay resulting from cross-system software logistics. (Also, the quality of the software is likely to suffer from poor testing.)

Consequence 6: No redundant development

The sixth consequence is the only one that ways actually intended. Instead of two customizing tables, only one was developed. Also, customers maintain the value of X only once (per customizing system) whenever it changes.

Weighing the costs for the organization and the software

Although reasonable when seen for themselves, the advantages of consequence 6 seem rather lousy when weighed against the costs and risks resulting from consequences 1 through 5.
The costs and risks of tolerating a redundancy must always be weighed against the costs and risks of a coupling through centralization.
For organizational processes, the cost of coupling is frequently lower and risks are not so high. Because business processes can be changed fairly quickly. Though it may not always seem so, organizational processes are still more agile and flexible than software. That is why centralizing functions is frequently a cost-saver in organizations.
In software development, centralization is frequently a cost-driver. The high complexity of enterprise software makes it difficult to oversee the consequences of making changes in a part that affects others. Reducing the scope of changes is achieved with encapsulation, which typically increases redundancy as a necessary side-effect. So frequently in software development, redundancy is good for you.

Encapsulation as a design principle for the organization

Some of the benefits attributed to encapsulation and higher degrees of redundancy in software can also be reaped in the organization of the development process. Decoupling departments in an organization can be just as beneficial as decoupling software components, if the cost of working together is high.
Suppose your central QA department is physically and conceptually far away: It costs time and money to travel, and time and nerves to connect on a conceptual level, until each side of the table understands the perspective of the other side. Also, your QA department may not be able to support your development project when you need it, so you have to wait for them or speed up your development to find a common time frame. These costs must be weighed against the advantages of a central QA department: standardized QA methodology across projects, synergies due to centralization, and so on.
If in your particular case, the costs of extra-project cooperation outweigh the benefits, you might be better off with a more decentralized approach: Have a small QA team within the development project. Don’t go for the synergies across projects, but reap synergies within the projects. The same applies for software logistics, server administration, and so on. Maybe here too, redundancy is good for you.

You must be Logged on to comment or reply to a post.
  • This is an excellent blog, and the rationale is clear. nicely done.

    However, let us say we extend the scenario and say the company developing this is SAP (easiest to pick on since it is the most familar to all of us LOL)and users are the hundred thousand clients of SAP. Now, all your points still apply to SAP themselves. But if SAP internally decided to allow this redundancy – guess who will have to live with the consequences – the many clients who use the two modules.

    Now the clients have to have two tables to maintain, two tests to do every time something changes and so on.

    So depending on the scenario – it might still be better to bite the bullet and live with the pain of redundancy. After all a software powerhouse like SAP or Microsoft mostly should have better organized development practices to minimize these compare to the client teams.

    • Hi Vijay,
      I agree with you. In scenarios with many layers of software and cross-vendor dependencies (customer – ISV – SAP application development – SAP ABA – SAP BASIS), the side effects of design decisions can multiply, and it’s wise to look beyond the development costs and also consider effects on the customer’s Total Cost of Ownership.
      I have the impression that SAP is quite good at enforcing reuse, allowing bigger redundancies only very rarely and for strategic reasons: I’ve never seen any ABAP component bring along their own BAL, and I understand why the Webclient UI doesn’t use Web Dynpro (although I wish it did).
      Best regards,
    • Hi Achim,
      good to hear from you. Please do remember to check your email every once in a while. 😉
      Thanks for pointing out the blog about Excel – I discovered this brilliant Joel on Software blog only recently and haven’t gotten to reading many of the very old entries yet.
      “Not invented here” is a syndrome software designers should eventually get over as part of the process of growing up. The art is to develop a good judgment of how good the stuff can come up with really is in comparison with a library that may have many bugs eliminated and whose code may cover use cases you haven’t thought of, but will run into, and to consider the costs of the dependency realistically.
      The more we learn about software design, the more heavily we rely on gut feelings, and the better we become at disguising this fact. That’s why the most controversial discussions are always the ones with the foundest arguments. 😉
  • Hi Torsten,

    I really appreciated reading your blog. You make a good and interesting point. Some thoughts …

    On what level should you redundancy? You mention having small QA teams. Do you mean that every team should come up with their own QA rules with no centralized guidelines and that it will help the decentralized projects? Why not make every project member responsible for their own QA?

    Lets reverse your table example: Every developer works creates his own table.

    Consequence 1: More meetings indefinitely. All developers need to attend all meetings to get an understanding for what needs to be done.

    Consequence 2: Software logistics. Multiple transports for all developers.

    Consequence 3: Tight release coupling. There is no release as every developer sends his changes to production when the development is done.

    Consequence 4: Unclear semantics. Every developer has its own interpretation of the business process. It is impossible to get a common answer on how the functionality works because every implementation is slightly different.

    Consequence 5: Difficult testing. Every development of all developers needs to be tested.

    Consequence 6: Redundant development: You get the intended redundant development.

    I believe the question s where to draw the line of “separation of concerns”. In my opinion, the table example is not a good “separation of concerns” and therefore causing the issues that you have experienced. I would have suggested encapsulating the two initial tables with a common interface. In that way you get redundancy but with a common interface (“guidelines”) that provides a contract with the calling programs.