Notes from the Ivory Tower Part 1: Why Now's the T...

former_member182046 · ‎10-22-2008

A Software Zealot is someone who defends or enforces some aspect of software engineering with great zeal. Now, to a laissez-faire guy like me, what an earth could be sufficiently important to make me act like a religious fanatic? I can name only two things which do that to me. The first are software bureaucrats not letting developers do their work, but I shall write about that subject another day. The second is unnecessary architecture degeneration.

Reasons for Architecture Degeneration

Maybe it is because I’m passionate about software development, but I just hate it when because of easily avoidable mistakes, good code goes down the drain by becoming irreversibly tied to a single, fleeting business process, or by becoming impossible to maintain due to spaghettization because stuff that belongs elsewhere is squeezed in.

Sometimes, a development team is short on resources and convinces management that a particular development task should be given to a different team. When this also leads to the code in question being allocated in the other team’s application, or packages, or software components, software architecture will almost certainly begin to stink. This is especially true if moving a task from one team to another leads to remote calls at run-time. I’m sure you’ve seen it happen, too. (If not, you probably live in the Athens of software development.)

Best Practices for Slicing and Dicing

A clean and well-designed component concept, slicing an otherwise monolithic body of software into a number of highly independent and de-coupled units (components) is crucial to any larger software development effort. It is indispensable for keeping maintenance costs low and release cycles short. The way you cut your software determines many important properties of the software:

Which parts you can take out of context and reuse in new and different scenarios
Which parts you can throw away after they have gone bad without throwing out the baby with the water
Which parts are affected by an internal change anywhere in the software
Which parts must go through every stage of the software lifecycle hand in hand (e.g. A’s new release cannot go out before B’s new release is also completed and through QA)

Clearly, making the wrong cuts across your application landscape and software components for whatever reason can cause great pain in the future, and in my experience always sooner than anticipated.

It’s hard to say how to make the right cuts. We cut software horizontally (often according along the information dimension), and vertically (often along the aspects and concerns and by degree of reusability). And then we sort of hope that the business processes will fall into place within one single software application, or larger component, and without crossing borders between components too often from beginning to end.

Amazingly, this method works most of the time, and cross-component business processes are the exception rather than the rule. (This is not a coincidence, however, because software design relies heavily on intuition, which means that many design criteria – e.g. maximizing process coverage of components – may be unconsciously but nevertheless strongly effective in the design process.)

Now, that’s all good and well, but why should one turn into a Software Zealot? Is there any imminent danger of software decay that wasn’t equally threatening three years ago? What in the world should cause us to get worked up and start preaching to others? Actually, there are a few things that could cause a lot of trouble if we don’t pay attention, and if we don’t herd a few stray lambs back to the flock.

1. It has never been so easy to provide and consume services cross-component, cross-application, cross-system, cross-platform, cross-organization.

When you exposed a BAPI three years ago, chances were it would be consumed locally, by neighboring applications. If you ever exposed an interface for the Oracle guys in your company, or with the MS-Access crazy student supporting the sales department, it was a big deal and occurred rather rarely. Consequently, the degree of cross-application connectedness in IT landscapes was generally quite low because of the technological hurdles.

Today, service publishing is done in three quick wizard-guided steps, and service consumption from whatever remote platform is only a few mouse-clicks away. All you need is the WSDL URL.

All else being equal, the result will be a huge increase both in the number of exposed services and in the number of service consumptions, which in itself is neither good nor bad, but may cause secondary problems as we will see later on.

2. Modeling methodologies are focusing strongly on business processes, sometimes not paying enough attention to information modeling.

Business modeling, for example in the context of SOA projects, focuses strongly on business processes and derives services definitions directly from the processes. Information modeling, business objects, and applications are easily forgotten or deemed irrelevant when it seems sufficient to model the processes in detail.

Yes, sure, we can always go cross-component, cross-application, cross-system. But if new backend functionality is developed without being reasonably allocated in a component, application, or system on the backend side, we are begging for trouble.

3. Methodologies focus on software users, while vendors make it up as they go along.

SAP, IBM, and other big SOA consultants visit prospective customers in order to sell their SOA methodologies. These include process modeling, service derivation, implementation support, and design-time as well as run-time governance methods and tools.

Working for a software vendor, I found that what we were offered could at best be used on a “take it apart and use some parts to make something new” basis because every step of every method assumed that we run the business processes modeled productively, have a corresponding department, etc.

It may be sheer ignorance on my part, but it seems difficult to find method framework for modeling, service derivation, and governance, that is aimed at software creators and vendors who don’t run the software productively, but design, program, deliver, and support the software for an external market. I guess everybody just makes it up as they go along, and is reluctant to let others in on their secrets. (I’d be most grateful if you point me to some useful resources, for otherwise I’ll have to make up a vendor-oriented mini-methodology myself.)

4. There is still a lack of best practices.

Again, I may display a shocking amount of ignorance here, but I believe we are still short of best practices.

It’s going to be years until you can speak with IT professionals about introducing SOA without provoking a load of questions about modeling tools, governance processes, policies, monitoring, deployment, single sign-on and identity management.

Take, for example, some questions involved with web service deployment: Tool support for handling UDDI content in development and production landscapes as well as mass configuration of web services is still fairly new. Concepts for modification-free enhancement of service definitions and implementations have emerged. Identity management solutions are competing. The everyday, bread-and-butter methods and tools of software engineering are one by one being translated into the SOA paradigm.

Once you can say “We run a plain vanilla SOA landscape” and just get a profoundly bored nod you will know that a set of well-established best practices is in place. (You’ll probably say it with a sigh and wish your company was more open to modern ideas.) In the meantime, we’ll all be busy doing the perfectly natural thing: making beginner’s mistakes.

Not a pessimistic view

We’re in a wonderful situation: Hurdles previously posed by platform specific interface technologies are quickly fading away. We’re able to lower the cost of software development because every function within our IT landscape is literally at our fingertips. The widespread use of standards gives us an immense wealth of tools that can be put to create use.

Increasing the degree of connectedness will make our IT more childlike, just as an infant’s imaginative brain is hyper-connected on the neural level compared to a boring adult’s brain.

Most of the difficult questions have been addressed. Governance is a big issue because everybody understands that the transformation of IT landscapes requires a structured approach. Companies will find their individual balance between creativity-killing committees of SOA bureaucrats and cancerous wild growth of unmonitored service provisioning and consumption.

Some warnings, though

The strong focus on business processes, combined with an enthusiasm-induced temporary blindness for good architecture on the application and software component level and the natural tendency to make beginner’s mistake when transforming application landscapes into service oriented architectures, may result in sloppy horizontal cutting of software and massive degeneration of software architectures.

In particular, more closely coupled applications can lead to software paralysis if previously encapsulated applications are coupled through unstable interface. If that happens, business changes will trigger new software releases more quickly than developers can put them through the software lifecycle, resulting in high software costs and unhappy customers.

The increase of connectedness may also result in a decrease of functional and data redundancy across application landscapes, which looks wonderful but hurts when previously identical requirements diverge and the shared code has to do the splits in order to implement the superset of all requirements rather than their intersection

What can you do?

If a SOA project is on the way in your company, and if new modeling methodologies with a focus on business processes are being implemented (along with new tools and templates), make sure you’re part of the initiative and become the software structure person.

In their enthusiasm, your colleagues and consultants may not want to hear about it, so you’ve got to make a stand for the static aspects of the software architecture.

Do make sure that when process descriptions on a purely business level are turned into technical concepts, there is a clear assignment of business applications and software components, and that all the tedious but nevertheless important questions concerning the package concepts of the affected components are answered. This concerns the package assignment of data types, design-time and run-time dependencies of packages and software components, and the definition of well-defined package interfaces. Different applications should be only be loosely coupled at design time, while permitting for closer coupling at run-time. My colleague Tobias Trapp has written an excellent blog about this topic: ABAP Software Architecture - Modularization and Composition of Huge Applications.

The bottom line

In Service Oriented Architectures, making the right horizontal and vertical cuts through software may appear to have become less important because the focus is elsewhere. On the contrary, it is more important than ever before. Failing to observe this dimension of software engineering will be penalized quickly and result in rapid degeneration of software architecture as well as reduced re-usability of code.

Nag your colleagues not to forget that, even if they call you a software zealot. And remember: You can always preach water and drink wine.

Link to the weblog series

Notes from the Ivory Tower (Introduction)