First of all, I have 😉 to say that the following recommendations are of my personal opinion, heavily influenced by latest trends in enterprise SW development like community dicsussions, personal experience and books. It's actually pretty hard to discuss this topic, because it involves a lot of tiers, both physical and logical. I will try to focus on the technologies application servers provide us. I will not touch hardware questions here, because they deserve their own coverage.
When we talk about application servers, we talk about middleware. We talk about services, exposed by the application server to the applications run inside via the standardized APIs like JTA, JDBC, JCA etc. It is also common to talk about application server having several containers, like web container and ejb container.
Myth
Typical myth of J2EE applications is that you have to use EJBs if you want scalability. Experience shows that from a number of beans EJB offers us, only SLSB (stateless session beans) and MDBs (message driven beans) are actually quite useful. If you need remote access using RMI/IIOP for some Swing application, for example, if you need declarative transactions and if you need security, then SLSB is your choice. They scale pretty well because there's no state to replicate and you can employ any load-balancing algorithm you like.
In most cases you do not need EJBs. Putting the logic of the application in the servlet tier usually provides very good results. You can achieve perfect horizontal scalability, replicating http session information over the network or via the database, as SAP WebAS does (if configured this way). Actually WebAS employs the concept of sticking sessions, and persists session modification in the database for fault tolerance reasons.
In your servlet tier you still have acess to all J2EE APIs. Of course, you might say - with EJBs I have declarative transactions, what about that in servlet tier? Are we supposed to code everything by hand? Well, there are a number of aproaches for solving this problem. The simplest is to employ Decorator pattern, though it requires a lot of manual hand coding. The better way would be to use one of the frameworks, which allow you to use declarative features of AOP (Aspect Oriented Programming). Ahm, sorry, this post is supposed to be about scalability ;). So, I must say, given that you have a stable design in your application, this problem becomes not so big.
Horizontal Scaling
When we talk about horizontal scalability, we talk about adding machines to the cluster; we talk about load distribution. One of the most common scenarios today is to have hardware load balancer, distributing requests according to a certain algorithm to the application servers, sitting behind the firewall. Application servers in turn talk to the database(s), legacy systems, message queing systems, whatever else. In an SAP landscape, WebAS talks to SAP R/3 system via JCA over JCO over RFC ;). But, again, the most common scenario is that application servers mostly talk to the database.
So, the question arises: what if the database becomes a bottleneck? Some DB vendors already provide solutions for clustering their databases, exposing them as one logical database, though in fact, it's a number of physical boxes, communicating to each other over a network. But some databases do not have such a possibility. Luckily, with WebAS and it's OpenSQL layer you have an option to switch to another database. In any case, whatever vendor you choose, I would recommend to make a good load/stress testing of your deployment on the test environment, verifying that you can handle the most heavy load you can expect, before going live. Of course, sometimes it's not that easy to determine the amount of users, which would use the application, especially when it's some kind of a public portal. But, still, it can be estimated in the phase of requirements gathering.
If you're interested in getting more information on this topic, please check documentation and presentations, available here on SDN. If you're really interested in this subject, please keep in mind that proper architecture and design decisions do influence scalability. And, what's even more important, given that you apply best practices in your project and code to interfaces, not classes, it becomes possible to make architecture refactorings. I remember myself doing this. I participated in one telco project and on a certain stage we decided to evaluate entity beans technology. It was several years ago. I can say that performance problems with it were spotted nearly instantly, because we progressed with our work using quick iterations, sending builds to the QA team every week. So, after one of the first load/stress tests it was found that entity beans perform inadequately slow. I remember myself getting rid of them in half a day. Of course, that wasn't too much code, because evaluation of the technology and vertical slice building was occurring at the very beginning of the project.
Your application can perfectly meet functional requirements, but fail to meet service level requirements. That is why it's so important to think about these in the earliest stages of the project, and create architecture and design with this knowledge in mind. It's even more important to do continuous tracking and evaluation of architecture and design during project evolution. In most projects, requirements change on the fly and sometimes it resembles shooting a moving target. Well, here we come to the topic of processes and best practices, which might be the subject for one of the next posts.
