Data Replication Architectures for Integrating Heterogeneous and Non-Heterogeneous Systems

[Javier-Martin]

This is the second post pertaining to the topic of integrating heterogeneous and non-heterogeneous systems using SAP Sybase Replication Server. In this post I’d like to outline the architecture of an SAP Sybase Replication Server system and describe the individual technology components comprising the architecture.

The following diagram shows the big picture for the architecture and it is valid for both heterogeneous and non-heterogeneous systems:

COMPONENTS

1.- Primary database and replication agent

As stated in my last post, primary database can be ASE, ASA, Oracle, MSSQL and DB2 and replication mechanism is log based delta replication. Replication agent is the component responsible for reading transactions from the primary database transaction log and sending them to Replication Server. There are 2 types of replication agents:

• ASE database replication agent  is an internal thread inside ASE.
• ASA, Oracle, MSSQL and DB2 replication agents are separate processes and in some cases they can even run on a different machine than the primary database server.

2.- Replication Server

Replication Server is responsible for receiving transactions from replication agents and sending them to the replicate databases.

Note that:

• A single Replication Server can have several primary databases (they can even be from different vendors) and it can also have several replicate databases (also  from different vendors).
• Replication mechanism follows a publish-subscribe model. You publish data in the primary databases and subscribe the replicate databases to the information they need.
• There are 3 kinds of objects that can be published for a primary database:

• Tables: Replication Server implements a selective mechanism for columns and rows in the primary table so that you don’t have to replicate the complete table if not needed. It is possible to specify a list with a subset of columns or use a condition to replicate a subset of rows.
• Stored procedures: Calls to the stored procedures are replicated with the parameters specified. The stored procedure code can even be different in the primary and replicate databases.
• Complete database: In this case the replicate database is a warm-standby database that can be used as a software HA solution. This is supported only for ASE and Oracle.

In SAP Sybase Replication Server we can distinguish three main modules:

1. Replication Server Catalog: This is a database that stores metadata information for the replication server system (replication server connections to primary and replicate databases, information published at each primary, subscriptions to each replicate, information about message repository … etc). The supported DMBSs for storing the catalog are ASE and ASA.
2. Rules Engine: Internal mechanism to determine the information published at each primary database that has to be replicated to each of the replicate databases.
3. Message Repository. This is the persistence layer that guarantees transactional and sequential consistency in replicate databases. It basically consists of one or more physical devices (could be file systems at OS level or raw partitions) responsible for storing messages that are sent from primary databases to replicate databases. In case of any problem or failure  this persistence layer allows Replication Server to recover exactly from the point it was when the problem or failure appeared.

3.- Replicate Database

      There are three kinds of possible replicate databases or systems:

• SAP Sybase servers: ASE, ASA (SQL Anywhere) or IQ. In this case connection from Replication Server can be established directly without using any middleware.
• Non SAP Sybase servers: Oracle, MSSQL, DB2, Informix, AS400, SAP HANA and theoretically any DBMS system accessible via ODBC (SAP HANA will be certified Q1 2013). In this case we need a component responsible for the connection to the Non SAP DBMS. We have 2 kinds of components:
  • Direct Connects:  Separate server processes that can run in a different box from Replication Server and from the replicate database.
  • Express Connects: Modules shipped with SAP Sybase Replication server that is able to directly establish a connection to the replicate database.
• Rep Connector: Used for sending information to a messaging service. Connectors for following messaging services are  available:
  • TIBCO RV.
  • IBM MQ Series.
  • Sonic MQ.
  • JMS.
  • Custom: There is an API that can be used to build a custom connector

That’s all for now. In my next blog I will show how to configure heterogeneous replication from ORACLE to ASE.