Skip to Content

The other day, I struck up an interesting conversation with a colleague who had some strong views about SCM tools, especially concerning SAP’s new Design Time Repository ( DTR ).

“I’ve taken a look at DTR” she said, “and I still cannot figure out why we had to build our own Version Control tool when there are so many mature tools in the market. I mean, take Perforce or ClearCase – they’ve been around for years and have proven their strength in supporting large scale distributed projects. So why reinvent this technology ?”

“There are a number of sound reasons behind that” I replied.

“I’d really like to hear them out” she said. “To me, DTR looks similar to Perforce – you have the same file/folder checkout-checkin model, ‘changelists’ in Perforce are called ‘activities’ in DTR, and Perforce Branches map to DTR workspaces. Old wine in a new bottle, isn’t it ?”

“Not really. I’ll explain the differences in a moment, but regarding the abstractions like ‘activities’ and ‘workspaces’ – these are from the DeltaV standard which DTR implements and not- “

“What’s DeltaV ?” she interrupted.

DeltaV is the versioning extension to the WebDAV standard. WebDAV defines a standard way to access and manage files over the web. DeltaV adds versioning support to this and defines the protocol to perform tasks related to configuration management and version control. Implementing an SCM tool based on this standard – and not on some proprietary technology the way Perforce does – is in line with SAP’s strategy of creating ‘open-ended’ systems.”

“That sounds okay, but it doesn’t justify the development of a full blown SCM tool to meet this end !”

“True. The reasons for that are different. You work with Perforce from Bangalore, don’t you ? How fast is it ?” I asked.

“Irritatingly slow. But that’s understandable considering the fact that the Perforce server is in Walldorf and we access it over the WAN.” She seemed to have accepted her fate calmly.

“Understandable yes, but acceptable ? I think not. Perforce has only one model for distributed development – a central server ( what they call Depot ) to which clients connect from different places, even across continents. But with DTR, you can also replicate sources between repositories in different locations. So if you were to move your sources to DTR, there will be a DTR Server configured in Bangalore to which you will connect, and the sources you modify will be replicated to the DTR Server in, for example, Walldorf, and also in the reverse direction. “

“How does that work ?” she asked. “I mean, what is it in DTR which allows it to replicate sources in a way Perforce cannot ?”

“Perforce has no concept of propagation of sources in a distributed landscape. You can integrate your changes from one Perforce Branch to another one in the same depot, not across depots. In DTR, the analogous abstraction ‘workspace’ is not tied to a physical repository – so integration of changes is possible between workspaces in different repositories. If the target workspace is in another repository – in another location – then you only need to export your changes from the source repository, import it into the target repository, and then integrate the changes into the target workspace.”

“Much better, I agree. But what about ClearCase – they’ve been supporting replication for years!” There was a twinkle in her eyes; she probably thought she’d got me here.

“True. ClearCase has what they call a VOB – Versioned Object Base – which is a source tree in the repository. So in a way it is something like a Branch in Perforce or a Workspace in DTR. And a VOB can be distributed over a WAN – this is done by having multiple replicas of a VOB, and allowing users in different locations to work with their replica of the VOB. Then, at regular intervals, you can synchronize sources between these different replicas and thus get the changes made in other locations.”

“Just what we need, isn’t it ?!” she asked, still excited.

“For distributed development, perhaps yes.” I said. “But SAP has other requirements which ClearCase does not fully meet. The most important one is support for delivery of sources to customers. As you know, SAP delivers its application sources to customers, and these sources can be modified by them. So we need an SCM tool which supports the functionality of propagation of sources to customers in such a way that it satisfies some important attributes.”

“What attributes ?”

“Firstly the tool must support upgrading software delivered earlier such that customer modifications are not overwritten. This is possible only if the tool is able to automatically detect conflict situations caused by propagation in any direction. In DTR, this is possible since it maintains a global version history – a version history of a resource that spans multiple repositories. So if you deliver a specific version of a file to a customer, additional meta-data related to its version history is also transported so that at the customer site the full version history can be calculated. This is used to decide whether or not a conflict must be reported when the next version from SAP is received.”

“Perforce has a linear revision history containing revisions in a single branch, right ?” she asked.

“Correct. Perforce uses what it calls ‘Inter file branching’, which means that when you integrate a file to a new branch, a new file is created. In DTR, this only results in another version of the same file.”

“But Perforce does maintain the branching information, doesn’t it ?”

“It does, but that is just a pointer from the source to the target branch. It does not keep track of subsequent integrates that were performed in both directions, and hence cannot indicate conflict situations correctly. For instance, back integration from a ‘correction’ branch to the ‘development’ branch always brings up a conflict, even though the development branch version was never modified.”

“Oh ! So that’s why we always had to do double-maintenance with our projects in Perforce ! If we could integrate in the reverse direction it could save us a lot of manual overhead!!”

“Yes.” I smiled. “With DTR you can simply integrate your bug fixes of older releases into the new releases. A conflict is reported only if the file was modified in both workspaces.”

“What are the other attributes related to this propagation functionality ?” She seemed more curious now.

“Another important attribute is sequence independent propagation. In a distributed landscape where there could be more than one provider of software – like SAP, and some partners of SAP – and more than one receiver of software – like SAP Partners and customers of SAP – the SCM tool cannot depend on the sequence of propagation. Current SCM tools which support propagation work on the assumption that if change A was done first, followed by change B, then these are transported and received in the same order.”

“Hmmm…”

I was not too sure if she understood the importance of this functionality. I continued anyway.

“Also, the granularity of propagation should be flexible. With DTR, changes are grouped in ‘activities’, and these activities – which contain versions from a set of files and folders – are units of propagation. Additionally, we can select arbitrary versions into another unit of propation called a ‘Propagation List’, which can contain any number of versions of distinct resources.” I looked at her to see if she had understood.

“So normally all changes recorded in activities are propagated, but we havethe additional flexibility of choosing precisely what versions we want to propagate – is that it ?” she asked.

“Thats right.”

“All this is fine, but if I remember correctly, ClearCase also has branches and supports merging across branches – right ?” She really was persistent.

“In ClearCase, a branch is something you create for every individual resource – there is no “global” branch which applies to all resources in a given code-line. This causes problems with administration and control.”

“How ? I did not understand. “

“If you remember, a VOB in ClearCase is like a source tree. But each resource – a file or folder – in this source tree can have complicated version graphs, based on the branches created for each resource. (So in this sense a VOB is like a virtual repository). Now let us say that a bugfix for release X needs a modification of file F1, F2 and F3. This must be performed in a separate branch for each file. To keep things under control, ClearCase recommends to use the same branch name while creating branches for changes that belong together. If this is followed, then a ClearCase View can be configured such that elements on this branch are visible in the context of this view. Since this decision – of naming the branches for individual files – is left to the user ( or the person who configures the View ), it is not comfortable in organizations where centralized and automated control is important.”

“What is a ClearCase View ?” she asked. I had used this term without explanation, and she probably liked to have a clear idea of everything.

“A View in ClearCase is like a ‘Virtual Workspace’, which provides isolation through access to a specific set of versions of distinct resources in a VOB. Think of it as a private storage – it allows different users to work independently on the same VOB.”

“So when I create a View, can I select any version of a resource in a VOB ?”

“Yes you can. So if you want to work on a bugfix of a release X, then you can create a View by selecting the latest version from the Bugfix branch of resources which must be modified for this fix, plus the latest version from the Main branch of all other resources.”

“That conveys a lot of flexibility, doesn’t it ?”

“It does, but that turns out to be a disadvantage in our case. We want to have control over what versions our customers can see. If we do not have this control, then customers could define their own views to the repository and ignore versions shipped by SAP, which could frequently lead to inconsistencies in the state of the software.”

“So how is this situation avoided when one uses DTR ?”

“DTR is well integrated into the Java Development Infrastructure of the Netweaver platform, and it is the Change Management Service – CMS in short – that creates and configures workspaces in a landscape. Once the workspaces – the codelines like Development, Correction, Consolidation – are created, things proceed in a controlled manner. So users cannot create workspaces on their own – they work on workspaces created and administered centrally. This applies to customer landscapes as well. So when an upgrade from SAP arrives, the workspaces – codelines, or branches – into which the changes must be integrated are known, and so we do not have any surprises after the upgrade.”

“So if the upgrade contains a new version of a file which the customer has modified, a conflict is reported, right ?” she asked.

“That is right. Since DTR maintains a global version history, it can calculate – based on predecessor successor relationships in the version graph – whether or not a conflict must be reported. And of course, DTR provides tools to view the differences between the two conflicting versions, and merge them – either automatically or manually.”

“That is fine. But assuming the customer resolves his conflict in his ‘development’ system – or workspace – then when the upgrade is imported into his ‘consolidation’ workspace, the same conflict will be reported there also, right ? So he has to resolve it there again ?”

“No. When you resolve a conflict in DTR, the merge arrow is persisted, and propagated along with the merged version that was created. So if you resolve a conflict in the ‘development’ workspace, and then integrate this change into the ‘consolidation’ workspace, the conflict in that target workspace will be automatically resolved due to the merge arrow that was also part of the integrated change.”

“Sounds cool ! I get the impression that apart from covering the functionality offered by major SCM vendors, DTR addresses rather well the specific issues that arise in the development areas within the landscape of SAP and it’s customers.” She seemed convinced, finally.

“Quite correct. You’ve probably noticed that we follow the ‘main-line’ model for development, which means changes in all branches are periodically integrated back to the main code-line ( or branch ). This was adopted based on our experience – it was found that most of the development within SAP follows this pattern. Now if you look at the strategy adopted by Perforce – Inter File Branching – it becomes clear that they do not expect integrates back to the main line of development ( and hence their model does not support such integrates well enough ). That strategy is good when you create a branch to work on a related but independent area, like, for example, a branch for platform dependent development. Since platform dependent code of a product is placed into a different branch, it is not expected to be integrated back to the main branch ( which contains platform independent code ). Since most of the development in SAP does not follow this pattern, this strategy of Perforce is ill-suited for us.”

“And the features needed to support delivery of software to customers – they seem to have been mostly ignored by leading SCM vendors !” She seemed a bit surprised that other vendors had not ventured in this direction.

“Again, this is specific to our requirement. Typical customers of SCM tools only want to use the tool to support their in-house development, and this need is met to varying degrees by different tools. As we saw, some tools like ClearCase are quite generic and flexible, but when it comes to our special requirements – like the support for regular delivery of sources to customers – they fall short of the functionality we need.”

“It is clear now.” she said, with a smile.

“Well, DTR is the result of years of experience with complex, distributed development landscapes. We learnt a lot from the predecessor of DTR – a versioning system called Mobile Application Repository ( MAR ) which is used by the Mobile Application Studio to store and version the CRM Mobile applications shipped by SAP. The MAR already contains a few of the features I’ve talked about, and having gone through the experience of building and productively using a large scale versioning system across a few releases has given us valuable insight into what works and what does not. Its a pity that SAP does not sell technology – otherwise DTR could have given nightmares to all major SCM vendors.”

“Considering all that you have said, I’m beginning to think so too. But I still have one complaint – why such a name like ‘Design Time Repository’ ? It sounds so … so technical, doesn’t it ?!”

“Er..um..well…”
 

To report this post you need to login first.

8 Comments

You must be Logged on to comment or reply to a post.

  1. Jeremiah Stone
    Thanks for the informative post, Manohar!

    When I first saw the rollout materials for the DTR, I was incredibly excited.  Having worked in industry before coming to SAP, I was initially amazed at the powerful content management tools that most SAP developers took for granted.  In the world that I had come from, delivery and patching of the product was one of the most challenging and error-prone phases of the software development lifecycle.  “If only we had something like this for the non-ABAp world”, I remember thinking.  Hallelujah! 

    The DTR promises to be a revolutionary offering for the market.  It’s great that there will finally exist a versioning system that supports globally distributed development, and delivery of source to customers.  My only question is, “why didn’t someone think of this years ago?”

    Best,

    Jeremiah

    (0) 
  2. Robert Neher
    Thank you for this interesting posting. It gives a lot of arguments to reconsider ClearCase decisions.

    What could motivate ClearCase heavy-users with several gigabytes (terabytes?) in repositories to migrate to DTR?

    These artifact cemeteries contain a lot of investments aggregated over time.

    Is there any ClearCase import possible?

    Robert

    (0) 
    1. Manohar Sreekanth Post author
      Hello Robert,

      DTR will be a part of the Netweaver platform on which SAP Applications will be built. Customers who by these SAP applications will get this tool as a part of the development infrastructure, which will allow them to customize and maintain these applications in their landscape.

      Since we see no requirement to migrate sources from ClearCase, no such import tool has been planned.

      Regards,
      Manohar

      (0) 
    1. Manohar Sreekanth Post author
      Hello Robert,

      Currently there is no such comparison available. There is even no urgent focus on this issue, as the value proposition of DTR is more in the direction of the availability of features not offered at all other SCM tools currently in the market.

      SAP will be the first customer of its own product, and therefore we will take a stepwise approach to migrate SAP’s current Non-ABAP Development Environment towards SAP’s own integrated Development Infrastructure (currenty named as JDI). This process has just started, and as the number of users grow we will gain experience regarding performance and sizing issues of large scale implementations. This experience will, of course, be shared with our prospects and customers for sizing issues and TCO calculations.

      Nevertheless, while positioning our product, our focus is not to compare simple perfomance figures with those of pure SCM vendors, as we have identified more relevant Key Performance Indicators when looking at the whole tool-based Development Process (i.e. TCO, ability to optimally support development and maintenance efforts in the context of extending/modifying/customizing SAP Solutions, average turn-around time for bug fixing in complex component-based development projects, etc.).

      Regards,
      Manohar

      (0) 
  3. Community User
    Hello Manohar,

    the DTR description sounds pretty impressive. I wonder if there are common roots to the open source system called subversion, which is also based on WebDAV and DeltaV?

    Besides, we as an ISV have a WebDAV based Repository system. Would it be possible to access a DTR Depot with it, what do you think?

    And of course a minor note out of curiosity, can I use the DTR plugin in plain Eclipse 3.0? I was already plaing around with the WebAS Dev Studio, and it has so many nice plug ins, it is understandable it does not track the latest milestone builds, even if it would be nice.

    Greetings
    Bernd

    (0) 
  4. Girish Ogirala
    Thank you for sharing the insights into the DTR Capabilities. Now there is only one wish, to bring ABAP Code out of SAP R/3 (at least Custom objects) and use DTR to TRANSPORT them between different SAP Systems.

    Girish Ogirala.

    (0) 
  5. Jan Van Achte
    Manohar,

    Assuming DTR is as powerfull as you say, what should we use to report on its contents ?

    We are currently trying to convince our J2EE developers to move to SAP J2EE, and one of the things we are missing are software development mangement tools like (cenqua’s) Fisheye.

    What is SAP using to monitor/manage/report on the assets with DTR ?

    Thanks

    Jan Van Achte

    (0) 

Leave a Reply