Skip to Content

Folks,

As you know from the previous blog, I’m following along the openSAP course Writing Testable Code for ABAP. We’ve just completed the week 2 and, sorry to say, it was a bit disappointing because of yet another “number in, number out” example. It felt like one of those “filler” episodes in the TV series that suddenly goes on to explore a subplot while a star actor goes to rehab.

We did learn about test-driven development (TDD) principle, which ABAP Unit Test is meant to support. The main postulate of TDD is that you create the tests even before you write any “real” code. So, you start with a test like “given X, Y is expected” and, naturally, it fails immediately because (duh) the code that is supposed to transform X into Y have not even been written yet. If to you this looks on a scale from silly to idiotic and bordering on insane, you are not alone. But any TDD apostle will tell you that this is not at all silly and, quite the opposite, a very effective methodology if only you fully embrace it and change forever the way you think about software development.

Fair enough. I’m not a stranger to the silly routines. For example, I always fasten the seatbelt in the car. No exceptions. Even if I only need to move the car out of the driveway, I still use the seatbelt. There is a fact-based evidence that seatbelts prevent serious injuries and death in car accidents. Even though the risk of having an accident while driving 5 meters at pedestrian speed is rather low, use of seatbelt simply become a reflex for me. Not to mention, it is the law in the state of North Carolina.

As any reasonable person, I like when there is factual evidence. But for some reason the discussions of TDD virtues seem to me less like scientists arguing about two different theories and more like the famous “electrolytes” dialogue in Idiocracy.

In this movie, humanity has become really dumb in future and it is visited by a time traveler, an average Joe from our times. The future humans have been using the drink called Brawndo (“The Thirst Mutilator”) instead of plain water to water the crops because the manufacturer told them that it “has the electrolytes” and that’s what plants need. The following dialogue occurs when average Joe suggests to use water:

“Joe: Okay, look. […] So why don’t we just try it, okay, and not worry about what plants crave?
Attorney General: Brawndo’s got what plants crave.
Secretary of Energy: Yeah, it’s got electrolytes.
Joe: What are electrolytes? Do you even know?
Secretary of State: It’s what they use to make Brawndo.
Joe: Yeah, but why do they use them to make Brawndo?
Secretary of Defense: ‘Cause Brawndo’s got electrolytes.”

There is no shortage of information how awesome TDD is. Honestly, I never even asked for it and somehow already know. Because this week has been slow and low at the openSAP camp, I went to Google to find the other side of story, i.e. the TDD criticism. Great thing about the Internet is you can easily find many facts and people who support your opinion. And then you can turn around and find just as many facts and people who support a completely opposite. 🙂

One of the top results in my Google search was a StackExchange question about the TDD disadvantages and it offers a great answer:

“Test-first assumes you are writing code that is:
– testable in a unit-test manner
– that what you are developing has an obvious approach and will not require extensive prototyping or experimentation
– that you will not need to refactor too heavily or that you have the time to repeatedly rewrite hundreds or thousands of test cases
– nothing is sealed
– everything is modular
– everything is injectable or mockable
– that your organization places a high enough value on low-defects to justify the resource sink
– that there is something of benefit to test at a unit test level

If your project does not meet those requirements you will have difficulty. The promoters of TDD have no good answers to this other to suggest that you redesign your product to better fall within those lines. There are situations where that is impossible or undesirable.”

Prototyping was mentioned in many other posts as an example of when TDD doesn’t seem to make sense. When met with such arguments, “the TDD zealots” quickly retract to “then don’t use TDD” but this starts sending the mixed messages.

Another StackExchange question on this subject might have as well been written by me: a solo developer struggling with the practical use cases.

The two most upvoted answers there (the second one being from Uncle Bob himself) do not really answer the question and go along the familiar lines of “how could you NOT love TDD?”. But slightly less popular answers offer more balanced and practical perspective. For example, there is an interesting comparison with gambling:

“Creating a test is a bet that the cost of bugs in a unit occurring and not catching them with that test (now, and during all future code revisions) is greater than the cost of developing the test. Those test development costs include things like payroll for added test engineering, added time-to-market, lost opportunity costs from not coding other stuff, etc.

Like any bet, sometimes you win, sometimes you lose. […]

Some types of bugs might be so unlikely to be made, or to make it out of any early sanity testing, as to be statistically not worth the time to create additional specific tests. But sometimes the cost of a bug is so great (medical, nuclear, etc.) that a company must take a losing bet (similar to buying insurance). Many apps do not have such a high failure cost, and thus don’t need the higher uneconomical insurance coverage. Others do.”

Development ROI is an important factor. Going back to my seatbelt example, what if it didn’t take just 2 seconds to fasten the seatbelt? What if it took 5 minutes? In that case you’d probably start skipping it when simply moving the car from one parking lot to another. (Side note: if you really want to enforce something it needs to be as effortless as possible. I guess that’s what ABAP Unit Test is meant to do for TDD in the SAP world. Not sure it succeeds.)

There are many other interesting articles offering different perspectives on TDD. This one examines it as a psychological phenomenon: is TDD a form of OCD (obsessive-compulsive disorder)? And this one analyzes the challenges of presenting TDD to the beginner developers.

If you have time to spend, you’ll easily find a full range of posts on how “TDD is the best thing that ever happened to software development”, “TDD is dead”, “TDD is not dead”, “TDD should die”, “why <something> is much better than TDD”. There is even a blog with peculiar title Almost, but not quite, Entirely Unlike TDD (by the way, it explains “code smell” term that confused the heck out of me in the week 1 of the course).

The latter blog features the quote “TDD is a learned skill…” But what if TDD is rather a tool? While discussing the openSAP course with the teammates, I mentioned that it reminds me of some recipes that involve melting chocolate. They start with long instructions on how to build a double boiler. To do that, you need a sturdy pan with just the right amount of boiling water and a heat-proof bowl that is just the right size to sit on top of the pan without touching the water. Then you slowly mix and melt chocolate in that bowl for undisclosed amount of time.

These are fine instructions if you are a kitchen maid in 1870s. In this century, we have microwave ovens that can melt the whole chocolate bar in 1 minute using just one bowl. Many professionals would claim that the double boiler method allows for better temperature control and avoids scorching the chocolate accidentally. Good point. From my experience though, as long as you check and stir the chocolate frequently, it’s hard to burn it and when it’s baked into brownies, not a single person can tell that I used a microwave.

Like with any tools, the real skill comes to finding the balance and best use of them. This is where “TDD or bust” is just as unreasonable as “we’ll just keep doing what we’ve been doing for the last 20 years the same way”. I feel another answer from StackExchange sums this up nicely:

“The biggest drawback I’ve seen is not with TDD itself but with practitioners. They take a dogmatic and zealot approach where everything must be tested. Sometimes (many times that is), that is not necessary. […]

A good engineer finds trade-offs and applies the right balance of when/where/how to apply test-first. Also, if you find yourself constantly spending much more time developing tests instead of actual code (by a factor of 2-3 or more), you are in trouble.

In other words, be pragmatic and reasonable with TDD (or anything in software development for that matter.)”

Amen to that.

P.S. There was sort of a hidden challenge in the previous blog and I’m surprised no one noticed that a slide about debugging featured the picture of a dragonfly. It’s not a bug and is, in fact, a beneficial insect because they are the natural enemy of mosquitoes. But I guess it’s a perfect allegory for TDD. 🙂

 

(Image source: Wikimedia Commons)

To report this post you need to login first.

16 Comments

You must be Logged on to comment or reply to a post.

  1. Jacques Nomssi

    Hello Jelena,

    please also check What Is The Point of Test-Driven Development? the first chapter of Growing Object Oriented Software Guided Tests.

    The benefit of writing test are obvious (clarify the acceptance criteria, encourage loose coupling, add a specification and a regression test suite).

    I see TDD as a valuable approach for skilled developers to achieve test coverage at all times. I have made my opinion about it clear.

    regards,

    JNN

    (2) 
    1. Martin English

      Question 1.

      So… should I (the developer) be designing the tests, or should this be done by (or in conjunction with) someone from the business, or a SAP functional expert ??

      In the real world, the owner of a particular part of the business process may be on one continent, the SAP functional team is on another, and the developers are outsourced by the SI to wherever is currently cheapest. This makes coordination difficult and starts breaking the prerequisites that make agile techniques like pair programming, sprint coordination, and yes, TDD) harder and harder to implement.

      However, if you leave the tests up to the developer, you’re going to get assumptions (such as with the WTC course MONEY_MACHINE exercise), where some people wrote something that listed all possible combinations of change for a given amount and others wrote something that produced the combination with the minimum amount of coins and notes for a given amount.

      Question 2

      What are the limitations of TDD and Unit testing ? Some evangelists seem to think they’re the answer to everything, but once the functional test are complete, I still need to know the real time impact that a given change will have on my system. Yes, my change to the select statement produces the correct results, but does it do it any more efficiently ? Can TDD provide a way to test performance metrics ?

      Martin E

      (0) 
      1. Jacques Nomssi

        Hello Martin,

        1)

        let us clarify first: TDD is the always write a failing test first  approach. Unit testing is test in isolation. Tests validate a specification at different levels.

        • User Acceptance Tests (UAT) – we are done (from the user perspective) when this test passes
        • Integration tests – we work correctly with other parts of the system the developer cannot change
        • Unit Tests – Test the parts we are implementing in isolation.

        The developer does Unit Tests. All assumptions and design decisions flowing in the code are made by the developer as you correctly pointed out in the money machine case. Without test, they are not documented in a reusable way.

        If the developer has to coordinate with anyone to test, then it is not unit testing, even if it might be helpful to write acceptance tests in ABAP Unit for test automation.

        So ABAP Unit is not just unit testing.

        Other test forms will still exist even with 100% unit test coverage.

        2)

        The true purpose of testing is to reduce costs.

        POODR by Sandi Metz: Chapter 9 – Designing Cost-Effective Tests

        If writing, maintaining and running tests consumes more time than would otherwise to fix bugs, write documentation and design application tests, then tests are clearly not worth writing.

        http://www.poodr.com/ Practical Object-Oriented Design In Ruby, An Agile Primer

         

        The limitation are subjective. Do I value the code enough? If yes, I decide to invest time securing the code with tests. An experienced programmer can decide tests are not worth it when prototyping without a first acceptance test. As soon as you see the code as an asset, you secure it with an insurance policy.

        From POODR: it is common for programmers new to testing to find themselves in the unhappy state where the tests they write do cost more than the value those tests provide. The generic solution is to get better at it.

        4 years ago we were discussing classical and OO programming in ABAP. What if you were asked about the limit of OO today? I would repeat Sandi Metz: to benefit from OO, you need to get better at

        • OO Design
        • Refactoring
        • Testing.

        JNN

        (0) 
        1. Martin English

          Thanks, Jacques.

          I think my original questions had some terminology issues 🙂

          Even so, I’m not clear in how you (or if !!) you answered my questions

          For point 1, I was asking whether the Unit Tests should be purely bey the developer. In the case of the MONEY_MACHINE example, it was possible (because of unclear specifications) to produce working code that provided the wrong result. In the real world, I would like to think the developer would seek clarity about the specification.

          In other words, would you expect to find these errors in assumption at integration testing time or at Unit Test time ?

          I think the way I’m looking at it is that the tests should be written against the specification. If so, then its up to the developer to seek enough clarity about the specification to be able to implement the test classes in the first place – for example, if the MONEY_MACHINE example was in dollars and cents, then the developer would need to seek clarity about whether it was Australian Dollars or US dollars (different coinage).

          For point 2, I could read from your answer that a realistic answer from a developer, given the example I gave), is

          “performance is not my problem”

          The developer doesn’t have control over the environment the code runs in, so performance in the specific production environment is someone else’s problem. After all the code passes the unit tests because it provides the correct answer.

          For that specific example, you can do volume testing, but you seem to be implying that this is for a different level of testing compared to Unit testing. is that correct ?

          Martin E

          (0) 
  2. Uwe Fetzer

    Hi Jelena,

    really nice summary of the current discussion about TDD. We can do it even shorter: “It depends…” 😉

    You may know I’m working now for a utilities company. Obviously we have two main framworks build in the last couple of years  upon te SAP IS-U addon: a pricing framework and a printing framework.

    Beside the fact that no developer knows how these frameworks work in deltail anymore, noone want to change these frameworks because you may break something without noticing it.

    What would I give if the developers would have used unit tests in these frameworks…

    Cheers, Uwe

    (4) 
    1. Jelena Perfiljeva Post author

      Thanks for the comment, Uwe!

      I can certainly relate to inheriting some awful or just convoluted and poorly (or not at all) documented code. In my experience though, this could’ve been remedied rather by just writing decent self-documenting code, leaving meaningful comments, etc. Looking back at some programs I had to deal with I’m having hard time finding an occasion where I’d think “gee whiz, if only this person added Unit Tests”. Usually the problems there go much deeper.

      One might suggest, of course, that if the original developer used TDD to begin with then we wouldn’t even have such problems. But, honestly, I suspect I’d just be dealing with the awful tests on top of the awful code…

      (0) 
  3. Jakob Marius Kjær

    Hi Jelena,

     

    I really like your critical  approach and blog post.  A few thoughts from my side. I don’t see TDD as only being for fleshing out bugs and for the developer. But also for the business/product owner/stake holder (call them what you want 🙂 to actually have a good think about their requirements.

    I would argue it isn’t for the developer to write the test case, but merely to execute it. This way I so believe we will come closer to the shared understanding and actually developing what was thought of originally and also to better classify a development as done. However I do agree with you that we shouldn’t write test cases to the n’th degree. It needs to be pragmatic and more focused on the outcome of the requirement that for you as a developer to write ten test cases for every single development you undertake.

    (4) 
  4. nabheet madan

    Hello Jelena

    One thing on which i agree with you 100% creating a TDD depends on situation to situation. For example for a complex report with many if’s/elses/calculations it mught make more sense then doing it for all reports.

    Your blogs actually helped me looking at it from both perspectives. Thanks keep blogging keep sharing

    Nabheet

     

    (1) 
  5. Paul Hardy

    Dear Jelena

    I just posted a blog about TDD with a slightly different point of view to yours….

    You are 100% right about the internet, where you can call up articles to back up whatever viewpoint you have on any subject. As an example, if I wanted to argue the world will end this year, there are ALWAYS articles on the internet to back me up.

    In regard to the following:-

    – nothing is sealed
    – everything is modular
    – everything is injectable or mockable
    – that your organization places a high enough value on low-defects to justify the resource sink
    – that there is something of benefit to test at a unit test level

    Those all seem wonderful to me. The above is my aim in life this year. The organisation has placed a high enough value to fly me from Australia to Germany for a year, either because I am such a wonderful snake oil salesman I can convince anyone of anything, or that they believed me when I talked about SOLID principles and unit testing and such forth.

    They are even doing an exercise on design thinking at work in the near future … who would have thought! We are only a few miles from Walldorf so some SAP thinking must rub off.

    I think you and me posting opposite points of view on the same content each week is a wonderful thing for the SCN as a whole, as it gets people talking / thinking / arguing and getting passionate about things.

    Cheersy Cheers

    Paul

    (9) 
    1. Jelena Perfiljeva Post author

      Thanks for the comment, Paul! Link to your blog (one must never pass an opportunity to pimp own content on SCN).

      We’ll see what the comedy duo will bring us in week 3 and whether I’ll have to file an appeal or open a new case against some other concept that everyone but ABAPers is doing already. 🙂 (The mere mentioning of pair programming is giving me the hibbie jibbies.)

      Either way I agree – it’s certainly great to have some discussions going on SCN and breath  life into our still not quite yet revitalized Community. Thanks for your contribution!

      (0) 
  6. Mike Pokraka

    Hi Jelena,

    I enjoy reading your posts and blogs, and the majority of the time I agree with you. This time however is an exception.

    I don’t think anyone sensible has ever suggested TDD is always appropriate. It isn’t.

    It’s a valuable tool in modern software development. By modern I don’t mean just the technique, but also the actual product we are developing. Applications and code was mostly linear once upon a time. In this type of app, TDD does not bring that much benefit, and even unit testing is of limited value.

    Nowadays UX is king and we develop lots of components interacting with each other on the basis of uncertainty of where the application is coming from or going next.

    SAP has been around, and a lot of the staple functionality is still linear. ALV reports, Interfaces (IDOC / PI etc.), batch processing. Old but faithful tech that serves a valuable purpose to any business. Someone who spends a lot of time in these areas will not be likely to see many benefits.

    A custom application with over a million lines of code and thousands of classes is a different beast.

    Another angle to consider is not just the learning curve, but the experience / insight curve which is even longer. My personal experience with unit testing was a long love/hate relationship and it took months to really start to see the benefits, some insights even came years later.

    (7) 
    1. Jelena Perfiljeva Post author

      I don’t think anyone sensible has ever suggested TDD is always appropriate. It isn’t.

      Not sure if “sensible” is the keyword here but indeed very many(if not most) software celebrities claim exactly that TDD is never NOT appropriate (which logically can only be translated into “always appropriate”). In this blog, I mention a StackExchange question as an example. The original question is literally “when is it appropriate not to use TDD” and the top answers from Uncle Bob is “How could anyone NOT want that suite of tests?” Is this it ? Is that the only argument? The person asked a very valid (IMHO) question about practical application and you offer them only the same propaganda they must have heard dozen times already. Hm…

      Based on your comment, I don’t feel you even disagree with me. 🙂 And even if you were it’s perfectly fine. It’s not my goal to have everyone agree with my point of view. I’m not an “anti-TDD zealot”. 🙂

      As you mentioned, there are [many] occasions in ABAP when TDD/unit test simply is not practical. But as you correctly pointed out, there is definitely a shift in how we write (or at least how we need to write) our SAP programs going forward. For example, even though we don’t have Eclipse or even 7.4, much less HANA, I’m already considering better separation of presentation and business logic. Sure, we should’ve been doing this better all along, yada-yada-yada. Well, it didn’t happen and I’m not going to spend time shaming the previous developers. We can’t change the past, we can only improve from now on.

      If you used TDD and “saw the light” 🙂 – why not write about it? And with some specifics, please. That’s what’s still largely missing IMHO.

      Thank you!

      (0) 
      1. Mike Pokraka

        To nitpick Uncle Bob’s quote: there’s a difference between wanting something and it being feasible or appropriate. There are a lot of “How could I not want”‘s in life that are just not practical.

        Many of those who argue against it are also those who haven’t really used it.

        The chap asking the question has tests for 1% of the code. Of course this will never bring you any significant benefit. It’s a bit like saying I stuck £100 into the stock market a year ago but I only made a tenner so I’m giving up on that idea.

        Re. Writing about it, firstly my comment was specifically about unit testing. But that itself is a precursor to TDD, so it really does take some time to get into the mindset.

        I have a few blogs in my head, but there has been a big shortage of round tuits for quite a while.

        (2) 
  7. Jelena Perfiljeva Post author

    There was a tweet sent out that my blog says “TDD is like fasten the seatbelt”. To be totally clear: it’s not at all what I was saying. If TDD was indeed as simple as one-click seatbelt application and legally mandated on top of that then we wouldn’t be having any discussion about it.

    In this blog, TDD is rather the double boiler: a good tool but with a more convenient (albeit not as great) microwave alternative.

    Well, I guess whatever it takes to bring people to SCN to read the blogs these days. 🙂

    (1) 
  8. Matthew Billingham

    Like any bet, sometimes you win, sometimes you lose. […]”

    That made me laugh. If you ever want to confuse a mathematician schooled in probability or statistics, tell them that either something happens or it doesn’t. Therefore the chances are fifty fifty. What’s the probability the world will end tomorrow? Well, either it does or it doesn’t, so 50/50.

    No programmming paradigm will address all evils. In my career* as a developer,  I’ve heard many nay-sayers contend against them all, These people tend to have common denominator – they don’t suggest an alternative that’s better.

    There is no medicinal compound to which’s creator we may drink a drink a drink. There is no universal panacea.

    There are tools and techniques that can make your code, your programming practice  better. TDD is one of them, and I’ve found it useful (since and during doing the course).

    I’m interested in producing efficient, maintainable code. Anything that helps me do that is a good thing. TDD has had immediate benefits for me. Don’t care about the philosophy.

    (0) 

Leave a Reply