Skip to Content
Author's profile photo Matthias Steiner

The Way of the Developer

/wp-content/uploads/2012/11/8193128076_3384a52a6b_z_157143.jpg

If you’re working in IT you probably have heard this urban legend of Rapid Application Development frameworks being the silver bullet for delivering great software at unprecedented speed.

It’s understandable that the business would love to simplify and accelerate the development of software up to the point where trained monkeys are able to do the job; yet software development is  a craft to be mastered and hence it remains questionable if ever quality software can be developed by simply “clicking it all together.

Sure, it may be desirable to reduce the complexity of software development and hereby reducing the time to market. Yet, betting on shiny tools sounds like betting on a dead horse to me. We may be better off acknowledging that it takes skills and practice to develop quality applications and start treating developers with more respect. In times where Software is Eating  the WorldDevelopers are the new Kingmakers – and business better let go of the dream of inexperienced developers being able to deliver the next “killer app”, just because of a oh-so-shiny tool or framework.

With this blog post I want to shed some light on the mindset of developers and provide some arguments why I believe that it will always require skilled coders to build great (enterprise) applications.  The way I see it there are  three commonly seen problems with Model-Driven Architecture (MDA) and Rapid Application Development (RAD) tools:

  • Hiding complexity
  • Runtime dependencies
  • Lack of extensibility

Let’s have a closer look on all three.

Hiding Complexity

Simply put: no magic tool can reduce complexity. They may be able to hide complexity by applying defaults to support the most-common straightforward scenarios and use cases in a given domain. If model-driven tools would support the whole set of functionality provided by the underlying programming model or API they would be equally complex.

So, what happens if you need to do something outside of the supported standard scenario? For many hyped tools the answer is: you  will waste the productivity gained using the framework trying to somehow make the damn thing tool do that one thing it cannot do out-of-the-box.

A related problem I see is that many developers only familiarize with the tool and never bother to look behind the scenes to truly understand what exactly it is doing. Those developers will never be in control, but  depend on the tool. Hence, there will be zero room for optimization. And be warned, sooner or later in your project leaky abstractions will surface and your project will be doomed if no one actually understands how things truly work under the hood.

Consequently, most  developers I know would always favor a well-documented API shipping with source-code over a shiny model-driven tool, because it allows us to look at the code and understand what it does. There’s nothing wrong in using libraries and frameworks (the opposite!) as long as we can have a look at what they do and how they work. This way, we actually learn and master all these layers of abstractions we use and build on top of!

Runtime Dependencies

The worst breed of development tools comes with runtime dependencies. Why? Because they are the prime example of proprietary technologies resulting in vendor lock-in! A Rapid Application Development tool that actually accelerates the process of coding during the design-time is a great thing – but only if it is just generating the code people would have typed in manually. No more, and no less!

And let’s be clear, “writing the code” is hardly the limiting factor in developer productivity! Either we know the syntax inside out or we got code-completion to help us out. “Writing code” is the least amount of the job… in fact, we spend much more time thinking about how-to best implement the requirements and how-to best integrate our code into the overall project (spells: Software Architecture).

Another kind of my ‘favorite‘ tools are those that actually generate coding! Seriously, let me say this loud and clear: blowing up the code base with redundant coding blocks is the least thing we want (unless you pay us by lines of codes!) Such coding blocks are much better refactored into a template, which one calls passing the unique parameters. (Have a look at Spring templates if you want to see a great example of this!)

Lack of Extensibility

Last, but not least any tool or framework that wants to win developers hearts better be extensible or provide means to write plugins. If a tool restricts the feature set and hereby limiting our choices, it’s heading into a dead-end street. Allow us to integrate our own favorite tools, plugins and frameworks and we will fall in love with the framework. Even better, we even help improving it by adding new capabilities. Hey, we may even start talking about it and spreading the word – increasing its user base!

There’s Hope

Now, don’t get me wrong here. I’m not saying that MDA or RAD tools are bad and useless per se. If they avoid the pitfalls outlined above they can be of great use! There are actually tools out there that do get it right, yet they may look different from what one would expect. And that’s because they target developers and not chasing the dream of having business-oriented people clicking together enterprise software.

One of my favorite RAD tools out there is Spring Roo: it’s console-based with first class code completion and hinting, it generates standard Java code and all runtime features can be simply removed if one should decide to continue the trail without Roo. It is extensible so the developer community can integrate new features.

Speaking of which… we actually just did that! Thanks to SAP Research we got Spring Roo Add-Ons for SAP NetWeaver Cloud, which support creating full-fledged web applications incl. persistence, exposing RESTful interfaces and MVC within mere minutes. And just a few days ago they released another Add-On allowing developers to add backend connectivity via Gateway with just a few console commands.

Now, that’s the kind of RAD tools developers love to use!

Synopsis

I fully agree that the “developer experience is broken”, yet improving the tooling is only one side of the equation. Improving the processes is equally (if not more) important. Let’s remove the barriers between business, IT and development (think of DevOps, Teams of Ten, etc.) Let’s embrace agile methodologies instead of making developers suffer from endless change requests following the waterfall approach.

As technology is accelerating it is more important than ever to provide developers with a bit of room to explore the new possibilities in order to keep our skillset up-to-date. Give us good training material and well-documented APIs and frameworks to use. Grant us the required time to up-skill the team – prior to development! And provide us easy access to  software/APIs to freely test-drive the technologies to familiarize with them.

When it comes to tooling there are more important things than nice looking modeling frameworks. Give us platforms that support hot-code deployments to reduce turn around times (aka “batch processes“). Give us better debugging and analysis tools. And give us tools that make it simpler and more efficient to develop in parallel across distributed teams.

If we talk about accelerating the process of developing and shipping quality (enterprise) applications (and not flashy PoCs, demos or prototypes) then we may be better off questioning the status quo of today’s common practices and not just tools. Stop rushing products out the door. Stop turning proof-of-concepts into products. But most and foremost: accept the fact that software development is a complex business and that it needs skills to do it!


Assigned Tags

      26 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Jason Scott
      Jason Scott

      Great blog Matthias. Now... How do we get the management of all sap clients to read this.... 😉

      Author's profile photo Matthias Steiner
      Matthias Steiner
      Blog Post Author

      🙂 Good question! Guess our best bet is word of mouth and developers promoting it...

      Author's profile photo Fred Verheul
      Fred Verheul

      Agree with the previous speaker: great blog post! How you managed to get this one out after a full week of TechEd is beyond me, but I'm glad you did.

      I fully agree that building quality software is often not possible with only RAD tools. Still this Spring Roo stuff sounds interesting (though I'm not a fan of command line tools, since I find it hard to memorize all the possible commands etc), so I'll give it a try.

      Even better (now that I think of it): let's make this the theme of our next CodeJam Huizen (which will be announced shortly).

      Thanks again!

      Cheers, Fred

      Author's profile photo Matthias Steiner
      Matthias Steiner
      Blog Post Author

      Thanks Fred! Well, the answer is simple: I had time to kill at the airport - besides I have been very passionate about this topic all my life, so the words just come out while decompressing from TechEd.

      Regarding Roo... give it a try, as the tool provides "hinting" and syntax completing, which makes it very productive to use.

      The tutorial is here: http://sapnwcloudlabs.github.com/nwcloud-roo-addon/tutorial.html

      All up for another CodeJam in Huizen - had a blast at the first one! You organize it and we'll be there - deal? 😉

      Cheers,

      Matthias

      Author's profile photo Tom Van Doorslaer
      Tom Van Doorslaer

      Can't agree more.

      Calling development the ultimate batch job (quote from a certain keynote) is not quite what I was expecting. It implies that development can be automated, and per consequence, the developer is nothing more than a trained monkey.

      For years already, businesses have been looking to replace their software engineers by tools, which lead to the idea that "developers" (I prefer the term software engineers) are inferior to business analysts.

      so yes, PLEASE accept that making good software requires crafted software engineers, just like you need crafted carpenters to create beautiful furniture.

      If what you want is a shack, fine, just don't come complaining that the roof leaks.

      Author's profile photo Former Member
      Former Member

      Tom,

      if anybody has a deep appreciation for the art of software development, it's Vishal, So, I really do think you read something into the keynote that was never meant to be implied. "Development as the ultimate batch job" referred to compile-link-debug cycles which Vishal argued need to be eliminated, but that didn't mean that the art of software development is a batch job.

      Cheers,

      Michael

      Author's profile photo Tom Van Doorslaer
      Tom Van Doorslaer

      I Michael,

      I know that Vishal definitely did not mean it in that way, but the expression was kinda unfortunate. And I can guarantee that some business leaders will interpret this the wrong way. (again)

      And yes, it's cool for us as software engineers that we have on the fly compiling and Unit-Tests running as we type, but it's just another stick for some to claim that "development" is easy, and that we can cut development times again, because we don't have to wait for compilation. Wrong, because 80% of our time is spent thinking about the solution and figuring out which data-model and architecture suits best.

      I also don't want to lash out to Vishal (far from), but I want to take a stand for all "developers". There is a reason why I prefer the term "software engineer". Engineers are regarded as very bright and intelligent people. They are regarded as the artists of the technical world. A crafted "developer" is exactly that: an artist in software engineering.

      And that's the message on which we should focus. Nurture your software engineers, they are rare talents. Tools are great, and I am grateful for having them, but they can't replace us.

      And that's what Matthias explains in such clear wordings. And I can't agree more.

      That's what we must focus on.

      Author's profile photo Matthias Steiner
      Matthias Steiner
      Blog Post Author

      100%?Agree. Tried to make that clear by referring to his quote when talking about faster turn aroud times via hot-code deployment. Vishal is very serious about developer empowerment - after all - it was him who iniated the change irt free dev licenses, easy access etc...

      Author's profile photo Harald Mueller
      Harald Mueller

      Great blog. Reading good code is like a reading a good book. It has a reasonable flow, can be inspiring, comprehensive and incredibly compact. Tools might help in getting started quickly but effective re-use needs good code, imagination and regular refactoring. This usually get's lost with most of the visual RAD or modeling tools. Developer productivity and fast turnaround is not just a question of tool support but depends massively on modularity, good interfaces and the overall design of the complete system which requires software craftsmanship.

      Author's profile photo Matthias Steiner
      Matthias Steiner
      Blog Post Author

      Thanks Harald! It's very comforting to know we have executives at SAP that acknowledge the craft of software engineering and support their teams in pushing for excellence!

      Author's profile photo Kumud Singh
      Kumud Singh

      Hi Matthias,

      Looks like you wrote this blog when someone demeaned the role of a software developer 😉 . On the other side, if we see there are many things which are getting turned into black box tools and are happily accepted as well. Certainly such tools have limitations in terms of handling complexity and other factors you have highlighted in your blog. And may be more limitations can be found once someone gets into the black box to understand what's inside. This is a great topic for an intense discussion, methinks!

      Regards,

      Kumud

      Author's profile photo Matthias Steiner
      Matthias Steiner
      Blog Post Author

      Hi Kumud,

      well, kind of. It wasn't like someone demeaned the role of software developers directly, but more of a recurring notion that Model Driven Tools will simplify the process and accelerate innovation cycles. I keep hearing this 'urban legend' over and over again and it's so far from the truth...

      While I would agree that in many cases simplifying the design of products is desirable and I don't mind getting a black box, I feel strongly that this approach is a very bad choice in regards to software engineering tools.

      If you like we can continue the discussion very soon in Bangalore 😉

      Cheers,

      Matthias

      Author's profile photo Vignesh R S
      Vignesh R S

      Nice blog Matthias..Liked it!

      Author's profile photo Matthias Steiner
      Matthias Steiner
      Blog Post Author

      Thanks Vignesh! 🙂

      Author's profile photo Thorsten Hopf
      Thorsten Hopf

      Hi Matthias,

      thanks for "writing out loud" what I seem to have been chewing on in my mind for quite some time.

      I see (part of) the development process as the creation of a piece of art. I almost feel insulted by the assumption that any tool could do that.

      Of course there is a lot of room for tools to make the life of the developer easier, but as you said: you will never get a "business owner" to "draw" the solution. Why? Because describing a problem in a way that it is working under all circumstances is -> programming! To do this in a "model driven way" is even more complex than doing it in "code". At least in general, of course there are problems that can be solved like that. 🙂

      And in addition to that if you have got a black box it is a very hard task to describe the parameters of said box in a way that it can be used correctly without actually knowing the algorithms inside the box. At a certain point in time I have always come to the point where "looking into it" actually did the trick...

      Best regards,

      Thorsten

      Author's profile photo Matthias Steiner
      Matthias Steiner
      Blog Post Author

      Thanks Thorsten for chiming in!

      I remember many occasions where we discussed the "craft of software engineering" and clearly many thoughts from these discussions are reflected in this post... thanks for having been a constant discussion partner over all these years (and enduring my venting once in a while!) 😉

      Author's profile photo Matthias Steiner
      Matthias Steiner
      Blog Post Author

      Just stumbled across this video from SAP TechEd - Interview feat. Craig Cmehil and James Governor "The Future Enterprise Developer"

      If you liked the blog you may want to check it out: http://bit.ly/futureenswdev

      Plus: There's also this great post from Thorsten Franz about the "The Most Balanced Piece I Could Write About the Stupidity of Ageism" - but you all read that one of course, right?

      Author's profile photo Mark Teichmann
      Mark Teichmann

      You are highlighting very important things here. By far the most prominent paragraph is the "Hiding complexity" one:

      It is somehow fast and easy to build simple applications using frameworks like Spring Roo without knowing much about the things happening in the background. But definitely you will come to a point where you need to know every little detail about the generated code and the dependencies between the single objects.

      Therefore you often get a false impression about how easy certain things are and how fast you can build something with that.

      Therefore the most significent message from this blog is for me:

      Make use of the powerful frameworks that you can find out there but always bear in mind that you understand what is happening behind the scenes. And take into account that you need 10 times more time to learn the backgrounds than it takes you to actually use the frameworks!

      Author's profile photo Matthias Steiner
      Matthias Steiner
      Blog Post Author

      Hi Mark,

      totally agree! What I wanted to convey is that tools like Roo are great in this regard as they speed up the process without hiding the complexity! So you can really get things down very fast and if you have to tweak it you have all the options as it's just standard coding. For sure, you have to understand the involved technologies and not rely on the tool used... that was one of my key messages! 🙂

      Cheers,

      Matthias

      Author's profile photo DJ Adams
      DJ Adams

      Superb post, Matthias. Balanced, well-argued and to the point. I'm in total agreement with the things you put forward here. You don't mention it explicitly, but my experiences at previous clients include horrendous quality issues in business-as-usual, and project overruns and re-work due to inexperienced developers, not as a result of using a framework or a shiny tool, but because of the mirage of outsourcing, which is made even worse, not better, by the almost infinite mass of process and procedures put in place to try to prevent quality and delivery issues.

      Here's a thought - why not be radical and employ craftspeople who know what they're doing?

      I know it's a bit of a rant, but I ended it deliberately focused the key word: craft. Software *can* be cranked out by machines, processes, and armies of people, but it's not software I could ever recommend for productive use.

      dj

      Author's profile photo Matthias Steiner
      Matthias Steiner
      Blog Post Author

      Thanks DJ for bringing the discussion to this very issue. Couldn't agree more... you need great engineers to develop something truly great. You may be able to deliver something on time and meeting the customer expectations. But only in the long run, you will be able to tell if the software developed is flexible enough to adjust as requirements change without having to rewrite the whole thing over and over again. And THAT takes serious skills...

      Author's profile photo Krasimir Semerdzhiev
      Krasimir Semerdzhiev

      Awesome blog Matthias. I think it hits exactly the target of where efficiency improvements to be addressed. Real value/speed will be boosted by enabling developers to spend more time on the actual code concept and algorithms instead of doing mechanical work or getting distacted by a serie of small (and time consuming) repetitive steps.

      Apart from being able to progress faster - by eliminating the regular repetitive tasks will eliminate (or greatly minimize) the interruption effect, which on its own leads to massive loss of productivity.

      I'm really happy to see that we're making great progress in addressing those aspects with SAP NetWeave Cloud, which will hopefully make it a more attractive environment for developers.

      Author's profile photo Former Member
      Former Member

      Great blog, very interesting subject, thanks for highlighting it.


      Like any software developers, I suppose that the people who develop RAD tools
      struggle to find a balance between simplicity and flexibility (I think one
      underappreciated aspect of the art of software development is in recognising
      the point at which allowing for a higher the degree of flexibility, whether it
      be in catering for exceptions, anticipating changes or allowing for differing
      requirements, crosses a threshold into the land of diminishing returns).

      Is there a danger that the desire of companies to seek short cuts with regard to development, by cutting costs and timelines, will drive out the the craftspeople DJ refers to above? In the last 3 years I've been involved in at least 4 companies where experienced, knowledgeable developers have been replaced by a combination of shiny code generators and oursouce models. In one case, the mistake was rectified within six months, but I imagine that is the exception. One problem I reckon is that there can be a fairly significant lag between the move into RAD / outsourcing beforee the ill-affects are felt, and in most cases what happens is a fairly gradual degradation of the system which might concern those at the coalface, but which doesn't always filter up the chain of command.

      Could not agree more with your final paragraph. I plan to quote it every time I am
      reminded that budget constraints or tight deadlines as a limitation on the time
      that can be spent to put in place a robust and flexible design which will be cheaper in the long run. To paraphrase from a very different world, “I wrote a long program because I didn’t have the time to make it shorter”.

      Author's profile photo DJ Adams
      DJ Adams

      Those are great comments you make, Adam, and they resonate deeply with me.

      There is indeed a big problem with "damage lag" - caused partly by the message from the coalface (as you nicely put it) being either ignored or treated as 'currently insignificant' by the message recipients, and partly because those same recipients have a core objective of championing the outsourcing. So they're somewhat disinclined to take on board and share the reality.

      It's a long slow process in which the quality of the code gradually diminishes. Early signs are change requests that take longer and longer, not because the change is complex, but because the code that needs changing has been so poorly modified in a previous change request that it's now much harder to make the next change. And so the disease sets in.

      I've been party to conversations that have gone something like this:

      Them: "Well, this change is going to take two days to make, as the code it affects is very complicated and convoluted".

      Me: "Well guess why? Because you *made* it convoluted the *last time* you modified it!"

      I had a home made poster on my desk that showed a picture of a grotty, moldy old peach, with the subtitle "Stop the Software Rot". It was very apt.

      dj

      Author's profile photo Former Member
      Former Member

      Great article Matt. I have always believed that good software cannot be built by modeling and generation tools as they are built with certain assumptions and certain scenarios in mind. And these assumptions will eventually be invalid.

      A big issue which you probably did not want to cover here is , when there are easy modeling tools new developers start building software without understanding the nuances. When the complexity increases , they do not possess the necessary knowledge to tackle the challenge and create poor applications. This habit sticks more or less to the rest of their lives.

      SAP has been a big proponent of modeling based tools (especially in the Java space) and it has hurt developers and SAP alike. I still feel that SAP believes more in building simplified modeling interface (as it can encourage a bigger ecosystem with lesser skills to jump in ). WDA / WDJ is a case in point. (WDA developers have no clue about HTML and JS , but build web applications). On a positive note , SAP UI 5 is simply great , has excellent API documentation and does not need modeling tools. 

      There have been some positive developments in the Neo space but the general direction is still the same.

      Tools should facilitate efficient discovery ,  reuse and maintenance. Simplifying a developer's task should not be its primary objective.. SAP still has a long way to go to incorporate your thoughts.

      A well balanced article , on the issue ,  is a nice start..

      Author's profile photo Matthias Steiner
      Matthias Steiner
      Blog Post Author

      Well said Bharathwaj!  I briefly touched upon the topic of developers focusing solely on modeling tools and the problems I see herein. As such I can only advice developers to thrive to dig deeper and understand the basics of all the abstraction layers added on top...

      Cheers,

      Matthias