Why we must take Functional Programming seriously.
Preamble
I was in the process of responding to Christian Drumm‘s excellent blog on Functional Programming in ABAP, when I realised that my mental ramblings should really be put into a blog (or perhaps more of a brain dump).
Please read Christian’s post for the context of the following comments…
I must say that over the last few years, I’ve somewhat lost touch with the latest language developments in ABAP, so its nice to see that in a 7.40 system, ABAP has these additional features and that they’re heading in the functional direction.
However, it does beg the obvious question though – why didn’t ABAP have these features 20 years ago? But anyway, they’re here now, and that’s a good thing.
Real Functional Programming in ABAP?
Well, yes but not entirely.
Until ABAP is able to treat a function as a 1st Class Citizen, then only partial progress can be made towards making ABAP a true functional programming language. For those who are not sure, a “1st Class Citizen” is any object that can be created and deleted dynamically, can be passed as a parameter to a function or subroutine, and can be passed as the return value from a function. An ABAP function module does not meet all of these criteria.
The underlying principle of functional programming is not simply being able to use the map or reduce functions, but rather in the fact that the software is designed using the systematic application of referential transparency.
Adding various operators like COND, WHEN and LET certainly allows solutions to be programmed in a more concise, functional style; but this only permits an occasional, ad hoc use of referential transparency, rather than its systematic application. Nonetheless, these additions to ABAP are a good step in the right direction and I’m sure people will find it very useful.
I’m sure the ABAP language developers are well aware of the need to create a unit of code that behaves as a 1st class citizen, and have already been thinking long and hard about whether and how such changes could be implemented.
Borrowing from Clojure?
Looking at the syntax and implementation concept behind the new ABAP operators like COND, WHEN and LET, they look very much like they’ve been lifted directly out of Clojure – which is certainly not a bad thing!
We just need to add operators like DEFN and FN and ABAP will have made the switch – but see the comments above about ABAP functions and 1st class citizens…
Functional vs Imperative Programming in General
I have been a long time advocate of the functional programming paradigm, and at the risk of sounding like a broken record, have been trying to convince people within not just SAP, but also the business programming world that this paradigm needs to be treated as a viable and beneficial development option.
However, the blunt fact of the matter is that there tends to be a large cultural divide between the functional and imperative programming camps.
I may use a little bit of exaggeration here, but this is only to highlight the polarities that exist between these two camps. Each camp tends to view the other with suspicion on the basis that “we have the correct priorities” and that “they are focused on all the wrong things”.
The business programming world (the imperative camp) is focused on writing software as a means to make money. Yes, yes, they want to solve customer problems, but in a world governed by a market driven economy, at the end of the day, the share price is king.
Over in the functional camp, this programming paradigm is presented as an implementation of Alonzo Church’s Lambda Calculus. Here, the Platonic concept of “pure intellectual thought” reaches one of its pinnacles. Pure functional programming languages such as Haskell have created rigorous implementations of this mathematical concept, and the result is a problem solving technique that, after you have first accepted the very restrictive modus operandi imposed by all functions needing to be pure (that is, they never have side effects), can produce solutions whose correctness can be determined with a very high degree of certainty.
Whilst there is far more to the situation than simply “making money” and “functional purity”, when you talk to people from these two camps about their differences, the conversation tends to come back to the importance of one or other of these priorities.
Why are the two camps so polarised?
Over in the Functional Camp
A large part of the polarisation here comes from the fact that many universities still teach functional programming as just an implementation of lambda calculus; therefore, if you’re not interested in mathematics, functional programming is of little use to you.
Having said that, universities are gradually seeing the need to change this approach. I’ve spoken with Prof Kevin Hammond of St Andrew’s University in Scotland about this, and he is actively working to join up the worlds of business programming and functional programming. However in general, functional programming is not really presented as a viable solution to the practical problems seen in the business world – such as creating warehouse stock management systems or running your accounting system. Instead, students of functional programming are often told to write their own language interpreter that outputs an abstract syntax tree, or to design an algorithm that decides if a graph tour can be performed in which each edge is visited only once.
That’s all well and good, but it doesn’t prepare students for work in the business software world. Consequently many students are implicitly taught to exclude this development paradigm as a viable option in the hard-nosed world of business (where success is measured solely in financial terms (Is this a good thing? Discuss))
For those software developers who do enjoy the supposed purity of mathematical thought, there is often the sense that being able to write in the functional programming style makes one intellectually superior to the lowly imperative programmers.
Well, there’s a small element of truth to this argument, but a larger element of blindness caused by intellectual snobbery. This blindness then causes otherwise intelligent people to suggest that all business software is written by money-grubbing companies interested only in meeting their greedy revenue targets. (Yes, one functional programmer I spoke to a while back implied that SAP’s market dominance meant I worked for “The Great Satan”)
Meanwhile, back in Gotham City… uh sorry, The Imperative Camp
From the imperative side of things, this programming paradigm is easier to pick up than functional programming in the same way that the mechanical engineering solution to a problem is “visible” (you can see wheels and gears turning, drive shafts spinning and valves operating) and the electrical engineering solution to the same problem is “invisible” (you can see the resistors, capacitors and diodes on a circuit board, but you can’t see the current flowing through those components).
There also tends to be the overriding attitude that “time is money”; therefore, software must be pushed out the door as fast as possible to avoid loosing (the perception of) market share. It is certainly true that from time to time, software companies push software out the door way too early (this is the “if it compiles, ship it” approach), but this is generally avoided.
Similarly, the business programming world must avoid the smugness of thinking that they deal with the “real” world, and dismiss the functional crowd as just a bunch of intellectual snobs who live an abstract world completely divorced from, and irrelevant to anything down here on terra firma.
Again, there’s a small element of truth here, but a larger element of blindness caused by thinking that success is measured solely in financial terms.
Can the two camps co-exist?
Yes, certainly. And I believe the first step that must be taken is to remove the sectarian boundaries that been erected to protect the supposed rightness of each camp’s particular point of view.
Both sides can benefit from each other by exchanging techniques that they have found to be successful.
On the imperative side, the software needs to meet the customer’s needs at a price the market is willing to pay. When a company does this well, that company makes money. But making money is by no means the sole measure of success.
And on the functional side, the use of functional programming techniques often allows for a more concise definition of the solution. Also the correctness of that solution can typically be determined with a higher degree of certainty than is possible with the imperative approach; however, the rigour with which the policy of functional purity is applied should not override the other factors inherent to running a business.
The division between the camps is certainly less than it was 10 years ago, but it still tends to be true (at the macroscopic level at least) that when these two camps come into contact, they generally regard each other with suspicion, throw a few shallow, poorly considered insults over the fence, then as they leave, they congratulate themselves for having the “correct” approach to software development and get back to business as usual.
However, I’m happy to say that developers on the imperative side of things are starting to look more closely at the functional way of doing things, and that the universities are trying to tailor their functional programming courses to include business software as a target problem domain.
Unfortunately, there are some other factors that reinforce this cultural divide.
Functional Programming at SAP
SAP is already a highly successful software company, and none of that success is attributable to coding written using a functional programming language, or even the functional programming paradigm. So this leads to two further problems; and I must stress that these problem are by no means unique to SAP, but apply generally to all fields where expertise has lead to success:
- The “inertia of expertise” problem. This is a well known phenomenon in which genuine expertise in a subject has on the one hand, produced genuine break throughs and resulted in the creation of valuable and highly successful solutions; yet on the other hand, that very success creates a large mental inertia that makes it very hard for experts to seriously consider the possibility that other, equally viable solutions may also exist.
- The “innovator’s dilemma“. This is also a well known problem that can be thought of as the flip side of the “inertia of expertise” problem. This is where it is very difficult (or even impossible) to innovate in an environment within which complex problems have already been solved, and solved effectively. Existing ABAP systems contain many highly successful solutions for solving business problems, so the overriding mindset is “if it ain’t broke, don’t fix it”. The resistance to the use of innovative languages is compounded by the fact that the solution created by the innovative approach will often not be any better than the existing solution. Thus the person trying to introduce a new way of solving an old problem is often viewed as someone who “doesn’t really get it” because “that problem has already been solved – move on!”. This makes indifference to innovation appear entirely plausible.
Having said that, the functional programming language R is used within the HANA system, but at the moment there are no tools by which a customer or partner could extend or develop their own R procedures.
I strongly believe that if the business software world (SAP) is to survive the next major phase shift in computing technology (Cloud computing), then at least two things need to change. By taking the perspective of an ABAP system, we could rather loosely describe these as “changes from within” and “changes from without”.
Changes from within (The Natural Growth Approach)
Adjust the ABAP programming language so that true functional programming can be performed. This will require ABAP to be able to handle functions as 1st class citizens; however, since I am not an ABAP language developer, I don’t know whether this proposal is even viable.
Assuming that such a change is viable, then the syntax that has already been used by the new COND, WHEN and LET operators could be extended to create a language that looks and behaves very much like Clojure. (Though there are other much deeper issues as stake here, particularly when handling concurrency).
However, this change (assuming its even possible) will be the slow option because it relies on the natural growth of ABAP software from within the developing system. In realistic terms, natural growth like this takes 3 to 5 years to become an integral part of the system.
Changes from without (The Innovative Approach)
Due to the “Innovator’s Dilemma”, the adoption of functional programming as a way of life cannot take hold in any area where mature solutions already exist. Therefore, new programming paradigms must occupy new problem domains. This avoids the plausible accusation of trying to re-invent the wheel.
I believe the area where this approach has the most potential is in the Internet of Things. This is a wide open field in which mature solutions do not yet exist. Therefore, the possibilities exist to create cloud-based software based on the functional programming paradigm that can handle the huge volume of data and connections demanded by a connected world.
I further believe this paradigm shift is absolutely necessary for the following reason:
To survive the IoT, we must be highly scalable and be able to support massive levels of concurrency
Sorry to burst anyone’s bubble here, but if you think that concurrency and scalability are topics that can be handled after you’ve got your business app up and running, then you’ll be in for a very nasty surprise come go live…
Scalability can generally be solved by throwing more hardware at the problem (up to certain limits of course), but concurrency is a much thornier problem. Concurrency poses a different type of challenge because it requires us to fundamentally change the way we write software. But there is a natural human resistance to change – especially if we have existing solutions that have worked well in the past. Sorry, but yesterday’s imperative solutions will not solve tomorrow’s concurrency problems.
Concurrency is of fundamental importance to any business scenario that expects to perform real-time communication with the number client devices expected with a realistic IoT solution. These devices could either be mobile smart phones or static devices such as pieces of machinery on a production line or low energy Bluetooth beacons.
The exponential growth in computing device numbers means that any business involved in the IoT will, in the reasonably near future, have to handle in excess of a million concurrent connections as its normal runtime state, rather than as a peak-load exception. From a database perspective, a HANA system can handle the number of queries generated by such levels of concurrency, but how about all the intermediate systems that connect the outside world with your backend HANA system?
A totally new approach is needed for the communication interface between the SAP world and the gazillions of devices all screaming for a timely response from the backend business system. Concurrency is an area where functional programming can provide a solution with far less development effort than would be required if an imperative language were used.
Therefore, it is of the greatest importance to choose both a programming language and paradigm that will contribute towards the solution and not become part of the problem. Such a language must not contain any unnecessary complexity, for as Tony Hoare has said:
“If our basic tool, the language in which we design
and code our programs, is also complicated,
the language itself becomes part of the problem
rather than part of its solution.”
Ignore this advice at your peril…
Therefore when writing a communication interface for the IoT, it makes perfect sense(to me at least) to choose a language in which concurrency is a natural feature, rather than an additional construct that must be implemented in that language. Here, the obvious language choices would be either Erlang or Clojure – both of which are functional programming languages and are both very good at concurrency.
For the specific task of an IoT communication hub, my first choice would be Erlang due to the fact that this language is already battle-hardened due to having been used in the Telecoms industry for over two decades. Among other things, Erlang was specifically designed to handle the massive levels of concurrency experienced by a telecoms switch, so this makes it an ideal candidate.
If you’re still sceptical about the commercial viability of a language like Er…whats-it-called (Erlang), then consider that WhatsApp is written entirely in Erlang and they regularly handle more than 200,000 messages per second. See here for reviews and technical details.
Also Massively Multiplayer Online games such as the different Call of Duty variants all use Erlang servers to handle the client-to-client transfer of game state information.
Whilst the idea of using a niche language like Erlang for handling massive levels of communication concurrency might scare you, it really is the best choice for this particular problem domain.
However, if you want a more general purpose language that runs on the more familiar environment of a Java Virtual Machine, then Clojure would be a more comfortable choice. This language is also very good at handling concurrency and since it compiles down to Java byte code, can easily be understood by experienced Java developers.
The choice to use either of the languages mentioned here (or any functional programming language for that matter – Haskell, Scala, OCaml etc) requires a willingness to move outside our comfort zone. Generally speaking, this willingness is not always forthcoming until we have first experienced the failure that comes from thinking old solutions will solve new problems (I.E. our persistent attempts to rationalise not having to move outside our comfort zone).
The bottom line here is that the imperative programming paradigm cannot solve the concurrency issues raised by having to support massive levels of simultaneous connections. If you don’t believe me, go away and write a solution in <pick_your_favourite_imperative_programming_language> and see how much scalability you can practically achieve, and how robust that solution actually is. After this painful experience of blood, sweat and tears, I venture to say you will be more willing to consider a solution based on the functional programming paradigm. 😆
So what? What does this have to do with SAP software?
With the whole computing industry moving towards Cloud-based solutions, providing massively scalable, centralised communication access to your backend system/s is fast becoming a make-or-break topic.
The potential exists to use something like a Cloud Foundry Custom Buildpack to implement an IoT communication solution based on any programming language you like. This in turn could be used to communicate with a HANA system…
The field is wide open here. Let’s seize this opportunity to create new solutions for the new environment, and not make the mistake of thinking that our comfortable, old solutions will be suitable – they won’t.
Chris W
If you aspire to functional programming in SAP, it may be better to muse upon SAPUI5 and JavaScript rather than ABAP. Per Douglas Crockford, JavaScript already treats functions as first-class objects. In HANA, you're able to use server-side JavaScript.
As resilient and robust ABAP can and will be over the long term, the writing on the wall is that at some point in the future really all development will take place in the suite of new technologies HANA introduces and pulls organizations/developers towards rather than ABAP. It could be as few as 10 years, but may be as long as 100. So when we're talking about something abstract like pure functional programming in ABAP, I consider that more of a hobby moreso than something to spend a lot of time aspiring towards -- especially when there are many more concrete business cases that we know can be fulfilled through ABAP OO design patterns.
Bottom line: if you really like to do functional programming, JavaScript is the better language to work from than ABAP. If you're really looking to innovate rather than provide standard business functionality, chances are good you're starting to drift towards HANA and SAPUI5 anyway.
With regards to other innovations in the area of Internet of Things, I do have a shameless plug for my latest contribution to the HANA Idea Incubator here. That would manifest a lot of the passionate ideas you have about functional programming and IoT.
Hi Ryan
I'm not talking about using FP simply for its own sake, neither am I talking about client-side programming. Having said that, I'm also writing SAPUI5 training material at the moment that contains sections on using JavaScript in a functional manner. But we have to be careful with JavaScript because it is a mixed paradigm language, and the vast bulk of JavaScript coding out there (including SAPUI5) has not been written with referential transparency in mind.
However, this is all beside the point. My point in this blog is to raise awareness that the IoT will bring with it concurrency challenges that imperative languages such as Java and ABAP will be hard pressed to meet. Therefore a paradigm shift is need in our thinking to accommodate these new demands.
I believe the FP paradigm presents us with the best set of tools for meeting this challenge.
I don't known whether ABAP will ever be able to allow true functional programming; and if it does, we then have the extra burden of own legacy code to deal with...
Regards
Chris W
Not sure if I'm getting through to you here : JavaScript isn't merely a client-side programming language. Node.JS and server-side JavaScript in HANA is evidence of that.
Certainly there are ways JS has been used that really doesn't fit the FP paradigm
I encourage you to check out some of Douglas Crockford's writings and presentations on functional programming and JavaScript before you presume too little of what it's really capable of, particularly with regards to functional programming.
This presentation he has on monads in JS is a good place to start:
Monads and Gonads - YouTube
I'm well aware of:
But that's part of my point. JavaScript is a multi-paradigm language which means you can write an app that contains a jumble of imperative and functional constructs. Unless a developer is both very disciplined and has a good understanding of how to apply referential transparency, then you'll end up with muddled code.
The point you're missing is that none of this JavaScript discussion is relevant for the problem at hand - which is handling concurrency.
JavaScript is a single threaded language that relies heavily on the presence of a global context. This makes it completely the wrong foundation upon which to build a massively concurrent software platform.
Languages like Clojure or Erlang are far better environments for this type of problem domain because they have been designed with this specific use case in mind.
As much as I like JavaScript, its the wrong tool for this particular job.
Hi Chris,
Thanks for the great blog.
Regarding the question why functional and imperative programming are divided up in two camps I want two add two more thoughts:
Christian
Hi Christian
For point 1, this is certainly true - its pretty much how I learned ABAP on R/2 systems!
Having said that, FP can be understood by anyone if it is taught in a way that makes it simple and approachable. Unfortunately, this reveals a huge problem in the way FP is currently taught. As I highlighted in the blog, universities tend to present the topic within an atmosphere of intellectual superiority, and this often alienates many, otherwise perfectly intelligent people from learning the subject.
ABAP programmers on the other hand, certainly will need to expand their horizons and allow their minds to take on new development paradigms. This is why I wrote a training series called "JavaScript for ABAP Developers". 🙂
As for point 2, yes, this is a very good point and it only adds weight to the argument that FP should now be taken seriously by the business programming world.
Regards
Chris W
Hi Chris,
I fall in this category. I have a degree in mechanical engineering, but i always had this fascination about CS.
It has been 6 years since i have been programming in ABAP-OO and i love it, more so the FP part. I am not aware of any of the afore-mentioned FP paradigms/languages. But i am still looking forward to more FP features in ABAP.
BR,
Suhas
PS - This is not a criticism of your comment, it's just my experience.
Sadly I've got to inform you that you are part of a minority (not because you have no CS background, but because you still managed to understand and apply those concepts) 🙂
Hi,
Thanks for sharing this! Fascinating topic. I used to enjoy coding in LISP and Scheme at university, so it's interesting to see these ideas popping up again in the SAP world.
cheers
Paul
Hi Paul
If you liked LISP, then take a look at Clojure - you'll feel right at home. And the great thing about this language is that it can run either server-side on a regular JVM, or client-side in a browser by using Google's Closure Compiler (cross compiles Clojure to JavaScript)
Regards
Chris W
Hello Chris,
you seem to advocate reactive software for the IoT. I think a mainstream functional language with reactive extensions will be a pragmatic approach.
best regards,
JNN
Hi Jacques
Yes, that is certainly true. The only effective approach to handling massive levels of communication concurrency is a reactive one.
Regards
Chris W
I encountered functional programming at university - first a lambda calculus, then as Miranda (David Turner was my lecturer). I didn't encounter it again until very recently.
Maybe functional programming will come into the language - perhaps ABAP 8! - and I for one would welcome it. The most important change there could be since the introduction of ABAP Objects. However, since I regularly encounter newbie abapers, and abapers with years of experience who still don't use SORTED tables (for example), I cannot see it being readily adopted. Worse, the sheer mass of what is described elsewhere of young and old dinosaurs, actively prevents the use of new paradigms. I know major projects (as do you Chris, since we've worked on some of the same), where we were forbidden to use OO because "the developers doing the support won't understand it".
It's good to see the comments about UI5 and Javascript. Maybe, as with webdynpro and ABAP Objects (to an extent), these will be the advanced forces for modernity into the SAP landscape!
Hi Mat
Long time, no speak!
Yes, the burden of SAP's own legacy is very large and is a perfect example of the "innovator's dilemma". So you're right. Even if some future version of ABAP does allow for true FP, then this will be a third paradigm it work: we started with imperative procedural coding, then extended it to imperative object-oriented, and now some attempts are being made to include "functional-like" constructs.
All that being said, the main thrust of my argument is not so much to insist that ABAP transform itself into a true FP language, but rather that when it comes to being able to handle the next phase shift in computing (Cloud and IoT), the FP paradigm provides the only workable solution to the problem of concurrency.
The heart of my argument is that the imperative programming paradigm simply cannot provide the required solutions to the problems I see looming on the horizon. And that problem is how to support massive levels of concurrency.
FP languages such as Erlang and Clojure have already solved these problems (in the case or Erlang, the concurrency problem was solved 25 years ago!), but since these languages lie way outside our comfort zone, there is strong resistance to consider them as viable solutions.
I see the IoT as the place where this resistance can be broken down.
Regards
Chris W
Going forward, I believe functional programming will have a better chance in HANA native then in ABAP. The ABAP development community is so resistant to change that a small amount of developers know how to program OO. Funcional programming is another leap that isn't going to happen.
I don't know how they will survive in a world where SAP seems to be sending the subliminal message that ABAP will be restricted to legacy, and is basically excluded from the public cloud. In the end it's their own fault for not keeping updated.
If you look at wikipedia for functional programming they compare functional to imperative using the fibonacci example which doesn't help to establish functional as an alternative to imperative, and rather a language apropriate for mathematics.
Hi Joao
I agree that trying to make all ABAP programmers adopt the functional (or even OO) style of programming is like asking an elephant to start climbing trees - its just not going to happen.
Its very difficult to modify a language like ABAP and still maintain backward compatibility with the huge legacy of existing code. What this means for ABAP's future is not exactly clear to me.
However, what is clear is that we cannot continue the way we are currently going - in spite of the impression created by Wikipedia (BTW, I wrote the Erlang example for the Fibonacci calculation in that article 🙂 )
Regards
Chris W
I speak for myself, but I for me the most difficult thing is to imagine common everyday problemas being solved with Functional programming. I want to update a business partner, how will I do it in functional vs imperative?
Every example I see of functional is mathematical related, and for that I can understand the point. No state is great for somethings, but if you work with Gateway you will know that for some stuff it's just a pain that leads to inneficiency.
Hi Joao
I think the difficulty you're experiencing comes from the fact that you simply have no experience yet in applying FP techniques to a business scenario.
I don't have this experience either, but I must continually remind myself that I should not confuse what I don't know how to do, with what cannot be done.
As for the stateful/stateless incompatibilities, this is one of the major design features in the S/4HANA architecture. It has be decided that all UI requests will be stateless, and this means that major areas of business processing are being redesigned.
Regards
Chris W
Yes I know, but that's why I wanted to see an example of FP applied to real business problems I face daily. That it can be done I have no doubt, but should it be done? Is more efficient? More maintainable (of great importance in enterprise software)?
As for the stateless, I hope S4/HANA deals with concurrency better then gateway, because right now people will either waste a lot of time with expired ETags or override other people's work.Not to mention the computational cost of having to rebuild the model (or having the model and business logic in the Browser/Javascript).
Hi Chris,
First of al, this is really good blog.
I notice a similar (but smaller) divide, when talking about Business Rules (more declarative, comparable to Functional) vs Business Process (more procedural, comparable to Imperative).
Cheers, Wout
Hi Chris,
very very interesting thing. I expressed most of my thoughts under the original blog by Christian Drumm's so there is not much more to add to it.
Maybe just one thing. I would not discriminate those without CS degree (you didn't say that, but generally the sentiment is that FP is more "scientific"), they can learn FP and use it if they want. It is the "volume of logic/abstraction" needed (sorry, struggling with English here) to solve a problem. I can do the imperative way as well as FP, but I would only use FP for very logical and very abstract and high-level-modelled things.
Shortly speaking I would use FP to build a model of something, but I don't see my ABAP programs as models of the world - they're just recipes how take a selection on cost centers and return the report data. As I said I struggle with the English language here but I hope it is clear what I want to say.
cheers Otto
Hi Otto
I have the impression that you think this way because you have picked up the assumptions made by the environment in which you were educated. Its not that universities ever say explicitly that FP is "only for academic or mathematical use", but its more the climate, or tone of voice in which they describe the use cases for FP.
Consequently, CS students leave university with a framework of thought that implicitly excludes certain uses cases. This is not because they were explicitly told "FP is not suitable for use in a business environment", but simply because no one has ever seriously taken on the challenge of applying FP to a business software solution. What I'm really talking about here is a mental predisposition that silently excludes FP from use in the business software world.
Having said that, I am certainly not advocating the whole sale adoption of FP into the ABAP world. This would confuse too many old-timers who still can't (or simply refuse to) understand object orientation! And besides, FP would not necessarily solve certain problems any better than current imperative solutions.
I need to be very clear here when I stand up as an advocate for FP. I am not wanting SAP to adopt FP simply on the basis of it being some novelty or that it is "way of the future".
To push FP as a novelty is to misunderstand the world of programming. LISP was first published back in the late 1950's and has been in continuous use ever since. This actually makes it one of the world's oldest programming languages. So from that point of view, why is everyone not using, or at least familiar with it?
I advocate the use of FP because it has a proven track record for solving concurrency problems.
Just to clear up a common misconception that I often hear: concurrency and parallelism are not the same. They are two very different, but related problems. The easiest way to remember the difference is as follows:
Concurrency is where many tasks are handled simultaneously by one person. Think of a teacher in a class of 30 small children ("Miss, I've lost my pencil", "Miss, Peter hit me!", "Miss, I need the toilet" etc... all at the same time)
Parallelism is where one task is divided into many pieces and each piece is handled by a separate person. Think of the same teacher now having several teaching assistants who can each handle the children's requests whilst the teacher does something else.
The IoT will introduce problems of large scale concurrency. If we further break down the solution to the concurrency problem by implementing parallelism, then that is a different situation.
The company that successfully solves the communication and request concurrency issues introduced by the IoT will be well on the road to success. And this is where I see SAP needs to start building cloud-based solution that will explicitly be able to handle the expected massive levels of request/response communication that will exist when the IoT kicks into gear.
Just to expand on my own recommendation in my blog, there are three possible solutions for solving concurrency problems that all have a highly successful proven track record:
1) Erlang
2) Clojure
3) Scala using the Akka Framework.
Options 2 and 3 above are both JVM based solutions so will be closer to many people's comfort zones.
Although Erlang is a niche language, it was specifically designed to solve the problem of massive levels of concurrency that occur inside a telephone switch. However, that problem domain has expanded beyond the boundaries of the telecoms industry - we now call it the Internet of Things. Therefore, Erlang should also be taken seriously as a language for building this type of solution.
Regards
Chris W
Hello Chris,
I know you mean well (that is how I read your article and comments). I also mean well. I don't need to be told how I think or what my problem is. I can do both FP and the "mainstream way" and find your thoughts cool and worth more than a minute to post a I-have-been-there-I-know-it-all comment. Which I didn't do. I am telling you that:
I can't see this happening any time soon and to make my point I am pretending I am a "normal mortal". That you are not. You are not a mainstream SCN user or a mainstream ABAP developer. Just because I am pointing out how the majority thinks I am not approving it or feel like a part of that majority.
I was given a framework of tools, thoughts and ways how to learn and adopt new things and that is THE OPPOSITE of quickly excluding things. What I wanted to say was that although I put a lot of thought into the topic, I can't see good examples why I would drop everything and start exploring FP way of ABAP in my free time. And I would do this and I would enjoy this. See people say cannot start with OO ABAP because they don't have a suitable project/topic - there it does not make sense for me. But because FP would never be the way for everything in ABAP, finding good use cases is crucial.
I don't know where to start or how to start - simply because I don't see enough use cases. I wanted to say you can help people (like me) by giving us some topics and I also wanted to say that I will be monitoring the discussion to learn more about good ideas by other people. I am eager to learn.
If you're trying to be Steve Jobs here selling people something that they don't even know they want or need - well, good luck with that. If you take this topic seriously (introduce FP to the ABAP world) and it is not just a random idea, maybe you should spend more time thinking about how to build bridges or better to say stairs from where we are now to where your vision is. No need to judge me or the world. I can see you know what the common problems are. That means you should be able to put together a process how to walk people through those problems. That is what I would do.
I just wanted to indicate there are people "like you" and wanted to send a positive signal. Unfortunately now I feel I failed doing that.
cheers Otto
Hi Otto
I'm not trying to tell anyone how they do, or ought to think. Yet at the same time, the way my mind works is to identity (as best as possible) the root cause of a situation. This necessarily means that I examine my own thought processes to try and understand why I think the way I do, and why I consider one way of doing things to be "better" or "worse" than any other way.
Having done this for several years now, I can see that I hold patterns of thoughts that are the product of many different factors (language, culture, peer group, ethical values etc), all of which have been assimilated into the framework of thought that I currently hold.
I don't wish to offend anyone by trying to tell them how think, yet I can see patterns of thought in others that are similar to the ones I've identified in myself. When I point out these patterns, people often become defensive and think that I'm being judgemental or "preachy". I am not, yet at the same time, how should I communicate what I observe?
If you do think I am being judgemental, then apart from not wishing to be, please understand that I have first been just as judgemental on, or critical of myself, and am reporting the results of my findings.
I don't want to cause offence to anyone here, yet in the case of the functional and imperative programming paradigms, I can see that much of the separation between the two camps comes far more from different ways of thinking than it does from any technical reasons.
All I'm trying to do is dig down to the root cause of why there is a reluctance in the business world to adopt FP. And the answer to this question is far deeper than simple questions of technology. Its also closely related to how the people using the technology think.
How to proceed then?
You're right - mainstream ABAP developers will probably never adopt FP unless they a) have the tools to do so, and more importantly b) have a legitimate use case in which to use those tools.
I too am looking for legitimate use cases for the application of FP within SAP, and at the moment, I don't really see any within the world of "traditional" ABAP programming.
However, associated with that domain are the worlds of HANA and IoT.
For example, processing stream based data is a task ideally suited for FP, so here I can think of a couple of business use cases:
Take the point-of-sale data coming in from all the tills in the branches of a large retailer like El Corte Ingles or Aldi or Walmart. This data can be stored in HANA and crunched using an FP language such as R. (HANA already uses R internally)
Another example is the requests that will come into a HANA system from IoT devices. Such requests will either be communication generated directly from active devices such as sensors on equipment or tags on livestock, but also requests from intermediate devices that have picked up a signal from a passive IoT device such as a low energy Bluetooth beacon.
I certainly do no consider myself to be like Steve Jobs "trying to sell the world something they don't need". Quite the opposite. I think FP is a very necessary tool for solving a problem I see looming on the horizon.
This is why I said in my blog that the place in which I see real innovation happening is not directly within the ABAP programs of tradition SAP systems, but in how we connect to the IoT and then handle the huge quantity of requests.
I'm struggling to find the right words here to describe what I see as the root cause of the problem. If I use the wrong words, as I seem to have done, people start defending themselves when no one is attacking (at least not intentionally). This is not the effect I wish to create, but at the same time I wish to highlight the fact that treating the problem as simply one of technology is to miss the more important underlying point.
Does that make things any clearer?
Regards
Chris W
Now I got it. We are on the same wave-length for sure!! I am suffering from all the problems and opportunities described above! If I see what you've said from this perspective, it makes perfect sense as well but with a completely different meaning. You've just found a friend if you need one 🙂
cheers Otto
p.s.: If that changes anything, try to think like you normally think when reading what I write. Have a nice day!
I believe you got the "quote" wrong. Steve Jobs, according to many, tried to sell the world what they didn't know they needed. Which is the same you are trying to do,
Like Steve Jobs. What Otto said is that you are failing to comunicate a use case, like Apple failed to do with the Apple Watch. Why do I need this?
I believe you are too focused on theorizing about why people do X, instead of showing us why we should do Y. We are rational people, we are "wasting" time discussing this, but we would rather read about use cases for FP then being psychoanalized. Instead of trying to change society as a whole, start by convincing this small group that there are use cases for this in the enterprise world.
In this last post you provided some real life use cases, which do get us thinking 🙂
I wouldn't say Chris failed to communicate. That's a bit too harsh. I missed a part of the message, my fault. It is clear as daylight that this is just a first step and much more must be done. Let's see the positive side of things 🙂
Do you mean "wouldn't"?
Oh god! Sorry, my sincere apologies. Brain<->hands out of sync. Of course. Geez. Thanks Matt
You missed part of the message, so did I. It's your fault? If someone is trying to send out a message, and convince people of something, communication needs to be clear so there are no misunderstandings.
It's not such a big deal, and besides it's just MY opinion. I'm not insulting the man, I'm not saying he is a bad professional, I just said that in this particular instance, in my opinion, he failed to communicate the use cases.
Hello Chris, thanks for taking you time into sharing your thoughts about ABAP, FP and IoT. I'll definitely read more about the topic.
I like to try Ruby on Rails on my spare time, and that's how I came across a Brazilian programmer named Jose Valim, a former Rails top contributor. It seems he's creating a new language called Elixir, built on top of Erlang VM. Maybe you already know about this language, as you're more into this topic than myself and as it seems Elixir has became somewhat popular already, with several books published (Learning resources - Elixir). Would love to hear some thoughts from you about Elixir if you ever had the chance to give it a try.
Anyway, just wanted to share the only thing I know about functional languages. It's obviously not much, but I hope I can change that in the near future 🙂 . Thanks again!
Hi Mauricio
I haven't used Elixir directly myself, but there's quite a lot of Twitter activity about that language now on the hash tag #ElixirConfEU. One tweet shows how one system based on several different tools (Node.js, Upstart and Pusher) has been replaced with an entirely Elixir based system.
This type of system could easily act as a frontend for HANA - see Harry Weppner's blog on connecting Erlang with HANA.
Regards
Chris W
Also check out Elixir - The next big language for the web
Hi Mauricio,
I have played with several functional languages (e.g. Clojure) including Elixir. So far I think Elixir is quite interesting. Especially the Erlang VM features related to massively parallel systems convinced me to try Elixir in the first place.
Additionally, there is also a Rails inspired web framework called phoenix (http://www.phoenixframework.org/) which looks quite interesting. However, I haven't tried it myself yet.
Christian
Hi Chris,
Thanks for sharing your thoughts. I completely agree that the use cases for functional programming within the SAP world are not to be found within the ABAP and classical ERP domain, but in new territory such as iot and big_data (Apache Spark anyone?).
Personally I'm still trying to learn this new mindset. Takes me longer than expected (wonder what age has to do with that 🙂 ). Focusing on Scala at the moment, one of the advantages being the sophisticated type system (incl type inference).
Cheers, Fred
Hi Fred
Thanks for your comments. I'm also going through the process of "mental reconfiguration" to think in functional terms, and yes, I think increasing age slows the barain's ability to reconfigure itself...
However, this old dog has been learning some new tricks by working with Clojure, and I'm in the process of finishing a MOOC on Erlang programming run by the University of Kent here in the UK.
If you're looking at FP in Scala, then also take a look at Akka. This is a concurrency framework for Scala.
Regards
Chris W
As a direct result of this blog, I've started learning two (yes, count 'em) two functional languages. Both of them are natural extensions to my daily work. As a Java developer Clojure seemed the obvious choice. For the SAP side of things, R looks fascinating, since it is a language usable in HANA shared procedures. R is somewhat easier to get to grips with than Clojure - which frankly gives me headache at the moment.
The two books I'm using - and find quite good - are: Clojure Programming from O'Reilly, and The Art of R Programming by Matlof. What's interesting about the latter is that, unlike other books on R, it majors on programming rather than statistical analysis. It does helpfully point out that for programming in R, you don't need to be overly familiar with stats.
Hi Matt
That's great to hear!
I've learnt most of my Clojure by working through the problems on http://www.4clojure.com. I'm up to problem 89 so far.
What I've found helpful with this website is that once I've solved a problem, I can compare my solution to an expert's solution. This is really helpful because after I've been through the mental leg work of solving the problem myself, I can appreciate the skill of the expert solution which is usually much more concise and compact that my own.
This is how I've picked up many of the tips and tricks for solving a problems in a functional manner rather than an imperative manner.
I haven't done too much with R, but it looks much like the scripting language used in MatLab.
Regards
Chris W
Even better, I just received my first bit of real work involving R. 🙂 (Let everyone else chase after UI5, Fiori,... I want to munch on "big data").
What people mostly forget is that ABAP has a new purely functional language right builtin. I speak about Open SQL and CDS. It is not quite Turing complete now, but we are working an that ;-). If you are using external HANA calc views as well, you even have a Turing complete purely functional language at hand.
Every CDS view can be seen as a function in that way, which gets a set of parameters and returns some values in the SELECT list. There are no variables possible, like in purely functional languages like Haskell. You can use the resulting function in the next CDS view and built already many practical algorithms with it.
For demonstration purposes I created a chess move generator pureley with CDS and Open SQL. This move generator takes a chess position (with all chessman on the board) from a database table and generates the list of next moves possible, including castling, en passant, king would be in chess, other chessman in the way, etc. with just a single SELECT statement on a CDS view. If all features I used in this move generator are available publically, I might blog here about that ;-).
By nesting those CDS views (so for every move you want to see in the future, you have to create another view on top) and combining this with some aggregate view, you even can built a complete "chess engine" giving you the next move. You may get performance problems, even on HANA, however ;-).
So regarding functional programming: Before changing from ABAP to some other cool programming language, it is best to fully exploit the capabilities of the ABAP ;-). Try to detect series of ABAP statements and replace them by a single SELECT statement. And you will even get multithreading for free.
Best regards,
Kilian.
Hi Kilian,
what you have done with CDS sounds quite interesting.
It would be great if you could publish some more details, e.g. as a blog post on SCN.
Christian
Hi Killian
Nice to hear your comments. As I said at the start of this blog, I've somewhat lost touch with the latest developments in ABAP, so its good to hear your side of the story.
I've read a little bit about CDS (Core Data Services for those who don't know what the latest acronym means), but there was nothing in the design paper that I read (or maybe I didn't notice) that described a CDS view in these terms. Nonetheless, it goods to hear this.
One question though, are CDS views truly first class citizens? In other words, can they be:
If they are, then how will function composition and partials work?
Also, it sounds like you're wanting implement CDS views as pure functions? Since a CDS view provides access only to the data in a database via some SQL statement, do you see any legitimate use cases for side effects (E.G. sending and receiving data over the network)? If you do, then if CDS views are also purely functional, we will be heading down the road towards the need for monads - which tends to make many people run for the door... 😀
As an aside: even though Haskell is a purely functional language, it does have two categories of named values (or cells), mutable and immutable. Other impure functional languages (such as Erlang) only have immutable values because they only implement single binding. So a variable name is either bound or unbound.
In Haskell, you can say:
But then the value of foo is immutable.
Haskell also has mutable named values (called mutable cells):
Now, the value of v is mutable, but only through the specific functions [read|write]IORef
Regards
Chris W
Hi Chris,
of course CDS, like SQL, is not functional programming language in the
sense Erlang is. It is not a programming language either (as I say,
not even Turing complete). So CDS views (like tables or views in SQL) are not
first class citizens of the language.
But writing complex programs purely in SQL is done in the same way
than writing a functional program in say, Haskell, as you don't have
variables at hand and may only proceed by calling other functions
or doing recursion. Common Table Expressions and Joins can be seen
as lambda functions and creating new functions from existing ones.
And you have all the same benefits from using SQL than using a functional
language itself.
You get:
1. Free multithreading in ABAP ❗
2. Nearly side effect free behaviour
So calling CDS a functional language is overly optimistic. But for many
use cases they are an adequate replacement.
It must be noted, that HANA SqlScript also started as a purely functional language, but departed in other directions later on.
Best regards,
Kilian.