The Long Run

Change is the only constant in life. I see this here in Berlin where I live on a daily basis. The city is adapting structurally to the ever changing environment with new people from many different cultures moving in, new hipster eateries opening up, old houses either getting renovated or torn down to get replaced by new ones. While the west part of the city isn’t that much affected by this, the former east and especially the areas in the center have been turned upside down. This multitude of cultures coming together, an ever changing environment and a rich history make Berlin, for me and many other people, currently the most exciting city in Europe. SCN has seen a phase of functional stability over the last three years. With Maggie Fox and Jeanne Carboni spilling the beans in their recent blog posts, a major change is coming to SCN. The 1DX program has been the trigger to rethink our strategy and make it fit and adapt to the larger 1DX goals. The plan is to decommission the current platform by end of the year and have it replaced by a modern and longer-lasting combination of new core applications, supported by small, dedicated services built on top of a cloud infrastructure.

I would like to give you an insight into our decision process, why we think the current application that we use for SCN doesn’t fit into the overall 1DX vision and how the new SCN and its high-level architecture are going to work.

Status Quo

When designing a new collaboration experience in order to replace something familiar, something we have gotten used to over the last years, it is important to look back at what we currently have.

SCN in its current form is technically a single application. A big monolith, representing a fully integrated social network, which allows you to follow community member but also spaces and content items and get notifications and updates in your personalized activity stream. Especially the social integration aspects of the application are one of its strength, but also the very rich user profile, including the Gamification integration we rolled out close to two years ago.

On the other hand we have core functional elements for creation of blog posts, questions, documents and polls, which don’t live up to current standards. Content is organized by Spaces, which have proven to be inflexible with large amounts of content. A rich set of internal APIs and a plugin framework allow building integrations of external services and small enhancements, but due to its monolithic nature, integration of social functionality into different sites turns out to be difficult at best.

Although fully integrated by design, the content and member moderation features, key functionality for a large social collaboration site as SCN, have been a major issue. Moderators are struggling with abysmal capabilities at hand and the community managers had to fall back to after-the-fact-moderation for blog posts due to a major lack of built-in pre-moderation.

Mobile access is unfortunately a similar story. The application lacks responsive design and we had to rely on the mobile apps provided to us by the vendor. The native app comes out of the box with no acknowledgement of the Gamification extension (released by the same vendor), branding capabilities or integration of functional enhancement. Of course this is a limitation of native mobile apps in general, but the desktop focus of the web site prohibits any mobile representation of even the smallest extensions we rolled out in the last years.

Personally I see the social network aspect we got with the current SCN as a major win for community. There was nothing like that four years ago, and these days it is normal to @ mention somebody in a blog post or check your Communication or Activity Stream to see what is going on in your peer group or the spaces you are following. It has been an exceptional experience to see all of you adopting these new social gestures and capabilities and has formed all of our expectation on how a modern community interacts and communicates, even at large scale.

1DX Vision

Beginning of last year, we on the SAP IT side started the first conversation and workshops with the SCN team, trying to formulate what the 1DX vision actually means for the community. Maggie shared with you already the 1DX high level vision. Behind this are goals to harmonize the overall end user experience across all major SAP sites (like SAP.com, SCN, PartnerEdge, Service Marketplace, Support Portal, Help Portal and SAP Store). Let’s focus on the goals that will have a direct impact on the actual implementation of the future community platform and all of its components.

Unified Look & Feel

We had the SAP Gold theme now being the global brand for all SAP sites, but over time there were may interpretations about how these guidelines should get applied. 1DX not only defines a new, fresh look & feel (L&F) with proper implementation guidelines, but also common page templates e.g. to make different content types look and feel similar from a usage perspective. Ideally a Like or Share button should be at the same position on a content page, independent if it is a blog post, an idea or a product page on SAP.com.

Unified Header & Footer

Part of the L&F is of course the site header and footer. There might have been guidelines, but again interpretations have been many without strict governance and control. 1DX defines a unified header, to be used across the SAP sites, including navigation entries and common controls to access the user profile, SAP search, etc. Similar guidelines are defined for the site footer.

Unified Navigation

Navigation in the core SAP sites were never aligned or harmonized at all in the past. The most complex implementation of a navigation that we probably have is the one here on SCN. This is one of the most discussed items in the program, and a tricky one too. In the end, in an ideal scenario you would want to have a simple, clear navigation, consistent on all SAP sites, always getting you to the right place for the product, topic or industry solution dependent on the context you come from. A tough nut to crack.

Unified Profile

On SCN we have the most rich user profile of all SAP sites. It is loaded with useful information about the registered member, it has a social URL (containing your unique username), and it provides you insight into the member’s activity, content created and reputation earned. Again, 1DX strives to provide a single user profile for all. The main contributing site to this new profile will be SCN.

Unified Metadata

On SCN we have used a rich set of defined metadata for tagging since end of 2013. The goal was to provide a clearer indication to our members, what a blog post or discussion is about. SCN currently provides a fixed set of defined tags for topics, products, industries, events and tasks. With 1DX we do the next steps and align this metadata with the harmonized corporate metadata schema that will be used across all core SAP sites. The goal is to have all content pages tagged with uniquely identifiable keywords, which relate directly to an SAP product, industry solution or topic, maybe down to a specific product version.

SCN and 1DX

Many of these tangible and from an end user perspective much needed goals only became clear once we drilled deeper into the high level vision of 1DX, but we saw already very early on, that SCN in its current form of a single monolithic application would not fit into this vision. We sat down to define our own vision statement, which still for me personally brings it all together in a very beautiful way.

“Enable anyone on any device to collaborate, engage, network, and access relevant, quality community content across all SAP touch points.”

Or put even more simple, as Bill McDermott would: Community everywhere

If we try to harmonize all these aspects as part of 1DX, then this also should include social features that we have on SCN. We would still want to have an activity stream, but it should be able to show activities from other sites (e.g. likes and shares) and be easily accessible from anywhere. In a similar way, activities from across all sites should be able to contribute to the Gamification program and be presented in the unified user profile.

With the goal to harmonize the overall user experience in such a drastic way, an “isolated” social network application like SCN could simply not work. We would have to customize the current solution in a way where it could not be properly supported anymore or would defeat its original purpose and design goals, to provide a fully integrated community platform.

In addition, we also looked back to the last three years to see what was working really well and what not. Again, the integrated nature of the current solution shines, but many other aspects not so much. We heard many of you loud and clear over the years, that central functional pieces like blogging, discussions or moderation don’t live up to current standards. And looking at the roadmap of the third party solution that we use here, there was no salvation in sight. This application is never going to provide the best tools that our community needs to get the job done, neither for the members to find an answer to the pressing issue at the job, nor our army of moderators who try to ensure on a daily basis the high quality standards that we have for content.

This is how we came to the decision to go back to a best-of-breed approach and decommission the current solution.

Goals

Once this idea sank in, we started working out the details of how a future community could look like, where single-purpose applications would integrate into a social network that spans across multiple sites.

If you look at our current solution, you can identify a core set of functionality, which would need to be replaced in a future scenario, ideally with a better or at least equally good alternative.

• Blogging
• Discussion
• Documents
• Polls
• Bookmarks
• Group Chat / DM
• User Profile
• Content Browser
• Activity Stream
• Social Network
• Gamification
• Moderation
• Search

Some of these functions are of a contributing or collaborative nature, where a user would go in and create a blog post, question, or comment and by that contribute to the community in an active fashion and by that produce content and activities.

The others are more a type of service, where a user would search through all contributions, calculate and show a member’s reputation, browse the content, bookmark a blog post or use a personalized, time based view into all the contributions. These services consume the activities and content created by the producer applications and provide integrational value. I would categorize the user profile and social network (building your social graph, @mentioning, following spaces) also as services, with moderation somewhere in between (well worth another blog post).

All of the functional components allow the user to interact with content and other members. The purpose of a social network is to expose these interactions to your peer group in order to foster collaboration and to make it easy to interact with your social graph or discover other people and content of interest.

With that, all the end user activity becomes the glue that defines the social nature of such a scenario. In a community platform with distributed applications and members contributing and collaborating in them, we have to make sure to collect all their social gestures (post, like, share, follow, etc.) and distribute them to the social services for consumption. The same thing will happen here the moment I publish this blog post. The event will get tracked, packaged and sent to an application-internal component, where it will be persisted, accessible for other components like the user profile, activity stream or moderation. Except in the new best-of-breed scenario, we’ll have to deal with many distributed applications and services, each specific for its purpose.

Spaces vs. Tags

I have to quickly touch on the space/place concept, which our current platform uses to organize content. A space is mostly a bucket into which you put content. You see this every time you create e.g. a discussion–we ask you where to create the content, and you are in the delicate position to pick one of 500+ spaces available in SCN. Spaces are mostly organized by SAP products, industries, topics or events.

What we’ve seen over the years is that spaces are really hard to manage. Due to the massive amount of content we have in SCN, it becomes impossible to merge or split spaces and their content without an unacceptable long downtime of the system (because this system process should not get interrupted by users creating new content in the processing spaces). It is also worth mentioning that spaces are an artificial concept of the solution we use for SCN. Although tags and even categories exist already, we forced ourselves to organize the content and permissions for managing the content into this concept, which turned out to be a not-so-great idea.

In the new community platform, content will solely be organized by tags (and yes, this is worth its own blog post too). A team in the 1DX program is working on a metadata schema which will reflect all SAP products and topics, so that we can offer you a very rich set of metadata that everybody can use to tag the content. This will allow for a much greater flexibility and more granular contextualization of content and activities.

High Level Architecture

If you made it this far and didn’t get distracted by some push notification from your mobile (note to self: mention new focus on mobile below), you are probably really interested in how this is going to work. It all depends on the producing application being able to track all of the user activities, package them into a harmonized format and make them available to the consumer services that provide the social integration layer.

The top layer shows producing applications. Once you have registered and logged in, any of your contributions and social gestures will get sent to the Activity Aggregator (sort of a misleading name, because aggregation means something different in the SAP ecosystem…but this is what we call it), which distributes them to the consuming services in the lower layer.

The Activity Aggregator is based on Apache Kafka, an open source project originating from the needs of the LinkedIn platform. The purpose of the Activity Aggregator is to provide an API endpoint, which is always available to receive messages, store them for a defined amount of time, and make them available to the consuming services, based on a subscription model. The consuming services only get a subset, targeted to their specific use case.

Apache Kafka is a message broker, which is highly performant, redundant and scalable. It has gotten a following in the recent month and I highly recommend having a look into the project, in case you find yourself having to deal with a similar problem. There was a very good podcast episode by Software Engineering Radio where Kafka was discussed in great detail.

The benefit that this architecture gives us is that we have the ability to plug in new content- and activity-producing systems over time, and by that slowly extend SCN over time. Also all of the producer applications are separate servers and don’t have any hard technical runtime dependencies, and therefore can be extended and maintained separately. No more single downtime for all of SCN just because we have to release a single bug fix in a small component.

Same is true on the consumer side where we can add new consumers on demand, only providing them a targeted subset of all available activities for their specific use case.

We run all applications and services in our own SAP Cloud. For Kafka and the producer application we leverage our own SAP IT cloud called Monsoon. Some consumer services are also run on Monsoon, while other small, community-specific services are built on HCP as dedicated, single-purpose applications. This gives us a high fault tolerance, so that even when a single application fails, only small parts of the overall functionality become unavailable, while the rest of the community will be unaffected and you might not even notice it.

It is also worth mentioning that all applications and services will be mobile enabled using a responsive design approach. Small supporting services are custom built, using an “API First” approach, which will provide us the ability to build alternative UIs like mobile applications in the future.

Challenges & Wins

As you can see, this is no small undertaking and there are many challenges ahead. The biggest one from a user experience perspective is probably to make it look and feel as integrated as the current single application approach. I will be very straight here and say that we probably won’t get it right on a first try. I’m pretty sure that there will be functional gaps with version 1.0 to what we currently have. Think about something like @ mentioning across distributed applications and getting proper notifications. We know how to do it, but is still a substantial effort to get all the details in place to make it look and feel like a fully integrated experience.

Then again, I also see so many advantages over what we currently have:

• much richer single purpose applications, like a top-notch Q&A platform, one that comes close to what you see with stackoverflow.com
• a state-of-the art blogging platform with WordPress
• new tools and approaches to tackle moderation and Gamification
• the possibility to extend SCN way beyond what we currently have and integrate other SAP core sites in the future
• and finally, a decent mobile UI that adapts to various device sizes, along with the ability to build native mobile applications specifically for you

In case you got the impression over the last couple of months that SCN is not moving ahead, then this is because we have been busy with building the future of SCN. We have made good strides and we plan to provide you some early access to what we are building in the next couple of months.

This might not be as exciting as Berlin, but you are the citizens of this great community and we are building this for you. It will be a crazy ride and there will be bumps in the road ahead, but we are in it for the long run.