Open Source is an important working model in the software development industry today. It is very inefficient, if not impossible, to develop software without it. There are quite some very successful Open Source projects and many technology areas are even dominated by Open Source, such as big data, machine learning, Internet of Things. Thus, it might make sense to look in more detail into the way Open Source communities work and try to learn from their best practices. That’s what InnerSource does.
InnerSource is the practice of applying Open Source method and best practices for in-house software development. It aims to improve internal development processes based on the learnings from the Open Source style of collaboration.
Some of the benefits of this working style, inherited from Open Source are:
In this blogpost we will explore in more detail the InnerSource working model, as well as discuss its challenges and the actions we are taking to make it a success at SAP.
In a traditional development model, a development team (called “Service Provider” in the picture below) develops and provides a reuse component or service to its consumers, for instance other development teams who use that component or service for their projects.
A consuming team sends bug reports and feature requests to the providing team. The providing team develops bug fixes and features accordingly which are in turn consumed by the consuming team.
Since typically there is not only one consuming team but many, the providing team receives a lot of feature requests and bug reports. Because of its resource constraints, requests must be prioritized, and not all requests can be delivered on time. This, in turn, creates challenges for the consumers. Some of them might decide to not use the reuse component from the providing teams but develop a similar functionality themselves. This means double work, unnecessary competition, and redundant projects and is thus the opposite of reuse.
In an InnerSource model, the providing team would open the reuse project for contributions from the consuming teams. These teams could contribute bug fixes and features themselves instead of only reporting bugs and requesting features.
In such a model, there is no need for consuming teams to abandon the reuse component of the providing team to create a similar functionality. Instead, they could support the providing team by directly contributing the requested features – which would be cheaper from an effort perspective for both sides – and would make the reuse component more versatile and better reusable for other teams as well.
For consuming teams, this approach typically leads to higher satisfaction since they are actively involved and to higher motivation since they can influence the reuse project. They also get the required features and bug fixes faster.
Basically, InnerSource means developing rather in communities and in a collaborative way than in dedicated project teams. While these teams also exist in an InnerSource approach and – depending on the governance and collaboration model of the project – might continue to have the overall responsibility for the project, they are not the only ones who contribute to the project. In fact, there is a broader community of engineers that collaboratively develops a project.
Such an approach has certain benefits. Some of them such as shorter time-to-market, closer collaboration between providers and consumer, and higher security by transparency have already been mentioned, but there are more.
The increased collaboration is an important value as such. Knowledge across teams and business units (aka. “silos”) is distributed and cross-unit collaboration is increased. According to Conway’s law (see [9]) this helps to build better (integrated) software.
Typically, InnerSource projects also have a higher quality. This is because it requires good documentation to keep the barriers for external contributors low. This also facilitates the onboarding of new members to the project team. Also, the high degree of test automation that InnerSource requires helps to improve the quality.
At SAP we observed an effect in InnerSource projects that we also saw in Open Source projects. In Open Source projects, the development team cleans up the code because once published, everybody can inspect it and thus bad quality code would be visible to the entire world. We see the same pattern in InnerSource projects: Before opening a project for contributions by others, the development team improves the code quality.
The example in the section “How InnerSource Works” shows that InnerSource is particularly well-suited for reuse components. It helps to avoid that the providing team becomes a bottleneck, increases reuse, and the consuming teams get features faster and have more influence on the project.
Overall, InnerSource promises to be a more sustainable development model than a traditional approach.
What do we do to establish an InnerSource culture at SAP? There are multiple development projects that have used InnerSource already for many years. In recent years we see a raising interest in InnerSource, and SAP’s Open Source Program Office (OSPO) was approached by some development teams to seek advice on how to implement InnerSource best for their projects.
To support them we started the so-called “InnerSource Group”. That is a virtual team consisting of OSPO members and members of projects that use InnerSource already (and thus are already experienced in InnerSource) as well as InnerSource enthusiasts across the company. This group is consulting development teams with regards to the implementation of an InnerSource approach, promoting InnerSource at SAP, providing documentation and how-to’s, along with tooling.
To act as a role model, the team itself is also run as InnerSource project. This means that the communication and the backlog are open and visible for everybody inside SAP. Also, everybody can participate and support the team.
One of the first steps towards an InnerSource culture is to increase the visibility of source code. Ideally, every engineer has read access to the source code that is produced inside a company. This enables teams to figure out solution approaches of other teams for certain problems which they might want to adopt. Also, if teams stumble across bugs in software that was provided by another team, they could try to find out the reason for that bug and inform the providing team accordingly to speed up bug fixing. Thus, making source code visible inside the company is already a big step towards better collaboration.
SAP policy is that source code should be readable by every SAP engineer per default (write access must be restricted, of course). Only if there are good reasons, development teams should restrict read access. Examples of this are legal or contractual reasons.
Making it easy to promote and find InnerSource projects is very important. At SAP, we regularly execute internal surveys and ask our engineers about InnerSource-related topics. Some time ago we were asking about the reasons why engineers might not contribute to InnerSource projects. Number 1 reason was that it was difficult to find InnerSource projects, see also [6].
To solve that problem, the InnerSource Group did two things. First, together with our HR department we created a special job category in our internal job portal. The new category does specifically support contributions to InnerSource projects. This enabled project teams to post contribution requests (“in our project ABC, we need support with task XYZ. To help us you need skills abc and we assume that this will consume x% of your working time over the next y months”). With this, on the one hand project teams can promote their project and on the other hand engineers can find possibilities to contribute to InnerSource projects.
Second, we created an internal website, the “InnerSource Project Portal”. This website is generated from our internal Git systems. It shows each InnerSource project as a tile with some basic information such as project description, #forks, #stars etc. Users can sort and filter projects by programming language, for example. Also, a free text search is possible. The software behind that is available as Open Source (see [2]). The concept has even become a pattern at InnerSource Commons in the meantime. For more information, see [8].
Training and education are other important topics. Not all engineers and managers might understand immediately what InnerSource is (and what it is not), how it works and what the benefits are. Particularly, colleagues who are used to a more traditional development approach, not being familiar with the open-source-way-of-working, might have a challenging time moving to InnerSource. There is a lot of good training and education material available - from InnerSource Commons for example, see [1].
At SAP, the InnerSource Group also created SAP-internal training material such as documentation, how-to’s and a beginner’s guide. InnerSource is also part of our internal Open Source Curriculum. This material was inspired by requests from our engineers and is extended and adjusted over time as required. Particularly, we were often asked how an existing project can be turned into an InnerSource project. Thus, we created as part of the beginner’s guide a high-level how-to-guide for a step-by-step transition to InnerSource.
Despite all this material, people might still have fears and concerns wrt. InnerSource. It is important to take this seriously since otherwise it will be difficult or even impossible to convince them. At SAP, we created dedicated documentation where we collected and listed different fears and concerns that we heard in the past (“InnerSource means that we lose control over our project”, “InnerSource means software development anarchy” etc.) and responded to that. With that we hope that we can educate people, reduce their doubts, and convince them of InnerSource as a good development methodology.
Another important aspect is success stories. Such stories (i.e., projects that successfully implemented InnerSource) that show how InnerSource approaches were applied in practice are a powerful means to convince people about the benefits of InnerSource. At SAP, we collected “reference projects” and testimonials. For each of these projects we provide some basic information, such as a project description, why InnerSource was chosen, experiences and contact information. The contact people are happy to answer questions that interested engineers might have. Such success stories are important because they show that InnerSource really works.
InnerSource promises important benefits, such as better software quality, better collaboration, increased reuse and – as a result – higher development efficiency. Despite these benefits, introducing InnerSource in an organization will not take off by itself. It requires persuasion, documentation, education, advice, tooling and – maybe most important – time. At SAP, we believe that we made good progress with the activities that we started so far, but we are not yet where we want to be wrt. establishing an InnerSource culture.
[1] InnerSource Commons
[2] Project Portal for InnerSource (repository on GitHub.com)
[3] Growing an InnerSource Culture @ SAP (InnerSource Commons Spring Summit 2020)
[4] The Unexpected Path of Applying InnerSource Patterns (InnerSource Commons Fall Summit 2020)
[5] InnerSource Rocks! (SAP Podcast “The Open Source Way”, November 2020)
[6] Solving the InnerSource Discovery Problem (GitHub Blogpost, March 2021)
[7] GitHub, InnerSource and the benefits of an asynchronous and knowledge-sharing culture (Betatalks the podcast with Martin Woodward, June 2021)
[8] InnerSource & Discoverability (InnerSource Commons Community Call, July 2021)
[9] Conway’s law: “Any organization that designs a system […] will produce a design whose structure is a copy of the organization's communication structure.” (Melvin E. Conway, 1967). Source: Wikipedia
-----
If you wish to learn more about SAP Open Source:
Learn more: Webinars | Podcasts | SAP Community
Explore: Engagement in Projects & Foundations
Engage: GitHub
Follow us on: Twitter
If you have any question, feel free to ask it here.
InnerSource is the practice of applying Open Source method and best practices for in-house software development. It aims to improve internal development processes based on the learnings from the Open Source style of collaboration.
Some of the benefits of this working style, inherited from Open Source are:
- Shorter time-to-market on features and bug fixes - consumers take a more active role in the development of the software provided to them through contributions. With the added capacity of contributions, features and bug fixes can be delivered earlier
- Traceability – by using internally public bug tracking and backlog, written communication on these systems is encouraged. Written communication leads to more traceable documentation on software design and decisions
- Security by transparency – the source code is internally open for inspection, which means that the vulnerabilities can be spotted earlier by the community
In this blogpost we will explore in more detail the InnerSource working model, as well as discuss its challenges and the actions we are taking to make it a success at SAP.
How InnerSource Works
In a traditional development model, a development team (called “Service Provider” in the picture below) develops and provides a reuse component or service to its consumers, for instance other development teams who use that component or service for their projects.
A consuming team sends bug reports and feature requests to the providing team. The providing team develops bug fixes and features accordingly which are in turn consumed by the consuming team.
Since typically there is not only one consuming team but many, the providing team receives a lot of feature requests and bug reports. Because of its resource constraints, requests must be prioritized, and not all requests can be delivered on time. This, in turn, creates challenges for the consumers. Some of them might decide to not use the reuse component from the providing teams but develop a similar functionality themselves. This means double work, unnecessary competition, and redundant projects and is thus the opposite of reuse.
In an InnerSource model, the providing team would open the reuse project for contributions from the consuming teams. These teams could contribute bug fixes and features themselves instead of only reporting bugs and requesting features.
In such a model, there is no need for consuming teams to abandon the reuse component of the providing team to create a similar functionality. Instead, they could support the providing team by directly contributing the requested features – which would be cheaper from an effort perspective for both sides – and would make the reuse component more versatile and better reusable for other teams as well.
For consuming teams, this approach typically leads to higher satisfaction since they are actively involved and to higher motivation since they can influence the reuse project. They also get the required features and bug fixes faster.
Benefits of InnerSource
Basically, InnerSource means developing rather in communities and in a collaborative way than in dedicated project teams. While these teams also exist in an InnerSource approach and – depending on the governance and collaboration model of the project – might continue to have the overall responsibility for the project, they are not the only ones who contribute to the project. In fact, there is a broader community of engineers that collaboratively develops a project.
Such an approach has certain benefits. Some of them such as shorter time-to-market, closer collaboration between providers and consumer, and higher security by transparency have already been mentioned, but there are more.
The increased collaboration is an important value as such. Knowledge across teams and business units (aka. “silos”) is distributed and cross-unit collaboration is increased. According to Conway’s law (see [9]) this helps to build better (integrated) software.
Typically, InnerSource projects also have a higher quality. This is because it requires good documentation to keep the barriers for external contributors low. This also facilitates the onboarding of new members to the project team. Also, the high degree of test automation that InnerSource requires helps to improve the quality.
At SAP we observed an effect in InnerSource projects that we also saw in Open Source projects. In Open Source projects, the development team cleans up the code because once published, everybody can inspect it and thus bad quality code would be visible to the entire world. We see the same pattern in InnerSource projects: Before opening a project for contributions by others, the development team improves the code quality.
The example in the section “How InnerSource Works” shows that InnerSource is particularly well-suited for reuse components. It helps to avoid that the providing team becomes a bottleneck, increases reuse, and the consuming teams get features faster and have more influence on the project.
Overall, InnerSource promises to be a more sustainable development model than a traditional approach.
Establishing an InnerSource Culture
What do we do to establish an InnerSource culture at SAP? There are multiple development projects that have used InnerSource already for many years. In recent years we see a raising interest in InnerSource, and SAP’s Open Source Program Office (OSPO) was approached by some development teams to seek advice on how to implement InnerSource best for their projects.
InnerSource Group
To support them we started the so-called “InnerSource Group”. That is a virtual team consisting of OSPO members and members of projects that use InnerSource already (and thus are already experienced in InnerSource) as well as InnerSource enthusiasts across the company. This group is consulting development teams with regards to the implementation of an InnerSource approach, promoting InnerSource at SAP, providing documentation and how-to’s, along with tooling.
To act as a role model, the team itself is also run as InnerSource project. This means that the communication and the backlog are open and visible for everybody inside SAP. Also, everybody can participate and support the team.
Visibility of Source Code
One of the first steps towards an InnerSource culture is to increase the visibility of source code. Ideally, every engineer has read access to the source code that is produced inside a company. This enables teams to figure out solution approaches of other teams for certain problems which they might want to adopt. Also, if teams stumble across bugs in software that was provided by another team, they could try to find out the reason for that bug and inform the providing team accordingly to speed up bug fixing. Thus, making source code visible inside the company is already a big step towards better collaboration.
SAP policy is that source code should be readable by every SAP engineer per default (write access must be restricted, of course). Only if there are good reasons, development teams should restrict read access. Examples of this are legal or contractual reasons.
Discoverability of InnerSource Projects
Making it easy to promote and find InnerSource projects is very important. At SAP, we regularly execute internal surveys and ask our engineers about InnerSource-related topics. Some time ago we were asking about the reasons why engineers might not contribute to InnerSource projects. Number 1 reason was that it was difficult to find InnerSource projects, see also [6].
To solve that problem, the InnerSource Group did two things. First, together with our HR department we created a special job category in our internal job portal. The new category does specifically support contributions to InnerSource projects. This enabled project teams to post contribution requests (“in our project ABC, we need support with task XYZ. To help us you need skills abc and we assume that this will consume x% of your working time over the next y months”). With this, on the one hand project teams can promote their project and on the other hand engineers can find possibilities to contribute to InnerSource projects.
Second, we created an internal website, the “InnerSource Project Portal”. This website is generated from our internal Git systems. It shows each InnerSource project as a tile with some basic information such as project description, #forks, #stars etc. Users can sort and filter projects by programming language, for example. Also, a free text search is possible. The software behind that is available as Open Source (see [2]). The concept has even become a pattern at InnerSource Commons in the meantime. For more information, see [8].
Education and Cultural Change
Training and education are other important topics. Not all engineers and managers might understand immediately what InnerSource is (and what it is not), how it works and what the benefits are. Particularly, colleagues who are used to a more traditional development approach, not being familiar with the open-source-way-of-working, might have a challenging time moving to InnerSource. There is a lot of good training and education material available - from InnerSource Commons for example, see [1].
At SAP, the InnerSource Group also created SAP-internal training material such as documentation, how-to’s and a beginner’s guide. InnerSource is also part of our internal Open Source Curriculum. This material was inspired by requests from our engineers and is extended and adjusted over time as required. Particularly, we were often asked how an existing project can be turned into an InnerSource project. Thus, we created as part of the beginner’s guide a high-level how-to-guide for a step-by-step transition to InnerSource.
Despite all this material, people might still have fears and concerns wrt. InnerSource. It is important to take this seriously since otherwise it will be difficult or even impossible to convince them. At SAP, we created dedicated documentation where we collected and listed different fears and concerns that we heard in the past (“InnerSource means that we lose control over our project”, “InnerSource means software development anarchy” etc.) and responded to that. With that we hope that we can educate people, reduce their doubts, and convince them of InnerSource as a good development methodology.
Another important aspect is success stories. Such stories (i.e., projects that successfully implemented InnerSource) that show how InnerSource approaches were applied in practice are a powerful means to convince people about the benefits of InnerSource. At SAP, we collected “reference projects” and testimonials. For each of these projects we provide some basic information, such as a project description, why InnerSource was chosen, experiences and contact information. The contact people are happy to answer questions that interested engineers might have. Such success stories are important because they show that InnerSource really works.
Conclusion
InnerSource promises important benefits, such as better software quality, better collaboration, increased reuse and – as a result – higher development efficiency. Despite these benefits, introducing InnerSource in an organization will not take off by itself. It requires persuasion, documentation, education, advice, tooling and – maybe most important – time. At SAP, we believe that we made good progress with the activities that we started so far, but we are not yet where we want to be wrt. establishing an InnerSource culture.
Further Information
[1] InnerSource Commons
[2] Project Portal for InnerSource (repository on GitHub.com)
[3] Growing an InnerSource Culture @ SAP (InnerSource Commons Spring Summit 2020)
[4] The Unexpected Path of Applying InnerSource Patterns (InnerSource Commons Fall Summit 2020)
[5] InnerSource Rocks! (SAP Podcast “The Open Source Way”, November 2020)
[6] Solving the InnerSource Discovery Problem (GitHub Blogpost, March 2021)
[7] GitHub, InnerSource and the benefits of an asynchronous and knowledge-sharing culture (Betatalks the podcast with Martin Woodward, June 2021)
[8] InnerSource & Discoverability (InnerSource Commons Community Call, July 2021)
[9] Conway’s law: “Any organization that designs a system […] will produce a design whose structure is a copy of the organization's communication structure.” (Melvin E. Conway, 1967). Source: Wikipedia
-----
If you wish to learn more about SAP Open Source:
Learn more: Webinars | Podcasts | SAP Community
Explore: Engagement in Projects & Foundations
Engage: GitHub
Follow us on: Twitter
If you have any question, feel free to ask it here.
- SAP Managed Tags:
