Application Development Blog Posts
Learn and share on deeper, cross technology development topics such as integration and connectivity, automation, cloud extensibility, developing at scale, and security.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

A Github repository issue backup tool developed by ABAP

JerryWang
Advisor
Advisor
‎07-14-2017 9:16 AM

I personally prefer to use Github repository issue to manage my knowledge. For issue, this is my github repository to store ABAP and SAP CRM related knowledge of mine and currently it has already 285 issues.




There are two main reasons which drive me to choose github repository as my knowledge management approach:



  • great speed of full text search

  • powerful and flexible label management of issues



As the accumulation of issue number, I have to consider the possibility of issue backup, so that I can still have access to my knowledge repository when the public github is not available ( for example due to network issue or access within China banned by government due to political reasons, who knows ).

There are plenty of articles and tools about how to export/backup github repository issues:






However after I go through them, I found none of the tool can 100% fulfill my backup requirement, as a result I plan to write one backup too on my own using github public API documented here.


Basic idea of this tiny issue backup tool implemented by ABAP


1.1 Issues should be persisted to ABAP server


Since as an ABAPer we have powerful Netweaver behind us, I plan to develop a tool which can extract all issues of a given repository and stored them in an ABAP system, so that I can use all kinds of ABAP tools for example CDS view to make analysis on those issues later.

Only a single API endpoint is used in this tool, see how to use it below.

The format of this endpoint url is:

https://api.github.com/repos/<your github user id>/<name of github repository>/issues





In my example above, the response is a JSON string which is an array with each element representing an issue detail.



For me I am only interested with issue number, issue title, issue body text, created and update timestamp, so I create the following database table to store github issue information:



1.2 Delta-backup functionality support


In github repository each issue is marked with a number, which could be easily found in issue url and issue detail page.




The delta backup function means for example I have finished the backup of current 285 issues, and later when I create two new issues 286 and 287, and then I run the tool again, only 286 and 287 should now be exported and persisted into ABAP system.

1.3 Paging in issue API needs to be considered


In current Github REST API V3, by default only 30 issues are returned in a single API call.

If there are still left issues to be exported, there will be two indicators stored in HTTP response fields of this single API call, one (rel="next") contains url which returns the next 30 issues and the other(rel="last") contains the url which returns the issues in the last page ). This paging logic should also be supported in the tool.



1.4 API Rate Limiting is not considered for simplification reason


For more detail about API rate limiting topic, refer to Github API documentation here.


How to use this tool


1. Create one transparent table to store issue detail



And fill one entry for the github repository whose issue you would like to extract accordingly.

You can choose any name in column REPO_NAME which acts as an internal key for the github repository name.



2. execute method START_BACKUP of tool class, specify the internal repository name you specified in the first step.



Once finished, you could find that all issues are stored into table CRMD_GIT_ISSUE.



3. The body of a given issue could be downloaded locally using method DOWNLOAD_ISSUE, just specify the repository name, issue number and the path of local folder where the github issue file you would like to store:



Since most of my github issue contains Chinese characters, so I use code page 8400 in the code.



here below is the original issue opened in the browser:



Here below is the downloaded issue source code in markdown format:



More features to be added


I will enhance the tool with following features in the future

3.1 issue label information


Multiple labels could be maintained for a single issue and it is very convenient to search by these maintained labels.

Currently the label information is not stored in the database table.



3.2 CDS views for analysis purpose


Since now we have lots of content in our database table, we could develop several CDS view to meet with our specific requirement, for example:


  • calculate the total length of all issues belonging to a given github repository

  • get an overview of all labels and the concrete number of issues assigned with each label


Updated on 2017-07-17


Get an overview of how many issues created for each repository



Solution could be found from CDS view CRMV_GIT_ISSUE.



Get how many issues created per day


Solution could be found from CDS view CRMV_ISSUE_CREATION_DATE_COUNT.



Get an overview of how many images used in a given github issue


Solution could be found from CDS view CRMV_GIT_ISSUE_IMAGE_NUM.





3.3 image backup


Currently only the issue source code with markdown format is extracted and stored in database table in ABAP server. However most of my issues contains screenshot and in the issue source code only a reference to these image files are there - the binary content of image files themselves are not stored in ABAP server. This could be enhanced in the future.



Update on 2017-07-16


Image binary data backup has also been implemented.

Create a new transparent table below to store the image binary data.



Then simply execute this line below:


cl_abap_git_issue_image_tool=>start_backup( 'KM' ).

Once finished, all the pictures used in the issues of repository KM will be stored in that table.

It is good for me to know that two repositories of mine below have totally used 1528 images in their issues:






The source code of the tool could be found from my github.

















Labels in this area
Popular Blog Posts