Skip to Content


If you are an SAP Employee, please follow us on Jam.

This one goes out to my friend and colleague Aaron Williams who gave the idea for this 😉

For this one, we’re going to use Python and some nice libraries…


pip install pandas #For Data Analysis and Data Frames

pip install mechanize #Headless Web Browser

pip install beatifulsoup4 #For Web Scrapping

Of course…we’re going to use some Regular Expressions as well…but that’s already included in Python 🙂

So, the basic idea is that we need to log into SCN using our Username and Password and then read the first page of our “content” folder only for blog posts…then we can continue reading the following pages by using a parameter that will load the next 20 blogs…

Now…and before you say something 😛 This works (at least for me) only for the first 10 pages…because after that the HTML seems to be automatically generated…so there’s nothing to get more data from 🙁 or maybe my blogs come from a long time ago…my first blog ever on SCN was written on February 17, 2006 Tasting the mix of PHP and SAP

Anyway…let’s see the source code 😀

#coding= utf8

USR = ‘YourUserName’

PWD = ‘YourPassword’

import sys

import re

import mechanize

from BeautifulSoup import BeautifulSoup

import pandas as pd



cookies = mechanize.CookieJar()

br = mechanize.Browser()



res =““)


br[“j_username”] = USR

br[“j_password”] = PWD



res = br.submit()

result =

author =“username: \’.*”,result)

author = re.sub(‘username: \’|\’|\,’,”,

displayname =“displayName: \’.*”,result)

displayname = re.sub(‘displayName: \’|\’|\,’,”,

j = 0

df = pd.DataFrame()



  link = ““\

        “contentstatus[published]~objecttype~objecttype[blogpost]” %(author)


  link = ““\

  “contentstatus[published]~objecttype~objecttype[blogpost]&start=%j” %(author,str(j))

  j += 20

  res =

  Titles = []

  Likes = []

  Bookmarks = []

  Comments = []

  Views = []

  soup = BeautifulSoup(

  list_items = [list_item for list_item in soup.findAll(‘td’,{‘class’:’j-td-title’})]

  if(len(list_items) == 0):


  for i in range(0, len(list_items)):

  title =‘[^<>]+(?=<)’,str(list_items[i]))


  list_items = [list_item for list_item in soup.findAll(‘a’,{‘class’:’j-meta-number’})]

  for i in range(0, len(list_items), 2):

  like = re.sub(‘<[^>]+>|in.*’,”,str(list_items[i]))

  bookmark = re.sub(‘<[^>]+>|in.*’,”,str(list_items[i+1]))



  list_items = [list_item for list_item in soup.findAll(‘span’,{‘title’:’Replies’})]

  for i in range(0, len(list_items)):

  comment = re.sub(‘<[^>]+>|in.*’,”,str(list_items[i]))


  list_items = [list_item for list_item in soup.findAll(‘span’,{‘title’:’Views’})]

  for i in range(0, len(list_items)):

  views = re.sub(‘<[^>]+>|in.*’,”,str(list_items[i]))


  for i in range(0, len(Titles)):

  df = df.append({‘Title’: Titles[i], ‘Likes’: Likes[i], ‘Bookmarks’: Bookmarks[i],

                 ‘Comments’: Comments[i], ‘Views’: Views[i]}, ignore_index=True)



print(“Welcome ” + displayname + “\n”)

sum_row = df[[“Views”]].sum()

print(“Total number of Views” + ” ==> ” + str(sum_row.values[0]))

sum_row = df[[“Comments”]].sum()

print(“Total number of Comments” + ” ==> ” + str(sum_row.values[0]))

sum_row = df[[“Bookmarks”]].sum()

print(“Total number of Bookmarks” + ” ==> ” + str(sum_row.values[0]))

sum_row = df[[“Likes”]].sum()

print(“Total number of Likes” + ” ==> ” + str(sum_row.values[0]))

print(“\nTop 3 Blogs with most Views”)


df = df.sort_values(by=[‘Views’],ascending=[False])

for i in range(0, 3):

  print(df.iloc[[i]][‘Title’].values[0] + ” ==> ” + str(df.iloc[[i]][‘Views’].values[0]))

print(“\nTop 3 Blogs with most Comments”)


df = df.sort_values(by=[‘Comments’],ascending=[False])

for i in range(0, 3):

  print(df.iloc[[i]][‘Title’].values[0] + ” ==> ” + str(df.iloc[[i]][‘Comments’].values[0]))

print(“\nTop 3 Blogs with most Bookmarks”)


df = df.sort_values(by=[‘Bookmarks’],ascending=[False])

for i in range(0, 3):

  print(df.iloc[[i]][‘Title’].values[0] + ” ==> ” + str(df.iloc[[i]][‘Bookmarks’].values[0]))

print(“\nTop 3 Blogs with most Bookmarks”)


df = df.sort_values(by=[‘Likes’],ascending=[False])

for i in range(0, 3):

  print(df.iloc[[i]][‘Title’].values[0] + ” ==> ” + str(df.iloc[[i]][‘Likes’].values[0]))

If we run this code, then we’re going to have a nice report like this one 😉

SCN_Vanity_App.jpgOf course…it would look better with a nicer UI…but that’s not my forte 🙁 So…if anyone wants to pick the project and improve it…I would really appreciate it 😉

To report this post you need to login first.

Be the first to leave a comment

You must be Logged on to comment or reply to a post.

Leave a Reply