Skip to Content

We wanted to use a Unix mount as a long term repository having multiple layers of directories and sub-directories structure, Most of the directories with different retention days. We wanted to have a script for cleanup the old files which should take care of the complex retention periods and those distinct retention periods should not be hardcoded in the script because next time if someone wants to modify the retention period or create a new sub directory with different retention period he just need to add the information in a separate file, a config file, thats it! and the code should be dynamic enough to adapt it in next execution.

To understand it, lets have a look at below directory structure then we will look at the configuration file which will store the path of various directories and retention periods in unit of days.

DirStruct_1.jpg

The configuration file which stores the directory name with full path, space or tab then retention period in unit of days.

clScr.conf      {Configuration File}

/temp/zTmpPlGrnd1/C12/C12B/C12B4/C12B4A 40

/temp/zTmpPlGrnd1/A12/A12C              30

/temp/zTmpPlGrnd1/C12/C12B/C12B5        90

/temp/zTmpPlGrnd1/A12                   15

/temp/zTmpPlGrnd1/A12/A12B              62

/temp/zTmpPlGrnd1/A12/A12C/A12C4        13

With a overall retention period of 7 days and above configuration file retention periods for sub-directories the diagram will be like below:

DirStruct_2.jpg

But its not complete, To get the actual view, applying the logic that the sub-directory will have the same retention period as its parent directory unless it is not mentioned in configuration file. Here is the final view  🙂

DirStruct_3.jpg

ha ha ha….. I know …. it is ….
Below is the story needs to be told to the Operating system to achieve above scenario. So you will be needing two files at your OS level, one, this script file with execution permission and second, above file named “clScr.conf” with read permission. One can always modify the duration of retention period and add or remove lines from the file. You don’t have to worry about the number of blank spaces between directory name and digits plus you don’t have to keep them sorted.

We are initially taking maximum retention period as 07 days and our target is to reach the free space of the file system mount greater than or equal to 50%.

In order to achive it we are decreasing the retention period from 07 to 02 days at max [You need to decide these figures as per your requirement they are already highlighted in colors in below story]

The retention period mentioned in the configuration file will not be compromised through out the script. At the end, if this entire story will not be able to free space equal to 50% or more by the deletion activity then it will send a mail for further manual consideration.

Script

MAXDAY=7

THRESHOLD=”50

ZMOUNT=”/temp”

BASEDIR=”${ZMOUNT}/zTmpPlGrnd1″

CFGFILE=”${ZMOUNT}/zTmpPlGrnd0/reaal/clScr.conf”

INTERIM=”${ZMOUNT}/zTmpPlGrnd0/reaal/zintrm_”

LOGFILE=”${ZMOUNT}/zTmpPlGrnd0/reaal/CleanSPMout.log”

DELFILE=”${ZMOUNT}/zTmpPlGrnd0/reaal/DeletedFile.log”

rm ${ZMOUNT}/zTmpPlGrnd0/reaal/*.3nt

echo “Searching for Configuation file…”

echo “”

  if [ -s ${CFGFILE} ]; then

     echo “Configuration File found.”

  else

     echo “Configuration File Not found. Script is terminating!”

     echo “”

     echo “Hint: Please look for ${CFGFILE} file and start the script again.”

     exit 0

   fi

cat ${CFGFILE} | awk ‘{print $1}’ | sort    > ${INTERIM}CfgSrtd.3nt

cat ${CFGFILE} | awk ‘{print $1}’ | sort -u > ${INTERIM}CfgSrtdUnq.3nt

echo “”

echo “Scanning the entries in the configuration file…”

echo “”

z1Dupe=`sdiff ${INTERIM}CfgSrtd.3nt ${INTERIM}CfgSrtdUnq.3nt | grep \< | awk ‘{print $1}’`

  if [ “${z1Dupe}” == “” ]; then

    echo “The entries in the configuration file are consistent.”

    echo “”

  else

    echo “Duplicate entry found in the Configuration file for : $z1Dupe”

    echo “Please clean it up and rerun this script.”

    echo “”

    exit 0

  fi

find ${BASEDIR} -type d | sort -u > ${INTERIM}MasterList.3nt

for iDir in `cat ${INTERIM}MasterList.3nt`

   do

       echo “${iDir}   ${MAXDAY}_3vYwj1n9Tc5D” >> ${INTERIM}WorkList.3nt

   done

cat ${CFGFILE} | sort -u > ${INTERIM}SPLLIST.3nt

cat ${INTERIM}SPLLIST.3nt | awk ‘{print $2}’ > ${INTERIM}SPLDAYS.3nt

cat ${INTERIM}WorkList.3nt | awk ‘{print $2}’ > ${INTERIM}WLDAYS.3nt

n=1

for iDir in `cat ${INTERIM}CfgSrtdUnq.3nt`

   do

      splMaxDay=`head -${n} ${INTERIM}SPLDAYS.3nt | tail -1`

      m=1

      for iMstrDir in `cat ${INTERIM}MasterList.3nt`

         do

           CheckFamily=`echo ${iMstrDir} | grep ${iDir}`

           WLMaxDay=`head -${m} ${INTERIM}WLDAYS.3nt | tail -1`

           if [ “${CheckFamily}” == “” ]; then

              echo “${iMstrDir}   ${WLMaxDay}” >> ${INTERIM}WL2.3nt

           else

              echo “${iMstrDir}   ${splMaxDay}” >> ${INTERIM}WL2.3nt

           fi

           m=$((m+1))

         done

      mv ${INTERIM}WL2.3nt  ${INTERIM}WorkList.3nt

      cat ${INTERIM}WorkList.3nt | awk ‘{print $2}’ > ${INTERIM}WLDAYS.3nt

      n=$((n+1))

   done

cat ${INTERIM}WorkList.3nt | grep _3vYwj1n9Tc5D | awk ‘{print $1}’ > ${INTERIM}CmnList.3nt

while [[ $MAXDAY -gt 2 ]];

do

  USAGE=`df -k | grep ${ZMOUNT} | awk ‘{print $4}’ |tr -d “%”`

  echo “Current FS usage is: ${USAGE}” >> $LOGFILE

  if(( ${USAGE} < ${THRESHOLD} )); then

     echo “Cleanup job finished for comman Directories.” >> $LOGFILE

     exit 0

  else

     echo “Clean up files older than $MAXDAY days” >> $LOGFILE

     for cmnDir in `cat ${INTERIM}CmnList.3nt`

        do

          find ${cmnDir} -maxdepth 1 -mtime +$MAXDAY -print >> ${DELFILE}

          find ${cmnDir} -maxdepth 1 -mtime +$MAXDAY -exec rm {} \;

        done

     MAXDAY=$((MAXDAY – 1))

  fi

done

cat ${INTERIM}WorkList.3nt | grep -v _3vYwj1n9Tc5D | awk ‘{print $1}’ > ${INTERIM}SplList.3nt

cat ${INTERIM}WorkList.3nt | grep -v _3vYwj1n9Tc5D | awk ‘{print $2}’ > ${INTERIM}SplDays.3nt

n=1

for splDir in `cat ${INTERIM}SplList.3nt`

   do

      splMaxDay=`head -${n} ${INTERIM}SplDays.3nt | tail -1`

      find ${splDir} -maxdepth 1 -mtime +$splMaxDay -print >> ${DELFILE}

      find ${splDir} -maxdepth 1 -mtime +$splMaxDay -exec rm {} \;

      n=$((n+1))

   done

USAGE=`df -k | grep ${ZMOUNT} | awk ‘{print $4}’ |tr -d “%”`

echo “Current FS usage is: ${USAGE}” >> $LOGFILE

echo “Cleanup job finished.” >> $LOGFILE

if (( ${USAGE} > ${THRESHOLD} )); then

     echo “Mount ${ZMOUNT} is ${USAGE}% full after cleanup on “`hostname`” — “`date`| mail -s “${ZMOUNT} space alert” user@mailserver.com

fi

exit 0

rm ${ZMOUNT}/zTmpPlGrnd0/reaal/*.3nt

I’ll appreciate your suggestions/comments, and try to answer your queries. Peace!

LineSeparator.jpg

My other Blogs, if you have time…

What’s new in SAP NetWeaver 7.3 – A Basis perspective Part-I

What’s new in SAP NetWeaver 7.3 – A Basis perspective Part-II

Bye bye STRUSTSSO2: New Central Certificate Administration NW7.3

Escaping tough moments of SPAM or SAINT

SAP Software Provisioning Manager : with screenshots

Multiple/Bulk transports with tp script for Unix (AIX, Solaris, HP-UX, Linux)

Holistic Basis View: BusinessObjects BI 4.0 SP 2 Installation & Configuration

How to Rename the Oracle Listener & Change Listener port for SAP

OSS1 & RFC connections SAPOSS, SAPNET_RFC, SDCC_OSS

Start/Stop SAP along with your Unix Server Start/Stop

Interrelation: SAP work process, OPS$ mechanism, oracle client & oracle shadow process

Install and configure NetWeaver PI 7.3 Decentralize Adapter part-1

Install and configure NetWeaver PI 7.3 Decentralize Adapter part-2

Holistic Basis View: BusinessObjects BI 4.0 SP 2 Installation & Configuration

List of Newly added/converted Dynamic parameter in NetWeaver 7.3

Sunset for ops$ mechanism: No more supported by Oracle & Not Used by SAP

Essential Basis for SAP (ABAP, BW, Functional) Consultants Part-I

Essential Basis for SAP (ABAP, BW, Functional) Consultants Part-II

Essential Basis for SAP (ABAP, BW, Functional) Consultants Part-III

To report this post you need to login first.

Be the first to leave a comment

You must be Logged on to comment or reply to a post.

Leave a Reply