Taking additional file from another folder using File sender adapter

Once ago I worked on the scenario where plain text file had to be taken along with additional file. The main problem was that the additional file was placed in different folder and had naming scheme non-correlating with main file’s name.

Now I would like to demonstrate an approach I used to implement that scenario.

Let’s consider we have the main file containing some transactional data (for scenario simplification I use plain text file) and also we have to pick up additional file with some aggregated data for further processing.

The main files have naming scheme like: data_YYYYMMdd.txt and additional files are placed in subfolders within the source folder with Year/Month hierarchy:

…/2017/01/201701_totals.txt
…/2017/02/201702_totals.txt
…
…/2016/01/201601_totals.txt

and so on.

So, for each of our source files we have to pick additional file using year and month extracted from main file’s name as basis for additional file location and name determination.

The main idea is to use OS script to copy additional file from its original folder to the folder where the main file is placed and rename it accordingly.

Unix OS script written using Bash:

#!/bin/bash

dir="${1%/*}"
fname="${1##*/}"
namepart="${fname%.*}"
datepart="${namepart##*_}"
year="${datepart:0:4}"
month="${datepart:4:2}"
addsrcfile=$dir'/'$year'/'$month'/'$year$month'_totals.txt'
adddstfile=$dir'/'$namepart'_totals.txt'

cp $addsrcfile $adddstfile

This script extracts year and month from source file name, builds the path to additional file and copies that file to main file’s folder.

Key settings for File sender adapter look like:

I’ve placed file data_20170604.txt in the source folder and additional file 201706_totals.txt – to the subfolder: usr/sap/PK5/PI_DATA/Test/2017/06.

Result of processing in the File sender channel:

As we can see, main file is taken, then OS command is executed and after that additional file is picked up by adapter as attachment.

Approach is based on the observation that OS script is executed after picking the main file but before searching for additional file(s).

As the result, we have the message with our main file as payload and additional file as attachment.

Further we can use java mapping, for example, to build desired output payload.

Regards, Evgeniy.