SAP Data Services - Running & Scheduling Data Services Jobs from Linux Command Line using Third party Scheduler

[former_member189933]

SAP Data Services

Running & Scheduling Data Services Jobs from Linux Command Line using Third party Scheduler

Document Information
Document Title:	SAP Data Services - Running & Scheduling Data Services Jobs from Linux Command Line
Document Purpose:	The purpose of this document is to provide details on starting Data Services (BODS) Jobs from Linux/Unix Command Line utility and scheduling the Jobs on Unix Platform.
File Name:	SAP BODS - Starting Jobs from Linux Command Line.DOCX
Reference	Admin Guide: http://help.sap.com/businessobject/product_guides/boexir32SP1/en/xi321_ds_admin_en.pdf

Table of Contents

1.             Introduction.
2.             Using a third-party scheduler
3.             To export a job for scheduling.
4.             Setting up a cron job in UNIX-type operating systems.
5.             To execute a job with a third-party scheduler
6.             About the job launcher
7.             Job launcher error codes.

1. Introduction

SAP BODS Jobs can be started & Scheduled from other Operating Systems like Linux, HP-Unix etc in addition to windows using third party utilities.

This Document provides information on running & scheduling the SAP BODS jobs from UNIX command prompt utility (Crontab).

2. Using a third-party scheduler

When you schedule jobs using third-party software:

• The job initiates outside of SAP BusinessObjects Data Services

• The job runs from an executable batch file (or shell script for UNIX) exported from SAP BusinessObjects Data Services.

Note:

When a third-party scheduler invokes a job, the corresponding Job Server must be running.

3. To export a job for scheduling

1. Select Batch > repository.
2. Click the Batch Job Configuration tab.
3. For the batch job to configure, click the Export Execution Command link.
4. On the Export Execution Command page, enter the desired options for the batch job command file that you want the Administrator to create:

Option	Description
File name	The name of the batch file or script containing the job. The third-party scheduler executes this file. The Administrator automatically appends the appropriate extension: • .sh for UNIX • .bat for Windows
System configuration	Select the system configuration to use when executing this job. A system configuration defines a set of Datastore configurations, which define the Datastore connections. If a system configuration is not specified, the software uses the default Datastore configuration for each Datastore. This option is a run-time property. This option is only available if there are system configurations defined in the repository.
Job Server or server	Select the Job Server or a server group to execute this schedule.
group
Enable auditing	Select this option if you want to collect audit statistics for this specific job execution. The option is selected by default.
Disable data validation	Select this option if you do not want to collect data validation statistics for any validation transforms in this job. The option is not selected by default.
statistics collection
Enable recovery	Select this option to enable the automatic recovery feature. When enabled, the software saves the results from completed steps and allows you to resume failed jobs.
Recover from last failed	Select this option to resume a failed job. The software retrieves the results from any steps that were previously executed successfully and re-executes any other steps. This option is a run-time property. This option is not available when a job has not yet been executed or when recovery mode was disabled during the previous run.
Use password file	Select to create or update a password file that automatically updates job schedules after changes in database or repository parameters. Deselect the option to generate the batch file with a hard-coded repository user name and password.
Collect statistics for optimization	Select this option to collect statistics that the
	optimizer will use to choose an optimal cache
	type (in-memory or pageable). This option is
	not selected by default.
Collect statistics for monitoring	Select this option to display cache statistics
	in the Performance Monitor in the Administrator.
	The option is not selected by default.
Use collected statistics	Select this check box if you want the optimizer
	to use the cache statistics collected on a
	previous execution of the job. The option is
	selected by default.
Export Data Quality reports	Generates and exports all specified job reports
	to the location specified in the Management >
	Report Server Configuration node. By default,
	the reports are exported to
	$LINK_DIR\DataQuality\re
	ports\repository\job.
Distribution level	Select the level within a job that you want to
	distribute to multiple Job Servers for processing:
	• Job: The whole job will execute on one
	Job Server.
	• Data flow: Each data flow within the job
	will execute on a separate Job Server.
	• Sub data flow: Each sub data flow (can
	be a separate transform or function) within
	a data flow can execute on a separate Job
	Server.

1. Click Export.

The Administrator creates command files filename.txt (the default for filename is the job name) and a batch file for the job and writes them to the local LINK_DIR\log directory.

(E.g. C:\Program Files\Business Objects\BusinessObjects Data Services\log )

Note:

You can relocate the password file from the LINK_DIR\conf directory, but you must edit the filename.txt file so that it refers to the new location of the password file. Open the file in a text editor and add the relative or absolute file path to the new location of the password file in the argument –R "repositoryname.txt".

If you are Exporting the Job for Unix Environment append “.sh” extension to the File Name

4. Setting up a cron job in UNIX-type operating systems

Cron jobs can be used for setting up scheduled Job runs in UNIX-type operating systems, e.g. UNIX, Linux, FreeBSD and Darwin (Mac OS X).

Steps:

Note: The syntax may differ, depending on which version of cron is present on your computer.

Open a root shell and type the following:
crontab -u root -e
to open a VI style editor.  Press 'i' to insert text.

The crontab comprises five entries indicating the schedule time, and also the name and path of the program to be run. Use a space or a tab between each entry:

minute(0-59) hour(0-23) day_of_month(1-31) month(1-12) day_of_week(0-7) /path/Job_BODSJobNM.sh

NOTE: “Job_BODSJobNM.sh is the name of the .sh file Exported form BODS Administrator”

You can replace a field value with "*". So:
“0 10 * * * /path/script.sh” is the same as “0 10 1-31 1-12 0-7 /path/script.sh”
The Job would run at 10 each morning.

For example, to run a BODS Job at 8 o'clock each night in Linux, use:
0 20 * * * /usr/local/bin/directory/BODS

Names can be used (e.g. March) for month and day of week. In day of week, Sunday can be 0 or 7.

When you have created the crontab, press 'Escape' to leave insert mode.

Type 'ZZ' (upper case 'z' twice). A message similar to the following should be displayed:
/crontab.zUcAAFwPVp: 1 lines, 24 characters
crontab: installing new crontab

The crontab has now been set up. Cron will automatically send an email to root to confirm that the scan has run. For more information on cron and crontab, read the relevant manual pages. For example, type:

man crontab

to see the crontab manpage.

5.To execute a job with a third-party scheduler

1. Export the job's execution command to an executable batch file (.bat file for Windows or .sh file for UNIX environments).
2. Ensure that the Data Services Service is running (for that job's Job Server) when the job begins to execute.

The Data Services Service automatically starts the Job Server when you restart the computer on which you installed the Job Server.

• You can also verify whether a Job Server is running at any given time using the Designer. Log in to the repository that contains your job and view the Designer's status bar to verify that the Job Server connected to this repository is running.

• You can verify whether all Job Servers in a server group are running using the Administrator. In the navigation tree select Server Groups

> All Server Groups to view the status of server groups and the Job Servers they contain.

1. Schedule the batch file from the third-party software.

Note:

To stop an SAP BusinessObjects Data Services job launched by a third-party scheduling application, press CTRL+C on the application's keyboard.

6.About the job launcher

SAP BusinessObjects Data Services exports job execution command files as batch files on Windows or CRON files on UNIX. These files pass parameters and call AL_RWJobLauncher. Then, AL_RWJobLauncher executes the job, sends it to the appropriate Job Server, and waits for the job to complete.

Caution:

Do not modify the exported file without assistance from SAP Technical Customer Assurance.

The following shows a sample Windows NT batch file created when the software exports a job. ROBOT is the host name of the Job Server computer.

All lines after inet:ROBOT:3513 are AL_Engine arguments, not AL_RWJobLauncher arguments.

D:\Data Services\bin\AL_RWJobLauncher.exe

"inet:ROBOT:3513"

"-SrepositoryServer

-Uusername

-Ppassword

-G"b5751907_96c4_42be_a3b5_0aff44b8afc5"

-r100 -T14

-CTBatch -CmROBOT -CaROBOT

-CjROBOT -Cp3513"

Flag	Value
-w	The job launcher starts the job(s) and then waits before passing back the job status. If -w is not specified, the launcher exits immediately after starting a job.
-t	The time, in milliseconds, that the Job Server waits before checking a job's status. This is a companion argument for -w.
-s	Status or return code. 0 indicates successful completion, non-zero indicates an error condition. Combine -w, -t, and -s to execute the job, wait for completion, and return the status.
-C	Name of the engine command file (path to a file which contains the Command line arguments to be sent to the engine).
-v	Prints AL_RWJobLauncher version number.
-S	Lists the server group and Job Servers that it contains using the following syntax: "SvrGroupName;inet:JobSvr1Name:Job Svr1Host:JobSvr1Port;inet:JobSvr2Name:Job Svr2Host:JobSvr2Port"; For example: "SG_DEV;inet:HPSVR1:3500;in et:WINSVR4:3505";
-R	The location and name of the password file. Replaces the hard-coded repository connection values for -S, -N, -U, -P.
-xCR	Generates and exports all specified job reports to the location specified in the Management > Report Server Configuration node. By default, the reports are exported to $LINK_DIR\DataQuality\reports\repository\job. In order to use this flag, you must disable the security for the Export_DQReport operation in the Administrator > Web Services > Web Services Configuration tab.

There are two arguments that do not use flags:

• inet address: The host name and port number of the Job Server. The string must be in quotes.

For example:

"inet:HPSVR1:3500"

If you use a server group, inet addresses are automatically rewritten using the -S flag arguments. On execution, the first Job Server in the group checks with the others and the Job Server with the lightest load executes the job.

• server log path: The fully qualified path to the location of the log files. The server log path must be in quotes. The server log path argument does not appear on an exported batch job launch command file. It appears only when SAP BusinessObjects Data Services generates a file for an active job schedule and stores it in the following directory:

LINK_DIR/Log/JobServerName/RepositoryName/JobInstance

1. Name.

You cannot manually edit server log paths.

7. Job launcher error codes

The job launcher also provides error codes to help debug potential problems.

The error messages are:

Error number	Error message
180002	Network failure.
180003	The service that will run the schedule has not started.
180004	LINK_DIR is not defined.
180005	The trace message file could not be created.
180006	The error message file could not be created.
180007	The GUID could not be found. The status cannot be returned.
180008	No command line arguments were found.
180009	Invalid command line syntax.
180010	Cannot open the command file.