Using the Playback tool in Studio to simulate a live data source

The playback tool in the Streaming Run/Test perspective of (of HANA Studio with streaming plugin installed, or in ESP Studio) is a really useful tool for testing (and of course I use it for demo’s).  It reads input data from a file and can do it at a controlled rate where you can set the rate.  This lets it simulate one or more live sources (see this blog for a discussion of simulation options).

There are two ways to use the playback tool:

1. You can use the “record” feature to record data coming in from a real source – and then playback that file.  When you play it back you can play it back at the original rate or you can speed it up or slow it down
2. You can playback data from a file that you create – either in csv or xml format

The playback tool is – by default – one of the tabs in the lower left pane of the Run/Test perspective, and when you go to choose the file to playback,  you’ll see a button to choose which file type (.rec is the type for files created by using the playback tool to record a live source).

I normally use the .csv file, since it’s easy to create this file with Excel (just remember to save it in csv formal and not xls).  The important thing is to get the format right:

• Each row in the file is one input event
• Each rown must start with the following two columns:
  • The first column must contain the name of the input stream/window where this row will be sent.  This allows you to mix input to multiple input streams/windows in a single input file
  • The second column must contain the opCode.  This will normally be “i” for “insert”, but could also be:  u (update), d (delete), p (upsert)
  • Then there needs to be a column corresponding to each column in the input stream/window. Non-key columns can be null (in a csv file this is just ,,) but the number of columns must match the schema of the input stream/window, and they need to be in the order defined in the input stream/window.

So here’s an example of two rows for my input file, along with the ccl for the input stream it will feed. Note that the second row contains null values in the second to the last column:

MACHINEDATA,i,2DDDBW3TP,2014-06-03T12:00:00.000,TEMP,InsideTemp,45

MACHINEDATA,i,EKM49RTXK,2014-06-03T12:00:01.000,DOOR,,Door Close

CREATE INPUT STREAM MACHINEDATA

SCHEMA (

     MACHINEID string ,

     EVENT_TIME timestamp ,

     EVENT_NAME string ,

     EVENT_DESCRIPTION string ,

     EVENT_VALUE string ) ;

So to use the Playback tool, first create (or record) your data file

Then, in HANA Studio (or ESP Studio), go to the Streaming Run Test Perpective and find the Playback-Record tool (by default in the lower left pane)

Now:

1. Connect the tool to the project you want to send data tool
2. Choose your input file.  – be sure to set the filter to csv, xml, or rec as appropriate
3. Set the playback speed. If you don’t it will play back really, really fast.  Note that the speed is in rows per millisecond.  I usually use 0.01 for demos, but you may want to go slower to watch what’s happening.
4. Then click the “Play” button and watch the data flow in