Give your users the power of dealing with big text files using Agile Data Preparation

Hello!

Most of us have experienced this: We have a big text file (bigger than 500 MB, and with more than 2 millions rows). We want to get some information from it, where do we load it?

Agile Data Preparation!

Make sure your basis team has already set up a data provisioning agent connected to the HANA of Agile Data Preparation.

Ask them to register the adapter FileAdapterDataStore (Note that SDI has another adapter called File, this is a very suitable adapter if you need metadata about files, directories or Pattern-based reading; reading multiple files in a directory that match a user-defined partition, more info here)

In our case, we will use the FileAdapterDataStore as can be seen here:

Image 1: Showing the adapter in HANA Studio

Then your Basis has to tell your which is the folder where you can put your files so that the adapter can see them.

After that, get the data from Agile Data Preparation using the option “Data Source,” check the video for more details:

Be reminded that if your text file is not that big (less than 1GB approx.), you can upload/load it without the of the Data Provisioning Agent! (see the video)

Steps:

One time steps:
Basis team to register the FileAdapterDataStore and give you access to the folder of the adapter (called workspace by default)

On-demand steps as a user:
1) Put the file(s) in the folder of the adapter
2) Load your data into Agile Data Preparation using the instructions from the video.