Skip to Content

Intro

In my post SAP Data Services 4.2 SP4 New Features I highlighted some new functionality introduced in Data Services 4.2 SP04. One of the new functionality was the ability for Data Services to load data into Google Big Query. This blog focuses on how you can do that.

Google Big Query Setup

To get started with Google Big Query you need to

bigQuery.jpg

  • You will then be taken to project overview page. You will see option “Try BigQuery”. Click on it.

BigQueryProj.png

  • You will be taken to a page where you can compose a query. There is sample tables that you can query. As seen below a query has been created against one of the sample tables. Once run it shows the time and mb processed.

GoogleBigQuerySample.jpg

  • We now need to also sort out authentication. Navigate here

BigQueryAuth.jpg

  • Select the following options. When you click “Create Client ID” it will create an authentication file. Save the file, this is in json format, this is not the file we need. If using chrome it will save in downloads automatically.

BigQueryCreateClient.jpg

  • Also on the credentials screen  you will have more info as seen below. Click “Generate new p12 Key”, save this file as we need this when creating the datastore in DS. When you generate the key a popup will been show and will have a pass phrase. Write this pass phrase down. We need this to create the datastore in DS also.

GoogleBigQueryCert.jpg

  • You will also need to setup the trial, you cannot create a table without a trial. DS can only load so we need a table to be created.

BigQueryTrial.jpg

You now have everything you need to create a datastore in data services.

Data Services Google Big Query Datastore

You can now create a datastore in Data Services Designer. The only pars you need to fill in is the parts in the red. All the other parts are there by default. The fields you need to enter come from the Google setup part we did above.

BigQueryDataStore.jpg

You can now load data to Google Big Query.

Hope this helps.

To report this post you need to login first.

3 Comments

You must be Logged on to comment or reply to a post.

  1. Pan Cheng

    Hi,

    I am able to create the connection to Google Cloud. However i am not able to load the data from a table into Google Big query. Its giving exception. Its plain extract from a table into google big query table and its giving exception saying

    job_extract_flat_ger_emp_mgr|DATAFLOW df_extract_emp_mgr_file|STATEMENT LOAD MESSAGE DS_Google_Cloud..emp_mgr INPUT Could not find transform <emp_mgr>. Ensure that it exists.

    (0) 

Leave a Reply