GCP App Engine: Replicating Real-time survey response from Qualtrics to SAP HANA Cloud using App Engine as webhook

In the previous blog post, I have covered on how to use SAP Kyma as a webhook to replicate real time survey responses from Qualtrics to SAP HANA Cloud. In this blog, am going to discuss about using Google Cloud Platform App Engine[GCP App Engine] for python code deployment instead of SAP Kyma. Please review the previous blog to understand the end-to-end scenario.

If you want to understand the use case end-to-end,  I would suggest you to start the discovery mission as mentioned in this blog.

GCP App Engine is a fully managed serverless platform for developing and hosting web applications. You can opt for either App Engine Standard or App Engine Flexible for deployments. Since we already know we are planning to use python based deployment and planning to run for free or at very low cost 😉 , let’s use App Engine Standard.  You can refer this link to know more about the difference in environments

In this blog, am going to cover specifically the steps 1 to 4. The steps include

1. Users completing the Qualtrics Survey & Python Code Validation – These  steps are already covered in the previous blog and make sure you reach out to Mission Experts in case you do not have access to a Qualtrics Instance.  And the steps for validation is exactly the same for App Engine based deployments too.  If you have already validated the code based on the previous blog, you do not have to repeat the same.
2. Connecting to GCP, enabling the App Engine & adjusting the main.py script.
3. Test ,deploy the Application & access deployed URL from App Engine Dashboard
4. Setting up the App Engine URL as webhook in Qualtrics Survey and replicating the response    in SAP HANA Cloud. This Step is already covered in the previous blog. All you have to do is replace the Kyma based URL with App Engine URL.

Here is the video where I have explained the steps in detail

Let’s look into the steps in details. You can refer the repo https://github.com/IDGCOENA/UCR2802_AppEngine.git   to access the python code for validating in Google Colab as well as for the App Engine Deployment. We have added the deployment sequence in the Read.me file for your reference.

Step1: Users completing the Qualtrics Survey

As mentioned, please refer this blog  for step1.

Step2: Connecting to GCP, enabling the App Engine & adjusting the main.py script.

I assume you have already created your GCP trial account. If not please sign up for GCP Free account by looking into the Free Tier offerings  https://cloud.google.com/free/docs/gcp-free-tier. You can complete the free trial by clicking this link https://cloud.google.com/free. Please note that you would need credit card to sign up for free trial and you will receive 90-day, $300 free trial.  And of course this hands on for App Engine Standard environment deployment would not shoot up the price but you can disable the account anytime later as mentioned in this link https://cloud.google.com/billing/docs/how-to/modify-project

Login to your GCP login console using https://console.cloud.google.com/ and use the default project mentioned such as “My First Project”. IN my case the project name is btpadoption.

Or you can click on the default project name and it will open a sub screen where you can create your own project.

Once you are in the your project, enable Cloud Shell by selecting “Activate Cloud shell” on your GCP Home Page

Once you have the cloud shell provisioned, execute the command gcloud app create

If an authorization window appears, click Authorize

Clone the repository from Cloud Shell by executing

git clone https://github.com/IDGCOENA/UCR2802_AppEngine.git

Once you clone the repository, navigate to the folder UCR2802_AppEngine and you will find the following files as shown below. The only file you need to modify is main.py similar to the steps mentioned in the previous blog.

File app.yaml has the python runtime information and name of the service to be created. If this is the first service you are creating in the app engine, then it has to be named default. And the additional services you create  can be named as you like.

Changes in main.py file

1. Provide the Parameters in the lines 18 – 21 [HANA Cloud DB Username, Password , Qualtrics Survey ID , Qualtrics API Token] with your HANA Cloud Trial Logon details . And for Qualtrics tokens and IDs, please reach out to Mission Support.
2. Provide the HANA Cloud host URL at line 98.
3. Provide the Qualtrics Data center at line 192 – Please reach out to Mission Support to request the same.
4. Save the file after updating the necessary credentials and tokens.

There are couple of ways to edit this file.

Option 1:  Edit Locally and import in Cloud Shell

Download the main.py from the Git repository               https://github.com/IDGCOENA/UCR2802_AppEngine.git and store in your local desktop.  Adjust        the files as mentioned above in steps 1-4  and upload the file using cloud shell

  Option 2: Update directly in Cloudshell

Navigate into folder  UCR2802_AppEngine from cloud shell and use VI Editor to edit the file.  To use this option execute vi  main.py

You will be able to view the code directly from command line as show below.

Type i and it will take you to the insert mode as shown below. Now make the adjustments to the code as mentioned in steps 1-4 .

Once you have made the changes, press ESC key and you will be out of edit mode. Now save changes and exit vi by typing:   :wq . You will be back in the folder UCR2802_AppEngine.

Step3: Test ,deploy the Application & access deployed URL from App Engine Dashboard

Now that you have all the necessary files, we can test and deploy the application.

To test the application, we have to execute the following steps:

A. Create a virtual environment for Python.

Execute:  virtualenv –python python3 \~/envs/qreplicate 

qreplicate is the name of the virtual environment. You could have any name for your environment.

 B. Activate the virtual environment

Execute:  source \~/envs/qreplicate/bin/activate

C. Install the requirements from the repository file requirements.txt

    Execute: pip install -r requirements.txt

D. Test the main.py file.  Please make sure you have adjusted the main.py file using one of the               options mentioned previously in this blog.

Execute:  python main.py

Once executed successfully, you should see the logs in the cloud shell as below.

E. Deploy App

Once the test is successful, deploy the app.

Execute:  gcloud app deploy

Once you execute the command, you should see  target service named as “default” as per the app.yaml  and you should see the target URL that will be created for this service.

Once you continue and say yes for this deployment , it will take few minutes for deployment and you should see this message.

Finally, when you refresh you GCP console, you should see the App Engine in Dashboard

When you select the option “Go to the App Engine dashboard”, you will see the service URL.

When you select the service URL, you should see the same message as you noticed while testing the app.

Step4: Setting up the App Engine URL as webhook in Qualtrics Survey and replicating the response in SAP HANA Cloud.

Now that you have the deployed URL from App Engine, please follow the same steps mentioned for Step4 in the previous blog.   You are replacing the URL in the workflow of the survey with the App Engine URL.

Hopefully this helps you in understanding the app engine deployments. For more information about the end-to-end scenarios, please refer the Discovery mission.

I am excited to hear your feedback. Please feel free to comment and also reach out to me if you have issues in accessing the Git Repositories .