Skip to Content

Echo.jpg

Alexa meets Eva DIY


Steve Lucas opened TechEd Las Vegas with his visionary executive keynote focused on the digital era of new generation enterprise computing. One of the live demos on stage presented the concept hands free interaction with Enterprise systems.

/wp-content/uploads/2015/11/alexaeva_830908.png

The following step by step guide details a technical integration of the SAP Concur Travel Service with Amazon Echo Voice Enabled Device for Business by using SAP HANA Cloud Platform and SAP Cloud For Customer, Evature Expert Virtual Agent (EVA) and Prontoly proximity-based authentication technology. The integration has been implemented by SAP Innovation Centers Network, in collaboration with SAP C4C team.


***More details in the TechEdLive Experts Studio interview:


THE CHALLENGE

Simplifying the customer experience for travel booking, implementing “NO UI” approach.

THE USE CASE

Booking a flight and hotel from the company’s travel agency instantaneously, where the flight is on specific dates and the hotel should be close to our business destination which could be a customer’s office, a corporate branch or a prominent place e.g. the Eiffel tour.


THE SOLUTION

Voice enabled device connected to business cloud services with travel booking solution. Where:

  • Voice enabled device is Amazon Echo Voice Enabled Device for Business;
  • Business cloud services is SAP HANA Cloud Platform for business via SAP Cloud For Customer;
  • Travel Booking is SAP Concur;
  • User authorization is provided leveraging Prontoly ultrasonic proximity-based authentication signature;
  • Natural Language understanding is provided by the Evature Expert Virtual Agent (EVA).

THE FLOW

  1. User is authorized to access SAP services by activating Alexa: “Alexa start SAP”

The hands-free authentication process takes place in the background ultrasonically via microphones and speakers that each device (mobile phone and Amazon Echo) is already equipped with. Sound, imperceptible by the human ear, is played by the authenticating device (Amazon Echo) and is heard by the mobile device equipped with Prontoly. A special handshake between the mobile device and the login page (SAP HANA Cloud Platform Application) occurs through Prontoly’s Sound Tokenization technology and then the device equipped with Prontoly sends the verification to the application requesting authentication. During the authentication process, no passwords or user sensitive data are transmitted over the sound so it is impossible for anything to be sniffed or reproduced.

  1. User books travel from SAP Concur “Alexa ask SAP to book me a flight from X to Y”

    II.    Unstructured text is passed from Amazon Echo (Alexa) to Evature’s Expert Virtual Agent (EVA) for Natural Language understanding. Eva’s dialog engine completes missing information by conversing with the user and when sufficient information is gathered, a structured query is delivered to the Concur API via HANA Cloud Platform Application. Concur contacts Booking.com and GDS to receive the alternatives and returns the answer via API. The answer is being played to the user by Amazon Echo (Alexa). The user selects the desired flight and asks for a hotel. He doesn’t need to specify the date or the location, since Eva infers the information from previous utterances in the conversation already stored those in the complex object in the application on SAP HANA Cloud Platform.


Technical Implementation:

  1. Download the authentication sound file from Prontoly website and upload it to your Amazon account Link (Prontoly to provide the file generation service);
  2. Download and install the Prontoly authentication application to your mobile device Link (Prontoly to provide the app generation service);
  3. Book SAP HANA Platform Trial instance;
  4. Deploy the Travel Booking Java Application (Appendix I);
  5. Install Amazon Alexa Web Plugin or use the real Echo device (see development guide in Appendix II);
  6. Acquire access to the Evature Sandbox on AWS Link (register for a free Evature account at https://www.evature.com/registration/form);
  7. To further extend or change your HCP application, use the following Concur APIs  Link (you can access the link to the original article on Evernote or follow the steps in Appendix III);
  8. The explanation on the usage of SAP Cloud For Customer (C4C) is in Appendix IV.

Here is the video:

Now you are ready to go. Use the voice commands below to start your journey:

“Alexa start sap”

“Alexa ask sap to book me a flight from boston to chicago”

“Alexa play log in”  // Open Prontoly App Is needed

“Alexa ask sap to book me a flight from boston to chicago”

“November first”

“Alexa ask sap to book me a hotel in chicago” // might have time outs, just restart from the hotel booking

“November first “

“Three nights”

Extra commands:

Alexa ask sap clear session // if stuck or repeating questions many times

Alexa exit // If EVA is stuck on a question

Alexa ask sap to log out // clean session and to re-login with prontoly

THE BENEFITS

By combining Travel NLP by Evature, Concur by SAP and Ultrasonic Auth by Prontoly, we are getting natural, fast, secure and simple interaction between businesses and their consumers.


Appendix I

Deployment of the Travel App to SAP HCP.

Once you are in your HCP cockpit you will see:

Proceed to ‘Java Applications’

Click on ’Deploy’ link and a new widow will pop up

Specify the location of the WAR file with your skill, Application name, and press deploy. After the deployment will be completed you can immediately start your application.

Once application has started you will see the link where it can be accessed

We will need this link later for configuration the Amazon Echo skill.

Appendix II

  1. Implementing the skill

You can read an article about how to create amazon Alexa skill kit here: https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/getting-started-guide

Evature provides us a powerful NLP engine, so we just send them a phrase we have received from user, and Evature will send us parsed structured travel data.

import com.amazon.speech.speechlet.SpeechletResponse
import
groovy.json.JsonSlurper
import org.slf4j.Logger
import org.slf4j.LoggerFactory

import static groovyx.net.http.ContentType.JSON
import static groovyx.net.http.Method.GET
import static saplab.ConcurProxyService.*
import static saplab.SpeechletUtils.buildSpeechletResponse
import static saplab.Utils.createHttpBuilder

class EvaProxyService {

def static url = http://freeapi.evature.com/v1.0

private static final Logger LOGGER = LoggerFactory.getLogger(EvaProxyService.class);

static SpeechletResponse getEvaResponse(String inputText, SapSession session, String intent) {
def http = createHttpBuilder(url)

def sessionId = session.evaSessionId ?: ‘1’

LOGGER.debug(“Sending request to Evature $inputText)

http.request(GET) {
headers.Accept = JSON.acceptHeader
uri.query = [
site_code : ‘your_site_code’, api_key: ‘your_api_key’,
input_text: inputText, session_id: sessionId, from_speech: , ffi_statement: , ffi_chains: , uuid: session.id]
response.success = { response ->

def json = response.entity.content.text
LOGGER.debug(
“Received response from evature ${json})
def reply = new JsonSlurper().parseText(json)

def indexesToSkip = []
def replies = []
def title
def reprompt
session.evaSessionId = reply.session_id
// acquire authorization token from Concur
def token = getConcurToken()
def replyReady = false

// parsing the response from Evature
def flows = reply?.api_reply?.Flow
def flightFlows = flows?.findAll {it.Type == “Flight”}?.size()
def isFlightContext = flightFlows > session.flightFlows

def flowsToProcess = isFlightContext ? flows.findAll {it.Type == “Flight” || it.Type == “Question”} : flows.findAll {it.Type == “Hotel” || it.Type == “Question”}

session.flightFlows = flightFlows

LOGGER.debug(“New flow is ${flowsToProcess?.toListString()})

  flowsToProcess?.reverse()?.eachWithIndex { element, index ->
if (replyReady) return
if
(index in indexesToSkip) return

if (element.ReturnTrip) {
// overriding SayIt with ReturnTrip SayIt (from devs)
replies << element.ReturnTrip.SayIt
indexesToSkip << element.ReturnTrip.ActionIndex
}
else {
replies << element.SayIt
}

// if we have a question to user we should stop processing and ask for additional info
if (element.Type == ‘Question’) {
reprompt = element.SayIt
title = element.QuestionCategory ?:

replyReady = true
}

if (element.Type == “Flight”) {
// asking concur for a flight
def flight = searchForFlight(token, reply.api_reply, element)
LOGGER.trace(
“Flight search result is $flight)
replies = [flight]
replyReady =
true
session.flightFlows = 0
session.evaSessionId = 0
}

if (element.Type == “Hotel”) {
// asking concur for hotel
def hotels = searchForHotels(token, reply.api_reply, element)
LOGGER.trace(
“Flight search result is $hotels)
replies = [hotels]
replyReady =
true
session.flightFlows = 0
session.evaSessionId = 0
}
}
def output = replies.join(‘. ‘)
reprompt = reprompt ?: output
  buildSpeechletResponse(intent, output,
false)
}

response.failure = { response ->
LOGGER.error(“Received an error from eva ${response.entity.content.text})
buildSpeechletResponse(intent,
“Error connecting Eva service”, true)
}
}
as SpeechletResponse
}
}

  1. Registering the skill at amazon

Go to the amazon developer portal https://developer.amazon.com/edw/home.html#/

To the Alexa section

Go to the ‘Alexa Skills Kit’, now we need to fill up the skill information

Provide your skill name and the invocation name.  Put your link to the skill from the HCP in the endpoint field.

Click Save, and proceed to Interaction model.

Fill the utterances (the key phrases Alexa will listen for), and intent schema (type of the data and intent names are specified here)

See https://developer.amazon.com/appsandservices/solutions/alexa/alexa-skills-kit/docs/defining-the-voice-interface for more information

Add SSL option on ’SSL Certificate’ page

And you can test your skill on the ‘Test’  page.

We will have all the text being passed to Evature for processing.

You can find the project here:

https://www.dropbox.com/s/po60vx2999lb866/alexaWs.zip?dl=0


Appendix III

There is an example on how Concur and the Echo can be used together with Concur public API (article written by Chris Trudeau):

The Amazon Echo skill kit example makes it easy to build Echo apps that call cloud APIs. In this post, we’ll touch on the Concur JS SDK and how to integrate it with the Amazon Echo skill kit. Check out the github repo for an example of the code talked about here.

Example Usage

Alexa can get the next trip for your Concur user. With that you could add functionality to get the next city you are traveling to or any data related to your trips in Concur.

User: “Alexa, ask Concur when is my next trip?”
Alexa: “Your upcoming Concur trip is <TripName>”

Development

If you are familiar with NodeJS then the development is fairly straight forward after checking out invocations and intents. A user calls intents with their voice and invocations are the name that identifies functional that the user wants. Creating an IntentSchema for our skill in our example above and our SampleUtterances. Then using the Concur SDK you can retrieve a list of trips for a specific user like below. Then you can create new intents to get more functionality such as the hotel for the next trip, airline the user is flying or any other information located via the Concur Itinerary API.

//This will contain a list of Itineraries
var options = {
oauthToken:oauthToken
};

concur.itinerary.get(options)
.then(function(data) {
// Data will contain the Itinerary
})
.fail(function(error) {
  // Error will contain the error returned.

});

Conclusion

I’ve enjoyed working with the Amazon Echo. I use Echo at home to control my lights and thermostat and even as a radio. One issue with this integration is it does not support OAuth which means that it will only work for a single user token.


Appendix IV

SAP Cloud For Customer Integration

  1. You should have access to the OData service of your C4C tenant:
    https://<your_c4c_tenant>/sap/byd/odata/v1/c4codata/$metadata
  2. You can browse the available object collections and their data types. We will be using the AccountCollection:

https://<your_c4c_tenant>/sap/byd/odata/v1/c4codata/AccountCollection

  1. Import the OData helper classes from our sample code. They depend on Google Gson and the fluent Apache HTTP Client (fluent-hc)
  2. Using the helper classes, you can conveniently connect to C4C and query the account collection
String odataUrl = "https://<your_c4c_tenant>/sap/byd/odata/v1/c4codata/";
String c4cUser =
"User";
String c4cPassword =
"Password";

ODataClient odata =
new ODataClient(odataUrl, c4cUser, c4cPassword);

// Load 10 accounts
Account[] accounts = odata.loadItems(Account.class, "AccountCollection", 10);
for (Account account : accounts) {
System.
out.println(account.AccountName);
}
 
  1. You can use this data to answer the Alexa Questions:

public ServletResponse processAccountOwnerIntent(String accountNameSlotValue) {
ODataClient odata =
/* … */;
Account[] accounts = odata.loadItems(Account.
class, “AccountCollection”, 1, String.format(“substringof(‘%1$s’, AccountName)”, accountNameSlotValue));
if (accounts != null && accounts.length > 0) {
return buildSpeechletResponse(“AccountOwnerIntent”, “The account owner is ” + accounts[0].OwnerFormattedName, false);
}
else {
return buildSpeechletResponse(“AccountOwnerIntent”, “Sorry, I couldn’t find an account with that name”, false);
}
}

You can download the sample project from here:

https://www.dropbox.com/s/wexzt0to4l4c7mx/alexa-c4c-sample.zip?dl=0

ABOUT

***About SAP Innovation Centers Network

SAP Innovation Centers Network is the innovation gateway to SAP technologies.

With a generously equipped integration lab, cutting edge technologies in

fields of Machine Learning, distributed computing storage, networking and hardware accelerators, available to its partners, ICN is responsible for the constant growth of SAP eco system. Our team enables our partners to extend their offering by co-development of software strategy, architecture, and design. We provide deep expertise in software development, prototyping, and partner frameworks, with ICN staff function as an intelligent interface supporting your project.

Contacts for this PoC:

michael.kemelmakher@sap.com

b.mikhailovski@sap.com

leonid.bobovich@sap.com

inna.tokarev.sela@sap.com



***SAP Cloud For Customer /wp-content/uploads/2015/11/c4c_833625.jpg

This cloud CRM portfolio brings marketing, sales, commerse, customer service, and social CRM together- to help your team form powerful personal connections that drive customer engagement across all channels.

Contacts for this PoC:

oliver.conze@sap.com



***About Concur

Concur (an SAP company)  is the leading provider of spend management solutions and services in the world, helping companies of all sizes transform the way they manage spend so they can focus on what matters most. Through Concur’s open platform, the entire travel and expense ecosystem of customers, suppliers, and developers can access and extend Concur’s T&E cloud. Concur’s systems adapt to individual employee preferences and scale to meet the needs of companies from small to large.

Contacts for this PoC:

john.dietz@sap.com



***About Amazon Echo (Alexa)

Amazon Echo is designed around your voice. It’s hands-free and always on. With seven microphones and beam-forming technology, Echo can hear you from across the room—even while music is playing. Echo is also an expertly tuned speaker that can fill any room with immersive sound. Echo connects to Alexa, a cloud-based voice service, to provide information, answer questions, play music, read the news, check sports scores or the weather, and more—instantly. All you have to do is ask. Echo begins working as soon as it detects the wake word.

***About Evature (Eva)

Evature develops an Expert Virtual Agent (EVA), enabling free-text search for online travel.

Utilizing innovative algorithms to process search requests, EVA understands the users’ inputs and converts them to structured search queries with very high precision.

***About Prontoly

“Authentication by Proximity. Prontoly developed proximity-based authentication technology that identifies users via their mobile device. This connection is made through the emission and detection of imperceptible sounds using the device’s microphone and speakers and doesn’t require any extra hardware or user action.

To report this post you need to login first.

1 Comment

You must be Logged on to comment or reply to a post.

  1. Giridhari Keerthi

    Hi Inna,
    I’m working on a scenario where i need to integrate alexa and c4c through HCP.I want to control my C4C system using voice commands through alexa.How can i acheive it??please help.

    Thanks and Regards,
    Giridhar

    (0) 

Leave a Reply