Skip to Content
Author's profile photo Uladzislau Pralat

Fine Tune Sentiment Analysis Using Custom Dictionary

    Sentiment analysis can be fine tuned using custom dictionaries, for example, sentiment polarity changed  or additional sentiments extracted. In my blog I will explain how to archive something different – link sentiment to correct topic.

I run Voice of Customer text analysis against some reviews with standard EXTRACTION_CORE_VOICEOFCUSTOMER configuration. What I found is that “microwave safe bowl” was incorrectly interpreted as WeakPositiveSentiment.

TA_1.jpg

TA_2.jpg

“safe” sentiment was linked to incorrect topic e.g. “a microwave” instead of “bowl”.  I do not care about a bowl as such and the sentiment just screw up my analysis. Excluding “safe” adjective from custom dictionary looks like a simple solution, but solves one problem and creates another (what if I need to extract “microwave is safe” WeakPositive Sentiment). The right solution is to define a new neutral sentiment “microwave safe” adjective in custom dictionary. This way “safe” sentiment is linked to the right topic e.g. “bowl”.

TA_3.jpg

Then reference custom dictionary in custom configuration

TA_4.jpg

and finally recreate text index

TA_5.jpg

If I query against $TA table again, I will see that sentiments are correctly extracted

TA_7.jpg

You can try described demo in your system. Import attached TA_DEMO-sap.com.tgzDelivery Unit. ta_demo package will be created

TA_9.jpg

and finally execute install.sqlscript. The latter will create REVIEW table with data, grants proper authorization to TA_DEMO schema and creates text index with standard EXTRACTION_CORE_VOICEOFCUSTOMER configuration.

TA_8.jpg

TA_10.jpg

 

Installation Instructions:

  1. Import TA_DEMO-sap.com.tgz Delivery Unit
  2. Execute install.sql SQLScript

Content of the Delivery Unit:

TA_DEMO.hdbschema

TA_DEMO_ROLE.hdbrole

fine_tune.hdbtextconfig

fine_tune.hdbtextdict

fine_tune.sql

install.sql

query.sql

 

Note: here is a helpful link How to Import Delivery Unit to HCP HANA MDC

 

 

Assigned Tags

      Be the first to leave a comment
      You must be Logged on to comment or reply to a post.