Home - ruchikajulka/Tutorial--IBM-Watson-Discovery GitHub Wiki

Welcome to the Tutorial--IBM-Watson-Discovery- wiki!

Introduction

What is IBM Watson Discovery?

Watson Discovery is an award-winning AI-powered search technology that uses text analytics and natural language processing to break open data silos and retrieves specific answers to your questions, while also data mining to analyze and relationships buried in enterprise data. Watson Discovery is an AI-powered search that uses natural language processing to understand and easily be trained on the language of your domain. Unlike competitors, Watson Discovery can be deployed on any cloud or on-premises environment. IBM Watson Discovery is named a 'Leader' in “The Forrester Wave™ Cognitive Search Q2 2019 report, receiving the highest scores possible in information ingestion, intent intelligence, relevancy intelligence, and tuning tools criteria.

What is SDU?

SDU trains Watson Discovery to extract custom fields in your documents. Customizing how your documents are indexed into Discovery will improve the answers returned from queries.

With SDU, you annotate fields within your documents to train custom conversion models. As you annotate, Watson is learning and will start predicting annotations. SDU models can also be exported and used on other collections.

Current document type support for SDU is based on your plan:

Lite plans: PDF, Word, PowerPoint, Excel, JSON, HTML Advanced plans: PDF, Word, PowerPoint, Excel, PNG, TIFF, JPG, JSON, HTML

Business Case

Retrieve product information from a lengthy product manual quickly using Smart Document Understanding. Using the Watson Discovery Smart Document Understanding (SDU) feature, we will enhance the Discovery model to return us the most accurate answers from the product manual. Using Watson Assistant, we will use a standard customer care dialog to handle a typical conversation between a customer and a chatbot. When a customer question is about product information like a specification or a particular functionality, the Assistant dialog will call with the Watson Discovery API using a webhook.

Some Important terms which will help us understand IBM cloud platform capabilities used for building this PaaS experiment:

What is a webhook?

A webhook is a mechanism that allows you to call out to an external program based on something happening in your program. When used in a Watson Assistant dialog skill, a webhook is triggered when the Assistant processes a node that has a webhook enabled. The webhook collects data that you specify or that you collect from the user during the conversation and save in context variables, and sends the data to the Webhook request URL as an HTTP POST request. The URL that receives the webhook is the listener. It performs a predefined action using the information that is provided by the webhook as specified in the webhook definition and can optionally return a response.

In our experiment, the webhook will communicate with an IBM Cloud Functions web action, which is connected to the Watson Discovery service.

The Architecture of Our Experiment

  • The document is annotated using Watson Discovery SDU
  • The customer interacts with a chatbot (IBM Watson Assistant).
  • If the user asks a product operation question, a search query is passed to a predefined IBM Cloud Functions action.
  • The Cloud Functions action will query the Watson Discovery service and return the results back to the chatbot. architecture_dia

Lets Begin

Step 1:

Login to Queens IBM Portal

Step 2:

Create the Watson Discovery Service

Step 3:

Upload Data

Step 4:

Data Annotation & Model Training

Step 5:

Validate Results

Step 7:

Publish Watson Discovery Service

Step 8 (optional):

Create Watson Assistant (chatbot) and Consume Watson Discovery Service

References: