Summary

We all know that chatbots are AI's answer to improved customer service and cost savings. Chatbots are available in many user interfaces and input forms, and previous code patterns have shown how to create chatbots using different mediums such as Slack, web interface, and Facebook Messenger. In this code pattern, we use a web interface again, but instead of using text input, we'll use voice input and output. The code pattern uses Watson Assistant to control the conversation dialog, and Watson Speech to Text and Watson Text to Speech services to handle the speech recognition and playback.

Description

We've created several code patterns that use various user input forms, like the Speech Sandbox code pattern that uses a VR headset as input, the Watson Assistant and Slack code pattern that uses text and a Slack interface, and the Facebook Messenger and Watson tutorial that uses text and Facebook. This code pattern demonstrates a web-based application that uses Watson Text to Speech, Watson Speech to Text, and Watson Assistant. We use voice input and audio output as well as a web browser as the interface.

The main website is built using jQuery, and the API calls are made using Python flask. A WebSocket connection is created to make the calls to the various Watson services, and a sample insurance conversation is used for the dialog.

When you have completed this code pattern you should understand how to:

• Make a Watson Speech To Text call using a WebSocket connection
• Make a Watson Text to Speech REST API call
• Send and receive messages to Watson Assistant using REST APIs
• Integrate Watson Speech To Text, Watson Text To Speech, and Watson Assistant in a web app

Flow

1. User selects the microphone option on the browser and speaks.
2. The voice is passed on to Watson Speech to Text using a WebSocket connection.
3. The text from Watson Speech to Text is extracted and sent as input to Watson Assistant.
4. The response from Watson Assistant is passed to Watson Text to Speech.
5. The audio output is sent to the web application and played back to the user, while the UI also displays the same text.

Instructions

See the README for detailed instructions. These steps explain how to:

1. Clone the repo.
2. Create the services and deploy the web app.
3. Upload the Watson Assistant workspace.
4. Configure environment variables.