CVG 1.30.0 (23-May-2023)
ChatGPT is gradually finding its way into enterprise applications. To support this, Microsoft has now made Azure OpenAI generally available including the models gpt-3.5-turbo and gpt-4. This means that from now on ChatGPT can not only be used directly with OpenAI but also with Azure. You can now also choose between the providers “Azure” and “OpenAI” in our ChatGPT integration. In addition, we deliver a number of other ChatGPT integration improvements like faster output of replies and the ability to forward calls.
We continue to work on making your life as a bot developer easier. With this release, you can now call your voicebot directly from the CVG console via browser and enjoy less restrictive session handling. In addition, by popular demand, we have now enabled CVG-side collection of user input via a User Input Aggregation Timeout. This simplifies handling within Conversational AI, especially when entering longer data such as customer or invoice numbers.
Have fun building great voicebots!
Extended ChatGPT Integration
Azure or OpenAI as Provider
The arrival of ChatGPT in enterprise applications is taken into account in our integration with CVG, in which you can now also specify the provider of ChatGPT. You can choose between Azure and OpenAI.
In both cases, the entered data is not used for training, according to the manufacturers. While ChatGPT from OpenAI is currently always hosted in the USA, Azure also offers hosting in the European Union (Azure Region westeurope). This makes it much easier for many European companies to introduce ChatGPT based applications.
If you want to link your Azure OpenAI account with CVG, you need an API token as with OpenAI. in addition you need an Azure OpenAI Service resource with either the gpt-35-turbo or the gpt-4 models deployed. You can find more information directly at Microsoft.
Enter https://{your-resource-name}.openai.azure.com/openai/deployments/{deployment-id} as Microsoft Azure Deployment URL with your Azure OpenAI Service resource name and gpt-35-turbo or gpt-4 as {deployment-id}.
Check of OpenAI API Access Token and Model Availability
If you enter or change an OpenAI API access token, the model or the Azure deployment URL, the system checks whether this data is valid when saving. If not, an error message is displayed and saving is not possible. If a model or a deployment is not available in Azure, this may be because it has not yet been activated by the provider.
JSON Output from ChatGPT for Call Control
Do you want to provide your customers with a conversational IVR with the capabilities of ChatGPT that routes callers to the right department based on their concern? Or automate a customer request through the interaction of multiple bots built with ChatGPT or conventional conversational AIs on a case-by-case basis?
Then it is important that a bot built with ChatGPT can route calls with captured metadata to human agents or to additional bots.
We now enable this by using our ChatGPT integration in CVG to evaluate JSON expressions that ChatGPT generates when appropriate prompts are used. These JSON expressions are not output via TTS, but are used exclusively for call control.
Specifically, all key/value pairs in JSON expressions generated by ChatGPT are stored as custom data so that they can be read by downstream systems. If a call is to be forwarded, the JSON expression must contain the key “action” with the value “forward” and the key “destination” with the phone number to forward to in +E164 format.
Here is an example of a simple prompt (anonymized phone numbers; replace with real phone numbers to try yourself) that forwards the caller to one of the following departments depending on their request:
You are a friendly and humorous employee in the switchboard of VIER Insurance. Your name is Max Miller. Answer briefly in each case.
1.) Find out which department the caller should be connected to. There are the following departments with their respective topics and phone numbers:
- Department "Home and Housing"; Topics: Household insurance, building insurance; phone number +49xxxxxxxxx.
- Department "Vehicle"; topics: Motor vehicle insurance, e-car insurance, scooter insurance, quad insurance, boat insurance ; phone number +49xxxxxxxxx
- Department "Liability"; Private liability insurance, Animal owner liability insurance; phone number +49xxxxxxxxx
- Claims department; subjects: Claims notification, Damage to motor vehicle, Damage to building, Damage to household contents, Glass damage, Accident damage, Damage to boat; phone number +49xxxxxxxxx
- Department "Animals"; Topics: Animal health insurance, dog health insurance, cat health insurance, operation insurance for dogs and cats; phone number +49xxxxxxxxx
- Department "Occupational disability"; Topics: Occupational disability insurance; phone number +49xxxxxxxxx
- Department "Accident insurance"; Topics: Accident insurance, accident pension; phone number +49xxxxxxxxx
- Department "Old-age provision"; Topics: Basic pension, immediate pension, yield for more pension, fund pension; phone number +49xxxxxxxxx
- Department "Basic capabilities"; Topics: Basic ability insurance; phone number +49xxxxxxxxx
- Department "Investment"; Topics: Fund savings plan, one-time investment; phone number +49xxxxxxxxx
- Department "Company pension plan"; Topics: Direct insurance, support fund; phone number +49xxxxxxxxx
2) When you have found out the department, write your answer and say that you are forwarding. Add at the end a JSON expression containing the following key/value pairs:
- "department": department
- "action": "forward"
- "destination": phone number of the department
Faster ChatGPT Responses (Streaming)
To get faster responses from ChatGPT consider to activate the phrase by phrase output option. I.e. the response from ChatGPT is output sentence by sentence once a complete sentence is available as a partial result from ChatGPT. So the bot answers faster, but there can be pauses between the sentences.
Read Phone Numbers as Single Digits
If ChatGPT responses contain phone numbers in +E.164 format, these numbers are automatically formatted with SSML and read as single digits.
UI Improvements
To simplify your life as a bot developer, you can now call your voicebot directly from CVG via browser and enjoy less restrictive session handling.
Test your Voicebot with your Browser
During the configuration phase of a voicebot, a large number of test calls often take place: Is the selected TTS voice appropriate? Does the speaking rate fit? Are foreign words pronounced correctly? Are intents recognized accurately?
In order to make these test calls as easy as possible during configuration without having to reach for the phone, we have now integrated our WebRTC phone into the “Settings” page of the projects. This way you can easily talk to your voicebot via your browser and test your configuration.
Add our WebRTC Phone in your Web App
By the way, if you want to add WebRTC telephony to your web applications yourself, you can use our reference implementation published on GitHub.
Extended Session Timeout and Remember Me Feature
The old session timeout of 10 minutes was a bit short for many of our customers. We have therefore increased the session timeout to 1 hour. As long as the CVG browser tab is focused, the session does not expire at all.
If you want to continue using the sessions even after browser restarts, you can now click the “Remember Me” option.
Dynamically adjustable Timeout to aggregate Speech Input
New API Endpoint to dynamically set Timeouts
Sometimes it takes users a while to enter more complex data like a customer number or an invoice number. Until now you had to collect user input in your Conversational AI. If you want CVG to collect user input until the user says nothing for a configurable period of time, you can use the new /call/aggregation/start endpoint. This leads to transcribed utterances being buffered rather than being sent to the bot immediately. When the customer hasn’t said anything for a time period longer than the given timeout, the buffered messages are combined (separated by a space) into a single /message request.
Since all speech input is delayed by this timeout, the value should be chosen with care and just set in a dialog context where you expect more complex user input. Stop the input aggretion after you got this input by using the /call/aggregation/stop endpoint.
Some example:
Utterance Timeout = 300ms
user input aggregation timeout = 2000ms
User says "123" [500ms pause] "um, 456" [800ms pause] "789" [1800ms pause] "012" [>2000ms pause]
CVG collects "123", " um, 456", "789", "012" and sends "123 um, 456 789 012" as one (collected) user input to the bot
DTMF Input is not aggregated by this Modus
DTMF input is not collected, please continue to use the number prompt for this purpose.
Utterance Timeout, Inactivity Timeout, and Input Aggregation Timeout are independent
Note that the three timeouts, utterance timeout, inactivity timeout, and input aggregation timeout, are independent of each other.
The utterance timeout is used for detecting the end of an utterance (if Speech-to-Text is in non-streaming mode).
The inactivity timeout is triggered when the user is inactive. We recommend a value between 5 and 15 seconds.
The new input aggregation timeout should only be used in a context where longer user input is expected - otherwise it will delay all bot responses. A value between 2 and 5 seconds is recommended.
If the inactivity timeout is shorter than the input aggregation timeout, it will trigger before the aggregated input is delivered. You must account for this by either acting on it within your Conversational AI or setting a longer timeout for the inactivity timeout than for the aggregation timeout.
New Node in VIER Voice Extension for Cognigy
Accordingly, a new node has been provided in the VIER Extension for Cognigy. Here you can enable or disable input aggregation and set the timeout for input aggregation.
As long as version 4.5.0 is not available on the Cognigy marketplace, you can find the pre release of the new version here. You can update your VIER Voice Extension in Cognigy manually with this file.