Live ‘Speech-To-Text’ Recognition with Deepgram API in PWA

Publication date: 2023-03-10

Deepgram is a company that specializes in developing speech recognition and natural language processing technologies. They provide various products and services to help businesses and organizations convert audio and speech data into text, extract insights, and automate workflows. In this post, I will share my experience of using their service in web applications. You can see the demo at https://deepgram.demo.wiredgeese.com/

Deepgram Live STT Demo

Deepgram SDK

Deepgram offers a variety of SDKs for using its API with different programming languages. However, as web application development primarily uses JavaScript, I will be using the Node.js SDK in my app.

Using this SDK, you can directly access the Deepgram API in a serverless mode from within a browser. To do this, you need to include the SDK in your project by running the following command:

$ npm install @deepgram/sdk Once the installation is complete, make the ./dist/browser/index.js file available from the browser. You can do this by copying the file to the public directory of your web application using the following command:
$ cp ./node_modules/@deepgram/sdk/dist/browser/index.js ./js/lib/deepgram.browser.sdk.mjs Once you have completed these steps, you can use the library in your application in the same way it is described in the documentation. Simply import the Deepgram object from ./lib/deepgram.browser.sdk.mjs and create a new instance of it with your Deepgram API key:
import {Deepgram} from ‘./lib/deepgram.browser.sdk.mjs’;
const deepgram = new Deepgram(DEEPGRAM_API_KEY);

Authentication

Principally, you can interact with the Deepgram API using curl. Here’s an example code for transcribing pre-recorded audio:

curl –request POST

–url ‘https://api.deepgram.com/v1/listen’

–header ‘Authorization: Token ’

–header ‘content-type: application/json’

–data ‘{“url”:“string”}’ To authenticate your requests, you’ll need to generate a token (YOUR_DEEPGRAM_API_KEY) using the Deepgram Web UI:

Deepgram Web Console

However, this method is only suitable for analyzing pre-recorded audio files. To use it, you’ll need to send an audio file (speech) to the service through a POST request, and the service will respond with the result of the analysis (text).

Live Transcription

To translate speech into text live, you need to implement the following sequence of actions:

Open a socket with the Deepgram API, passing the configuration parameters (at least the speech language).
Launch a media recorder in the browser to record speech.
Transfer the recorded speech in parts through the socket to the Deepgram API.
Receive the transcribed text from the Deepgram API via the same socket and process it.

This is the code to open socket:

import {Deepgram} from ‘./lib/deepgram.browser.sdk.mjs’; const deepgram = new Deepgram(API_KEY);
const deepgramSocket = deepgram.transcription.live({language: LANG});
deepgramSocket.addEventListener(‘open’, onSocketOpen);
deepgramSocket.addEventListener(‘message’, onSocketMessage);

The code to start speech recording:

navigator.mediaDevices

.getUserMedia({audio: true})

.then((stream) => {

const mediaRecorder = new MediaRecorder(stream, {

mimeType: ‘audio/webm’,

});
mediaRecorder.start(DEF_TIME_SLICE);
}); The DEF_TIME_SLICE parameter refers to the interval at which the dataavailable event is triggered and recorded by the media recorder.

Transfer audio data to Deepgram API on every dataavailable event:

mediaRecorder.addEventListener(‘dataavailable’, async (event) => {
if (event.data.size > 0 && deepgramSocket.readyState === 1) {
deepgramSocket.send(event.data);
}
});

Get result with transcriptions:

function onSocketMessage(message) {
const received = JSON.parse(message.data);
console.log(received);
}

The transcribed text we need is in this attribute:

data.channel.alternatives[0].transcript

Result Analyzing

Here is the structure of a typical response from the service:

“channel_index”: [0,1],

“duration”: 2.16,

“start”: 0,

“is_final”: true,

“speech_final”: true,

“channel”: {

“alternatives”: [

“transcript”: “This is demo.”,

“confidence”: 0.9929168,

“words”: [

“word”: “this”,

“start”: 1.2143519,

“end”: 1.3736111,

“confidence”: 0.99766505,

“punctuated_word”: “This”

},
{

“word”: “is”,

“start”: 1.3736111,

“end”: 1.6523148,

“confidence”: 0.9929168,

“punctuated_word”: “is”

},
{

“word”: “demo”,

“start”: 1.6523148,

“end”: 1.9310185,

“confidence”: 0.56145895,

“punctuated_word”: “demo.”

}
]
}
]
},

“metadata”: {

“request_id”: “dca09eea-e392-4282-909d-f545f44724fe”,

“model_info”: {

“name”: “general”,

“version”: “2022-01-18.1”,

“arch”: “vega”

},

“model_uuid”: “c12089d0-0766-4ca0-9511-98fd2e443ebd”

}
}

Deepgram transcribes speech as it arrives and may revise an earlier result in a later response. Each response identifies the speech fragment with start and duration, and reports word-level confidence. The browser therefore needs to assemble the final transcript from these provisional and final fragments.

Conclusion

The Deepgram API allows web developers to integrate speech recognition and transcription capabilities directly into their browser side code. By using the SDK, developers can send audio files or live speech to the Deepgram API for transcription, and receive back a text representation of the speech.

The use of the service is not expensive. Deepgram offers a range of plans, including a free plan that allows up to 2 hours of audio processing per month. Their paid plans start at $1 per hour of audio processed and go up to custom pricing for enterprise-level customers with specific needs.

At Wiredgeese Devs, we specialize in developing high-quality, custom PWAs that leverage the latest technologies to provide the best possible user experience.

If you’re interested in taking your PWA to the next level by integrating Deepgram’s API, our team of expert developers is ready to help. We can work with you to create a customized solution that meets your specific needs, whether you’re looking to transcribe audio recordings, provide real-time voice recognition, or something else entirely.

Don’t let your PWA fall behind the competition. Contact Wiredgeese Devs today to learn more about how we can help you integrate Deepgram’s powerful speech-to-text recognition service into your app.

If you enjoyed this article, please give it a clap and follow me for more content!

Stay connected:

Thank you for your support!