Closed Captions

Closed Captions

ยท

3 min read

On my stream, I use a plugin that produces native closed captions for Twitch: github.com/ratwithacompiler/OBS-captions-pl..

I prefer this approach because:

  1. I don't need to have a browser window (or another app) open

  2. It's built into Twitch natively via the CC button in the native controls

  3. It does not require overlaying a potentially-buggy iframe on top of my entire stream video

Prerequisites

  • OBS version 27 -> this is an older version of OBS but the above-mentioned plugin works with it. Not sure if version 28 works, but if you've tried it and it does, let me know in the comments.

There are numerous Twitch extensions that offer closed captions but I don't use them mainly because I have concerns about the stability of Twitch extensions, specifically overlay and component extensions, and how that can impact the viewer experience.

On Twitch, you can build 3 types of extensions:

  • panel extensions, which run in the panels section on the About tab of your profile

  • overlay extensions, which run in an iframe on top of the entire video (overlaying it). Usually these are transparent but not always

  • component extensions, which run in the video area

I do not want to run overlay and component extensions as they can interfere with the viewing experience. I have also seen buggy overlay extensions (either due to the extension itself or Twitch's infrastructure) that have crashed, and caused a white overlay to be overlaid on top of a streamer's video. The only solution as a viewer was to inspect the element and delete the iframe DOM node so I can see the video again. Every overlay extension you install adds this risk.

The above-mentioned plugin uses the Google Cloud Speech Recognition API, which, at least for me, is not the most accurate. I don't know what services the available Twitch extensions use (the ones I've looked into do not disclose this information) but I would assume the most economical source would be built-in browser speech API's.

Building your own

You can build your own captioning by leveraging built-in browser API's. Many browsers have speech recognition API's built in which will translate your voice into text.

One of the things to note about closed captions is that they should be able to be turned off by the viewers. So if you want to support "closed captions," the viewers should be able to disable them. If they're burned onto your video, they're just captions.

I made a quick proof of concept where it would burn the text onto the video (i.e. captions and not closed captions). More work would be required to make them closed captions and togglable by users.

Here's a preview of it working:

Here's the code (which needs to be running standalone and not within the Codesandbox editor to work, and is currently only supported in Chrome):

ย