Closed Captions

Closed Captions

On my stream, I use a plugin that produces native closed captions for Twitch: github.com/ratwithacompiler/OBS-captions-pl..

This plugin uses the Google Cloud Speech Recognition API, which I find is good enough, and when it's inaccurate (see the featured image), it's at least mildly entertaining.

There may be other native closed caption plugins out there, but this is the one I use and like.

I prefer native closed captions because:

  1. I don't need to have a browser window (or another app) open to record captions, it just uses my microphone input from OBS. This is important to me for gaming streams since I stream and game from the same PC, and having extra apps open (especially a web browser like Google Chrome) can impact the performance of my gaming laptop.

  2. It's built into Twitch natively via the CC button in the native controls, allowing users to use the UI elements Twitch built to turn them on, off, or reposition them.

  3. It's supported on Twitch's mobile apps: many Twitch extensions, due to platform limitations or developer discretion, aren't supported on mobile. Twitch (via Apple) also requires developers to fill out additional paperwork for Apple, which, in order to comply with, the developer needs to have an Apple Developer account ($100+ per year as a developer) to offer their extension on iOS, so you may even see some extensions supported on Android but not iOS.

  4. It does not require me as a streamer to install extensions on my channel and overlaying a potentially-buggy iframe on top of my entire stream video

  5. It supports multiple languages – I have French and Spanish mode channel point redemptions during my coding streams since I speak English, French, and Spanish. When someone redeems one of these, I can quickly change the language in the plugin so that it's reflected in Twitch's native closed captions.

Since it uses Twitch's native closed captions feature, it's supported everywhere Twitch has built closed captions support, like on mobile. Here's a screenshot of native closed caption support on Twitch's Android app.

You can turn them on for the live stream or the vod under the settings menu under the viewing options, which are under the quality options.

This section will be quite critical of using a Twitch extension for closed captions. As someone who has developed a Twitch extension myself (like command showcaserrr panel), I have a bit of experience in the platform and its limitations.

There are numerous Twitch extensions that offer closed captions but I don't use them mainly because I have concerns about the stability of Twitch extensions, specifically overlay and component extensions, and how that can impact the viewer experience.

On Twitch, you can build 3 types of extensions:

  • panel extensions, which run in the panels section on the About tab of your profile

  • overlay extensions, which run in an iframe on top of the entire video (overlaying it). Usually these are transparent but not always

  • component extensions, which run in the video area

Buggy extensions can interfere with the video

I do not want to run overlay and component extensions as they can interfere with the viewing experience on desktop web, as described above. I have also seen buggy overlay extensions (either due to the extension itself or Twitch's infrastructure) that have crashed, and caused a white overlay to be overlaid on top of a streamer's video. Other than bugging the streamer to disable the extension and then having all viewers refresh the page, the only solution as a viewer was to inspect the element and delete the offending extension's iframe DOM node so I can see the video again. Every overlay extension you install adds this risk.

Panel extensions do not cause this risk, but generally to view panel extensions, you need to scroll to the panels, which is not where you want to spend most of your time when viewing a stream so closed caption extensions are not likely to be built as a panel extension.

Extensions may not be supported on mobile

Another problem with extensions is that they may not be supported on mobile. One closed captions Twitch extension I was looking into featured live streams that used the extension. From there, I could see that the closed captions were supported on web, but when I went into the Android app for that stream, they weren't supported. Nearly 40% of my viewers are on Android, and just over 10% on iOS—if I used an extension that wasn't supported on mobile, that means half my viewers wouldn't be able to use the closed captions solution I set up.

In the Twitch mobile app, Twitch supports closed captions on both iOS and Android natively with a toggle.

Extensions may request more permissions than you're comfortable with

Twitch apps and extensions often ask for a lot of permissions that may not be required. While I haven't looked too closely into the closed caption Twitch extensions to see what permissions they request, in the past I was evaluating text-to-speech extensions and was really not a fan of them asking for all the permissions that they asked for so I built my own text-to-speech apps on macOS and Windows (you can read more about the excessive permissions in the blog post about the Windows TTS app I built).

Extensions may require you to have an additional app open

Many Twitch extensions are web-based, so they may require you to have your web browser open. This may not matter if you're doing a coding or just chatting stream, for example, but if you're doing a PC gaming stream and you don't want to have extra apps open because it could impact the performance of your game, this isn't ideal.

Conclusion

In summary, there are multiple solutions for closed captions. If you use OBS, I'd recommend the above-mentioned plugin. If not, and there isn't a comparable plugin for the broadcasting software you use, and you don't want to move to OBS, your only other option may be a Twitch extension.