Why not use the unused space in letterboxing?
November 10, 2022 6:14 PM Subscribe
Serious question: when movies and shows are presented letterboxed... why don't the captions/subtitles make use of the unused black space at the bottom of the screen instead obscuring part of the image? I assume there's a reason.
Closed captioning technology is quite the rabbit hole, as evidenced by the long article about it on Wikipedia. "Old" 608 CC worked with analog TV (the NTSC standard) and "new" 708 CC works with the digital TV starting in the 2000s. However the latter often just mimic'd the former, digitizing the analog signals onto line 21 of the video signal (the WP article gets more into that).
To get to your question, the key is that typically A) your TV is doing the letterboxing but B) your set top box (STB) is doing the CC decoding. The WP article has a whole section that explains this, but basically the newer CC data is part of the compressed TV stream that is getting delivered to your house (internet stream, cable, satellite, even DVD) and your STB uncompresses that and produces the big fat video signal (HD video, 5.1 sound, CC, everything). It then sends THAT to the display device(s), and that display device then presents it how you want. For example, somewhere in that display device might be a menu setting that says you want to CROP video instead of letterbox (the horror!). The STB doesn't know what you are doing there on the display device, so it keeps the rendered CC text in the video frame.
Now, there is one case where this DOES work how you might want: when the display device is also the decoder. If you are using an antenna input directly into your TV (free HD!), then in theory it might be smart enough to put the CC off the image. But honestly it's such a low use case (not many people use OTA, and I say this as a huge fan of OTA) that the TV manufacturer's software team probably doesn't bother.
As an aside, newer CC is quite a bit more advanced in that the content author does have more control over the CC text rendering. For example, they can specify colors (oh no white background!), placement (oh no sports score at bottom!) or even font (oh no they hate san serif!). In the old days, you got white and bottom and blocky and you liked it.
posted by intermod at 7:48 PM on November 10, 2022 [10 favorites]
To get to your question, the key is that typically A) your TV is doing the letterboxing but B) your set top box (STB) is doing the CC decoding. The WP article has a whole section that explains this, but basically the newer CC data is part of the compressed TV stream that is getting delivered to your house (internet stream, cable, satellite, even DVD) and your STB uncompresses that and produces the big fat video signal (HD video, 5.1 sound, CC, everything). It then sends THAT to the display device(s), and that display device then presents it how you want. For example, somewhere in that display device might be a menu setting that says you want to CROP video instead of letterbox (the horror!). The STB doesn't know what you are doing there on the display device, so it keeps the rendered CC text in the video frame.
Now, there is one case where this DOES work how you might want: when the display device is also the decoder. If you are using an antenna input directly into your TV (free HD!), then in theory it might be smart enough to put the CC off the image. But honestly it's such a low use case (not many people use OTA, and I say this as a huge fan of OTA) that the TV manufacturer's software team probably doesn't bother.
As an aside, newer CC is quite a bit more advanced in that the content author does have more control over the CC text rendering. For example, they can specify colors (oh no white background!), placement (oh no sports score at bottom!) or even font (oh no they hate san serif!). In the old days, you got white and bottom and blocky and you liked it.
posted by intermod at 7:48 PM on November 10, 2022 [10 favorites]
Response by poster: I can buy that for closed captioning.
But what about subtitles as rendered by an app?
posted by DirtyOldTown at 9:14 PM on November 10, 2022
But what about subtitles as rendered by an app?
posted by DirtyOldTown at 9:14 PM on November 10, 2022
If it's a movie screening theatrically, it's not supposed to have black bars – the screen should be masked, which involves putting an actual physical matte black frame around the image, adjusted in size according to the film's intended aspect ratio.
Theatres increasingly don't care about masking, so you're probably seeing black bars more often in the movie theatre, but you shouldn't be. Masking helps the audience feel more immersed in the film and also enhances the colour contrast in the images, whereas black bars make the film look more washed out. It's a fundamental principle of good projection that unfortunately is increasingly neglected and so a lot of people don't know that they should expect it.
posted by RubyScarlet at 5:13 AM on November 11, 2022 [4 favorites]
Theatres increasingly don't care about masking, so you're probably seeing black bars more often in the movie theatre, but you shouldn't be. Masking helps the audience feel more immersed in the film and also enhances the colour contrast in the images, whereas black bars make the film look more washed out. It's a fundamental principle of good projection that unfortunately is increasingly neglected and so a lot of people don't know that they should expect it.
posted by RubyScarlet at 5:13 AM on November 11, 2022 [4 favorites]
Once on desktop VLC I was able to get subtitles to display in the black space (useful since I was on a 4:3 monitor...) but it took a bit of messing around with the settings - iirc you couldn't just say "display outside of the actual video", you had to figure out coordinates that would work.
posted by trig at 5:33 AM on November 11, 2022
posted by trig at 5:33 AM on November 11, 2022
As to reason... My guess is subtitles just don't get prioritized by developers.
posted by trig at 5:35 AM on November 11, 2022
posted by trig at 5:35 AM on November 11, 2022
Response by poster: I'm strictly talking about television.
posted by DirtyOldTown at 5:39 AM on November 11, 2022 [1 favorite]
posted by DirtyOldTown at 5:39 AM on November 11, 2022 [1 favorite]
Response by poster: My guess was that it was either seen as too much contrast, or the very act of letterboxing on a digital TV set means no information is sent to the black bar area at all, and sending the subs there would also require sending "black" data, which would increase data needs.
posted by DirtyOldTown at 5:57 AM on November 11, 2022
posted by DirtyOldTown at 5:57 AM on November 11, 2022
When you display the subtitles on the image you're certain that whatever happens they will not get cropped. There's like a safe zone on the image that pretty much guarantees that whatever the size or the ratio of the screen on which the shows is airing the subtitles are going to be visible.
There could be a tech solution for that but it's a hassle to make and no one cares to really spend the time and money to create it. Until recently nobody cared about subtitles, it was for foreign films buff and people with disabilities that's not enough people to really try and create a technical standard adapted to the whole range of where subtitles are used. So the easiest way is to stick them where they won't cause problems.
posted by SageLeVoid at 6:04 AM on November 11, 2022 [2 favorites]
There could be a tech solution for that but it's a hassle to make and no one cares to really spend the time and money to create it. Until recently nobody cared about subtitles, it was for foreign films buff and people with disabilities that's not enough people to really try and create a technical standard adapted to the whole range of where subtitles are used. So the easiest way is to stick them where they won't cause problems.
posted by SageLeVoid at 6:04 AM on November 11, 2022 [2 favorites]
Because physical home media/television broadcasts/streaming don’t control the aspect ratio of your television and so the only way to guarantee that the subtitles are visible is to put them on the actual image.
posted by rhymedirective at 6:56 AM on November 11, 2022 [2 favorites]
posted by rhymedirective at 6:56 AM on November 11, 2022 [2 favorites]
"rendered by an app" and "strictly talking about television" seem contradictory to me, but in general, I think SageLeVoid has it.
In many circumstances, the software rendering the subtitles doesn't actually know how the final screen that is displaying it will crop the image. In theory, digital video technology (HDMI etc) is supposed to tell the software this, but often TVs and monitors have essentially broken implementations of the HDMI spec, so even if tou wanted to put in the effort to get it right as a developer, it's basically impossible to make something that would work in all cases.
Smart TVs should be able to do this because they're more tightly integrated — I'm not sure whether some do. The ones that don't, it's almost certainly just that it wasn't prioritized in the development, rather than a technical limitation.
posted by wesleyac at 9:20 AM on November 11, 2022
In many circumstances, the software rendering the subtitles doesn't actually know how the final screen that is displaying it will crop the image. In theory, digital video technology (HDMI etc) is supposed to tell the software this, but often TVs and monitors have essentially broken implementations of the HDMI spec, so even if tou wanted to put in the effort to get it right as a developer, it's basically impossible to make something that would work in all cases.
Smart TVs should be able to do this because they're more tightly integrated — I'm not sure whether some do. The ones that don't, it's almost certainly just that it wasn't prioritized in the development, rather than a technical limitation.
posted by wesleyac at 9:20 AM on November 11, 2022
Response by poster: "rendered by an app" and "strictly talking about television" seem contradictory to me
For instance, Netflix is an app a person can watch on their television, on which they can turn on subtitles.
posted by DirtyOldTown at 9:27 AM on November 11, 2022
For instance, Netflix is an app a person can watch on their television, on which they can turn on subtitles.
posted by DirtyOldTown at 9:27 AM on November 11, 2022
Response by poster: I should also be clear: I'm not talking about widescreen-to-4-by-3 TV letterboxing, I'm talking about 21-by-9 to 16-by-9 letterboxing, widescreen cinema ratio to widescreen TV letterboxing.
posted by DirtyOldTown at 9:59 AM on November 11, 2022
posted by DirtyOldTown at 9:59 AM on November 11, 2022
1. Why deal with exceptions when you can deal with the constant of putting text over the viewable picture.
2. Most digital release are cut to a 16:9 ratio, which is the same ratio as a typical HD tv, so there isn’t a letterbox space below the picture. (see 1)
posted by furtive at 11:38 AM on November 11, 2022
2. Most digital release are cut to a 16:9 ratio, which is the same ratio as a typical HD tv, so there isn’t a letterbox space below the picture. (see 1)
posted by furtive at 11:38 AM on November 11, 2022
Response by poster: Most digital release are cut to a 16:9 ratio, which is the same ratio as a typical HD tv, so there isn’t a letterbox space below the picture.
They're not, though. Watching prestige TV, a startlingly high percentage of shows are letterboxed. It actually boggles the mind. Why would you make a show specifically for a 16:9 device and not make it on 16:9?
Staying with Netflix, currently these shows are all letterboxed, to one extent or the other: The Watcher, The Crown, Stranger Things, The Witcher, Dahmer, The Good Nurse, Inside Man, All Quiet on the Western Front, The Bastard Son of the Devil Himself, Devil in Ohio... That's ten of the first twelve Netflix originals from the trending bar, all letterboxed. Some of those are 2.35:1, most are just 1.85:1. But even in the case of the latter, why not orient the subtitles against the bottom of the screen so as to block as little as possible of the image?
Netflix, I might add, uses some of the space above and below the letterboxed for some of its labelling content for these shows while they are on preview, so it's not like they cannot access those spaces.
posted by DirtyOldTown at 12:02 PM on November 11, 2022
They're not, though. Watching prestige TV, a startlingly high percentage of shows are letterboxed. It actually boggles the mind. Why would you make a show specifically for a 16:9 device and not make it on 16:9?
Staying with Netflix, currently these shows are all letterboxed, to one extent or the other: The Watcher, The Crown, Stranger Things, The Witcher, Dahmer, The Good Nurse, Inside Man, All Quiet on the Western Front, The Bastard Son of the Devil Himself, Devil in Ohio... That's ten of the first twelve Netflix originals from the trending bar, all letterboxed. Some of those are 2.35:1, most are just 1.85:1. But even in the case of the latter, why not orient the subtitles against the bottom of the screen so as to block as little as possible of the image?
Netflix, I might add, uses some of the space above and below the letterboxed for some of its labelling content for these shows while they are on preview, so it's not like they cannot access those spaces.
posted by DirtyOldTown at 12:02 PM on November 11, 2022
I don't think this is related to the aspect ratio at all. Netflix can sense the aspect ratio of your tv (the status bar extends the entire way on ours), and I think they could put them down there if they wanted. I think overlaid subtitles are better from an accessibility standpoint though. It integrates them into the scene and action more (making them feel more like speech bubbles rather than narration text in comics kinda?) and the physical distance your eyes need to look down to read is reduced, meaning you're keeping more of the facial/scene cues in your peripherals despite having to be reading the subtitles simultaneously. You're not "leaving the scene" to look down at the subtitles.
I watch tv on my computer with portuguese subtitles turned on because I'm learning the language. The program I use puts them below the image, and I find them much easier to ignore than when they are on the image. (this is bad for my portuguese but would be great for netflix where I can hear and understand everything in theory but find dialogue on tv very mumbly sometimes.)
posted by euphoria066 at 2:09 PM on November 11, 2022 [3 favorites]
I watch tv on my computer with portuguese subtitles turned on because I'm learning the language. The program I use puts them below the image, and I find them much easier to ignore than when they are on the image. (this is bad for my portuguese but would be great for netflix where I can hear and understand everything in theory but find dialogue on tv very mumbly sometimes.)
posted by euphoria066 at 2:09 PM on November 11, 2022 [3 favorites]
This thread is closed to new comments.
posted by blueberry at 7:13 PM on November 10, 2022 [2 favorites]