Does this exist? A video explainer comparing image and sound processing
October 4, 2019 10:34 AM Subscribe
What I would love to watch is a video that compares all the various image editing techniques and concepts (which as a designer I’m familiar with) with their equivalents in sound editing (which I’m not). I’ve wanted this for years – surely something like this exists?
The most recent episode of the excellent Twenty Thousand Hertz podcast, The [COMPRESSED] History of Mastering, gives an excellent example of exactly what I’m looking for:
This is great, but there are so many more visual techniques that must have a sonic equivalent, which surely are part of the toolkit of every sound engineer, only by different names: gamma adjustment, hue/saturation, inversion, opacity and layering, blend modes, sharpening, blur, posterising, noise, distortion, filters, low or high resolution, bit depth, JPG compression, mosaic, pixellation, cropping, pattern, collage, composition, symmetry, gestalt … I could go on and on. I would *love* to know what their equivalent would be in sound, and what they are used for.
Are there things in image processing that have no useful equivalent in sound? Equally, are there things in sound that have no visual equivalent? What are the visual equivalents of attack / decay / sustain / release? Of arpeggios? Of waveforms?
As interesting as the above podcast episode is, this needs to be a video. I need to both see and hear the effects being compared.
And if this doesn’t exist, please, someone, make it! Is there a YouTuber that I should suggest this to?
The most recent episode of the excellent Twenty Thousand Hertz podcast, The [COMPRESSED] History of Mastering, gives an excellent example of exactly what I’m looking for:
The other analogy is that mastering is like Photoshop for audio. So, we've all taken photographs, you know, on a mobile phone or a camera, and then maybe you have one that you actually want to print out or put on the wall. And you look at it, and actually you suddenly realize it's not quite as good as you thought it was. So, maybe you want to tweak the color balance, or enhance the contrast and the brightness, or maybe take out some red eye from a flash.In the episode they talk about the sonic equivalents of brightness, contrast, colour balance, noise reduction, and even compare the arrangement of songs on an album with the layout of a photobook or exhibition.
This is great, but there are so many more visual techniques that must have a sonic equivalent, which surely are part of the toolkit of every sound engineer, only by different names: gamma adjustment, hue/saturation, inversion, opacity and layering, blend modes, sharpening, blur, posterising, noise, distortion, filters, low or high resolution, bit depth, JPG compression, mosaic, pixellation, cropping, pattern, collage, composition, symmetry, gestalt … I could go on and on. I would *love* to know what their equivalent would be in sound, and what they are used for.
Are there things in image processing that have no useful equivalent in sound? Equally, are there things in sound that have no visual equivalent? What are the visual equivalents of attack / decay / sustain / release? Of arpeggios? Of waveforms?
As interesting as the above podcast episode is, this needs to be a video. I need to both see and hear the effects being compared.
And if this doesn’t exist, please, someone, make it! Is there a YouTuber that I should suggest this to?
I've worked in both and they're different enough that you can't make useful comparisons most things other than things like bit depth. A lot of audio mixing is black magic that comes from combining different sources through various analog or digital equipment that mashes them together in pleasing ways.
What are the visual equivalents of attack / decay / sustain / release? Of arpeggios?
Remember those horrible video effects that were used in 60s and 70s music shows? That.
posted by Candleman at 12:45 PM on October 4, 2019 [1 favorite]
What are the visual equivalents of attack / decay / sustain / release? Of arpeggios?
Remember those horrible video effects that were used in 60s and 70s music shows? That.
posted by Candleman at 12:45 PM on October 4, 2019 [1 favorite]
What are the visual equivalents of attack / decay / sustain / release? Of arpeggios?
attack -- fast zoom
decay -- dolly out
sustain -- hold on character
arpeggio -- quick cutting
release -- cartoon character standing mid air, notices, whoosh
posted by sammyo at 3:32 PM on October 4, 2019
attack -- fast zoom
decay -- dolly out
sustain -- hold on character
arpeggio -- quick cutting
release -- cartoon character standing mid air, notices, whoosh
posted by sammyo at 3:32 PM on October 4, 2019
One thing that’s really important to consider is that any sound is inherently dynamic, which is not true of an image. When you pause a movie you get a still image. When you pause a record you get silence. Even when you break it down to its most elemental, a pixel alone carries some information (color, or at least on-or-off). A single audio sample is entirely meaningless outside the context of the samples that come before and after it.
posted by STFUDonnie at 4:07 PM on October 4, 2019 [3 favorites]
posted by STFUDonnie at 4:07 PM on October 4, 2019 [3 favorites]
Response by poster: Sorry, I should clarify: I don't want to compare sound with video, I want a video that compares sound with image.
And I'm surprised people don't see many parallels. If you look at my list:
• sharpening: surely sound engineers know how to make dialog stand out better from background noise?
• gamma adjustment: ways of adjusting brightness and contrast, but with the emphasis on the higher frequencies or lower frequencies
• layering and collage: Photoshop made it easy for digital artists to overlay and blend images in ways that were difficult for artists in the past (e.g. double exposure and other darkroom tricks). In the same way sound mixers can overlay and blend any amount of input sources (e.g. starting with experimenters like Brian Eno)
• inversion: a photographic negative looks weird and has startling colours, but it also looks cool and is often used to aesthetic effect. Isn't this comparable to how reversing a sound sounds weird, but is often used to cool effect? Like I often hear backwards percussion nowadays?
• distortion and noise: you can make photos look more interesting and affect its 'mood' with noise (it might sometimes even be called a 'grunge' filter), and obviously we love this sort of thing in sound too.
I don't mean to threadsit, but I'm convinced there's a rich vein here that can be illuminated with visual and sonic examples, with explanations of what the artist / editor is doing.
posted by snarfois at 1:41 AM on October 5, 2019
And I'm surprised people don't see many parallels. If you look at my list:
• sharpening: surely sound engineers know how to make dialog stand out better from background noise?
• gamma adjustment: ways of adjusting brightness and contrast, but with the emphasis on the higher frequencies or lower frequencies
• layering and collage: Photoshop made it easy for digital artists to overlay and blend images in ways that were difficult for artists in the past (e.g. double exposure and other darkroom tricks). In the same way sound mixers can overlay and blend any amount of input sources (e.g. starting with experimenters like Brian Eno)
• inversion: a photographic negative looks weird and has startling colours, but it also looks cool and is often used to aesthetic effect. Isn't this comparable to how reversing a sound sounds weird, but is often used to cool effect? Like I often hear backwards percussion nowadays?
• distortion and noise: you can make photos look more interesting and affect its 'mood' with noise (it might sometimes even be called a 'grunge' filter), and obviously we love this sort of thing in sound too.
I don't mean to threadsit, but I'm convinced there's a rich vein here that can be illuminated with visual and sonic examples, with explanations of what the artist / editor is doing.
posted by snarfois at 1:41 AM on October 5, 2019
> Isn't this comparable to how reversing a sound sounds weird, but is often used to cool effect?
Reversing a sound is the same as reversing a movie. You can’t really “reverse” a still image in that sense. You can mirror a still image, which I suppose you could think of as somewhat analogous to switching the right and left audio channels.
Inverting a sound (where signal is high, make it low and vice versa) results in a sound indistinguishable from the original to the human ear, unless you layer it with the non-inverted signal, in which case they’ll cancel each other out (destructive interference).
posted by STFUDonnie at 6:44 AM on October 5, 2019 [1 favorite]
Reversing a sound is the same as reversing a movie. You can’t really “reverse” a still image in that sense. You can mirror a still image, which I suppose you could think of as somewhat analogous to switching the right and left audio channels.
Inverting a sound (where signal is high, make it low and vice versa) results in a sound indistinguishable from the original to the human ear, unless you layer it with the non-inverted signal, in which case they’ll cancel each other out (destructive interference).
posted by STFUDonnie at 6:44 AM on October 5, 2019 [1 favorite]
You may also be interested in spectral audio editing, which can be thought of as converting the sound to an image, editing the image, and converting the image back to sound. I find that it sounds pretty shit for anything other than surgically removing certain sounds isolated in a particular area of the “image,” which is maybe not unlike red-eye reduction? I’ll sometimes graft a nearby section of the image to cover the hole left by the excised sound.
posted by STFUDonnie at 7:47 AM on October 5, 2019 [1 favorite]
posted by STFUDonnie at 7:47 AM on October 5, 2019 [1 favorite]
Response by poster: Yes, in Photoshop you’d do that with the clone stamp tool. I also thought sound editors probably do something like this for surgical noise removal.
posted by snarfois at 9:17 AM on October 5, 2019
posted by snarfois at 9:17 AM on October 5, 2019
in Photoshop you’d do that with the clone stamp tool. I also thought sound editors probably do something like this for surgical noise removal.
Sort of - if there's nothing else going on in the audio, you can do the equivalent (like if there's a cough in between movements in a classical recording where you want to keep the time between the pieces. But you can get away with a lot more modification of photos than you can with audio - there's millions of pixels in a modern photo and a RAW will have a lot of different information that can be modified. You can sample what's around an image and come up with a pretty good guess what would be there if there wasn't a flaw obscuring it. It's a single moment in time.
Audio, there's just a soundwave and even minor modifications can have huge impacts on how we perceive it. You have to have a flowing narrative that makes sense to the ear and mind.
sharpening: surely sound engineers know how to make dialog stand out better from background noise?
Yes, but it is done with techniques entirely unlike sharpening.
gamma adjustment: ways of adjusting brightness and contrast, but with the emphasis on the higher frequencies or lower frequencies
Equalization will adjust frequencies but in a way completely different that gamma adjustment.
inversion: a photographic negative looks weird and has startling colours, but it also looks cool and is often used to aesthetic effect. Isn't this comparable to how reversing a sound sounds weird, but is often used to cool effect? Like I often hear backwards percussion nowadays?
They're different, because one is changing something static (what the colors are) while the other is changing perception over time.
distortion and noise: you can make photos look more interesting and affect its 'mood' with noise (it might sometimes even be called a 'grunge' filter), and obviously we love this sort of thing in sound too.
There's some accurate comparison there, particularly with the distortions caused by different types of analog film and analog audio tape. And all kinds of distortion are used in audio, obviously.
As I said, a lot of audio is black magic in a way that video/images aren't. There's not a mathematical formula that will explain why using a Neve 1073 will be magical on some audio sources but on others you'd want to reach for an API or a Manley. Or it might be perfect and you shouldn't touch it at all with an EQ. Whereas running a good sharpening algorithm on almost any image (that's not already been sharpened) will be perceived by almost everyone as improving it.
I'm convinced there's a rich vein here that can be illuminated with visual and sonic examples, with explanations of what the artist / editor is doing.
I've worked in photography, video, and professional audio and I don't. Most of the comparisons are so strained that they don't make sense to do. I could argue that it makes more sense to compare cooking with audio mixing, because it's more about bringing different elements together in very subtle ways, but it wouldn't necessarily be a useful video. If you want to learn about audio, why not just learn about audio without comparing it to something that's actually pretty dissimilar?
To the original analogy, I wouldn't compare mastering to Photoshop - processing a photo is closer to mixing. But again, it's very different to the point I wouldn't compare them in any meaningful way, other than to say that they're both steps in the process.
layering and collage: Photoshop made it easy for digital artists to overlay and blend images in ways that were difficult for artists in the past (e.g. double exposure and other darkroom tricks). In the same way sound mixers can overlay and blend any amount of input sources (e.g. starting with experimenters like Brian Eno)
Mixing different inputs together is kind of the same but what would a video to do to show it? I could make a video showing two images being blended together from 0 to 100% and then playing two audio sources being blended together from 0 to 100% but then what?
posted by Candleman at 10:57 AM on October 6, 2019 [3 favorites]
Sort of - if there's nothing else going on in the audio, you can do the equivalent (like if there's a cough in between movements in a classical recording where you want to keep the time between the pieces. But you can get away with a lot more modification of photos than you can with audio - there's millions of pixels in a modern photo and a RAW will have a lot of different information that can be modified. You can sample what's around an image and come up with a pretty good guess what would be there if there wasn't a flaw obscuring it. It's a single moment in time.
Audio, there's just a soundwave and even minor modifications can have huge impacts on how we perceive it. You have to have a flowing narrative that makes sense to the ear and mind.
sharpening: surely sound engineers know how to make dialog stand out better from background noise?
Yes, but it is done with techniques entirely unlike sharpening.
gamma adjustment: ways of adjusting brightness and contrast, but with the emphasis on the higher frequencies or lower frequencies
Equalization will adjust frequencies but in a way completely different that gamma adjustment.
inversion: a photographic negative looks weird and has startling colours, but it also looks cool and is often used to aesthetic effect. Isn't this comparable to how reversing a sound sounds weird, but is often used to cool effect? Like I often hear backwards percussion nowadays?
They're different, because one is changing something static (what the colors are) while the other is changing perception over time.
distortion and noise: you can make photos look more interesting and affect its 'mood' with noise (it might sometimes even be called a 'grunge' filter), and obviously we love this sort of thing in sound too.
There's some accurate comparison there, particularly with the distortions caused by different types of analog film and analog audio tape. And all kinds of distortion are used in audio, obviously.
As I said, a lot of audio is black magic in a way that video/images aren't. There's not a mathematical formula that will explain why using a Neve 1073 will be magical on some audio sources but on others you'd want to reach for an API or a Manley. Or it might be perfect and you shouldn't touch it at all with an EQ. Whereas running a good sharpening algorithm on almost any image (that's not already been sharpened) will be perceived by almost everyone as improving it.
I'm convinced there's a rich vein here that can be illuminated with visual and sonic examples, with explanations of what the artist / editor is doing.
I've worked in photography, video, and professional audio and I don't. Most of the comparisons are so strained that they don't make sense to do. I could argue that it makes more sense to compare cooking with audio mixing, because it's more about bringing different elements together in very subtle ways, but it wouldn't necessarily be a useful video. If you want to learn about audio, why not just learn about audio without comparing it to something that's actually pretty dissimilar?
To the original analogy, I wouldn't compare mastering to Photoshop - processing a photo is closer to mixing. But again, it's very different to the point I wouldn't compare them in any meaningful way, other than to say that they're both steps in the process.
layering and collage: Photoshop made it easy for digital artists to overlay and blend images in ways that were difficult for artists in the past (e.g. double exposure and other darkroom tricks). In the same way sound mixers can overlay and blend any amount of input sources (e.g. starting with experimenters like Brian Eno)
Mixing different inputs together is kind of the same but what would a video to do to show it? I could make a video showing two images being blended together from 0 to 100% and then playing two audio sources being blended together from 0 to 100% but then what?
posted by Candleman at 10:57 AM on October 6, 2019 [3 favorites]
Response by poster: Well, that's a bit disappointing, but thanks for the thoughtful answers, everyone! Perhaps I should have spoken of "metaphor" / "analogy" from the start, rather than "equivalent". And I wasn't looking for something "useful" to someone learning either trade, merely interesting. The Photoshop analogy I quote at the start wasn't mine, btw, it was made by Ian Shepherd, an experienced mastering engineer, in the podcast episode I link to.
It was also fascinating to learn of spectral audio editing; thanks @STFUDonnie
posted by snarfois at 1:49 AM on October 7, 2019
It was also fascinating to learn of spectral audio editing; thanks @STFUDonnie
posted by snarfois at 1:49 AM on October 7, 2019
This thread is closed to new comments.
Perhaps you'd gain more from learning about music and sound for what it is rather than through someone forcing it into a definition that isn't accurate.
Narhe Sol makes excellent videos based around piano, but they often include discussions of structure and color, and she also uses visual aids to make things clearer.
posted by jonathanhughes at 12:17 PM on October 4, 2019 [4 favorites]