Complex Fourier Drawing to Sound?
January 11, 2022 8:58 PM Subscribe
I am wondering if it's possible to take a Complex Fourier Drawing (yt link with animation showing), decompose the frequencies of each cycle then sum them into a wave form (and then rendered to audio) - if so - does anyone know of any applications that can do this?
I think the question is mostly self-explanatory?
Given a vector drawing and taking something like an inverse transform of each circle (I imagine that ultimately each circle is just a different frequency/phase) that could then be converted into wave forms.
Perhaps something where the larger circle is the primary every smaller circle is the next partial, and then summing each cycle into a single waveform that can be played back?
I imagine the issue here would be ensuring the frequencies of these are in auditory frequencies which might require some scaling, and possibly this may not work as a general case (e.g. maybe larger more complex figures would require a lot more customization to make the frequencies fit, vs one with fewer circles and thus less frequencies).
Has anyone seen anything like this? I see various things that already do the complex fourier series drawing, but I don't understand how it inverts it and how you can do an "inverse fourier transform"(I'm thinking this would be the ultimate final step after you break down the frequencies of each circle).
How hard is something like this to program? I would be willing to pay someone to do this if it wasn't too expensive/time consuming.
I think the question is mostly self-explanatory?
Given a vector drawing and taking something like an inverse transform of each circle (I imagine that ultimately each circle is just a different frequency/phase) that could then be converted into wave forms.
Perhaps something where the larger circle is the primary every smaller circle is the next partial, and then summing each cycle into a single waveform that can be played back?
I imagine the issue here would be ensuring the frequencies of these are in auditory frequencies which might require some scaling, and possibly this may not work as a general case (e.g. maybe larger more complex figures would require a lot more customization to make the frequencies fit, vs one with fewer circles and thus less frequencies).
Has anyone seen anything like this? I see various things that already do the complex fourier series drawing, but I don't understand how it inverts it and how you can do an "inverse fourier transform"(I'm thinking this would be the ultimate final step after you break down the frequencies of each circle).
How hard is something like this to program? I would be willing to pay someone to do this if it wasn't too expensive/time consuming.
[Many assumptions and guesses ahead. Corrections welcome.]
That technique starts with a drawing in the form of a single curve. That curve can be parameterized as a single point moving through a two-dimensional space over time (it's in the time domain). Via fourier transform, that's then converted into the phase and amplitude for each frequency (each rotating circle) in the set, and now we're in the frequency domain.
It sounds like you want to take that frequency domain information and convert it back into the time domain. Audio is a one-dimensional value (sound pressure) changing over time. But you started with the time domain information. I don't think you gain anything for your application by doing the Fourier transform and then transforming it back. You could just take the initial curve and use it as your audio waveform.
There's one complication there, in that the curve is 2D and audio is 1D. You could at least start by taking just the X- or Y- coordinate of the time-parameterized curve to get a 1D component out of it. Scale that in time as needed to get audible frequencies out of it.
So I'm not sure this gets you too much closer to an implementation, but it may save you some trouble if my understanding of what you want is correct and you can avoid the Fourier transform entirely.
posted by whatnotever at 10:23 PM on January 11, 2022 [1 favorite]
That technique starts with a drawing in the form of a single curve. That curve can be parameterized as a single point moving through a two-dimensional space over time (it's in the time domain). Via fourier transform, that's then converted into the phase and amplitude for each frequency (each rotating circle) in the set, and now we're in the frequency domain.
It sounds like you want to take that frequency domain information and convert it back into the time domain. Audio is a one-dimensional value (sound pressure) changing over time. But you started with the time domain information. I don't think you gain anything for your application by doing the Fourier transform and then transforming it back. You could just take the initial curve and use it as your audio waveform.
There's one complication there, in that the curve is 2D and audio is 1D. You could at least start by taking just the X- or Y- coordinate of the time-parameterized curve to get a 1D component out of it. Scale that in time as needed to get audible frequencies out of it.
So I'm not sure this gets you too much closer to an implementation, but it may save you some trouble if my understanding of what you want is correct and you can avoid the Fourier transform entirely.
posted by whatnotever at 10:23 PM on January 11, 2022 [1 favorite]
Mathematica has some Fourier related sonification features (I don't know enough to say if this is helpful of not :).
posted by dhruva at 7:25 AM on January 12, 2022 [1 favorite]
posted by dhruva at 7:25 AM on January 12, 2022 [1 favorite]
Best answer: It sounds like you want to take that frequency domain information and convert it back into the time domain
which is exactly what the code that renders any such Fourier drawing as an animation is doing. Some step in the rendering process has to generate a repeating set of x, y coordinates that repeatedly traces out the shape of the drawing over time.
Those coordinate pairs would most likely be handled as a list or stream of complex numbers (i.e. pairs of floats) at some point in the process. So if you've already got code that can make an animated Fourier drawing, you should be able to extract the audio you want from what the existing renderer produces by tweaking it to spit out x coordinates alone, or y coordinates alone, as suitably scaled audio sample values saved in some suitable essentially-raw audio file format like WAV.
Might be fun to use x and y to make left and right audio channels of a stereo pair. Because each of the partials in a Fourier animation renders as a circle, both channels would end up with the same frequencies at the same amplitudes and would sound the same when heard individually; but the signal at any given frequency would be 90° out of phase between channels, which I would expect a human brain to try to assemble into some kind of spatial effect if both were heard at the same time.
Also, drawings that have visually detailed and/or irregular shapes are probably not going to decompose to Fourier partials that resemble actual harmonics much. Expect lots of roars and hisses. Not that there's anything wrong with roars and hisses.
posted by flabdablet at 7:38 AM on January 12, 2022
which is exactly what the code that renders any such Fourier drawing as an animation is doing. Some step in the rendering process has to generate a repeating set of x, y coordinates that repeatedly traces out the shape of the drawing over time.
Those coordinate pairs would most likely be handled as a list or stream of complex numbers (i.e. pairs of floats) at some point in the process. So if you've already got code that can make an animated Fourier drawing, you should be able to extract the audio you want from what the existing renderer produces by tweaking it to spit out x coordinates alone, or y coordinates alone, as suitably scaled audio sample values saved in some suitable essentially-raw audio file format like WAV.
Might be fun to use x and y to make left and right audio channels of a stereo pair. Because each of the partials in a Fourier animation renders as a circle, both channels would end up with the same frequencies at the same amplitudes and would sound the same when heard individually; but the signal at any given frequency would be 90° out of phase between channels, which I would expect a human brain to try to assemble into some kind of spatial effect if both were heard at the same time.
Also, drawings that have visually detailed and/or irregular shapes are probably not going to decompose to Fourier partials that resemble actual harmonics much. Expect lots of roars and hisses. Not that there's anything wrong with roars and hisses.
posted by flabdablet at 7:38 AM on January 12, 2022
Incidentally, the process of translating in both directions between lists of time-series data and lists of Fourier partials is a mainstay of scientific programming, and efficient implementations are readily available in many well supported and well documented libraries.
posted by flabdablet at 7:56 AM on January 12, 2022
posted by flabdablet at 7:56 AM on January 12, 2022
Response by poster: Yeah my use case would actually be a fairly simple drawing, nothing like the video provided, actually
I wouldn't expect it to sound pretty by any means. Some odd FM style dissonant harmonics.
My use case would be to have this vector shape logo and play that sound at the start of the channel like a "station id" beep/noise/tone.
I imagine what I'd need is to get the x,y for each circle and render as a sine wave , then sum them for the final tone.
I've used a little numpy for a different project with waves, so I have a little experience on that front. I guess the thing for me is knowing where/how to generate these circles. I found a Javascript page that does it and open source code, but I suck w/JS and larger projects that I'd have to find the info buried, but I guess I can look there to see how they do it.
Thanks for the replies, folks! I think with some of the comments here they've given a general idea.
I am definitely going to see if there's already a library/software that can generate the circles frequencies and then maybe I can just sum them without having to do much of the legwork (ha yeah right).
posted by symbioid at 1:16 PM on January 12, 2022
I wouldn't expect it to sound pretty by any means. Some odd FM style dissonant harmonics.
My use case would be to have this vector shape logo and play that sound at the start of the channel like a "station id" beep/noise/tone.
I imagine what I'd need is to get the x,y for each circle and render as a sine wave , then sum them for the final tone.
I've used a little numpy for a different project with waves, so I have a little experience on that front. I guess the thing for me is knowing where/how to generate these circles. I found a Javascript page that does it and open source code, but I suck w/JS and larger projects that I'd have to find the info buried, but I guess I can look there to see how they do it.
Thanks for the replies, folks! I think with some of the comments here they've given a general idea.
I am definitely going to see if there's already a library/software that can generate the circles frequencies and then maybe I can just sum them without having to do much of the legwork (ha yeah right).
posted by symbioid at 1:16 PM on January 12, 2022
Best answer: I imagine what I'd need is to get the x,y for each circle and render as a sine wave , then sum them for the final tone.
Generating and summing x, y samples per circle at fixed time intervals is exactly what the animation renderers for these things are already doing. All you'd need to do is grab the final x, y pairs out of the renderer before they actually got used to plot graphics, scale them, and write them out in an audio file format. Pick a file format that supports floating-point samples and you probably wouldn't even need to scale them.
posted by flabdablet at 3:09 PM on January 12, 2022 [1 favorite]
Generating and summing x, y samples per circle at fixed time intervals is exactly what the animation renderers for these things are already doing. All you'd need to do is grab the final x, y pairs out of the renderer before they actually got used to plot graphics, scale them, and write them out in an audio file format. Pick a file format that supports floating-point samples and you probably wouldn't even need to scale them.
posted by flabdablet at 3:09 PM on January 12, 2022 [1 favorite]
Response by poster: For anyone curious, a bit more googling later I found a quick overview (by Dan Schiffman) in P5 how to do the drawing of an arbitrary image (working from basics of a simple wave).
Which helps understand the drawing process, and mentally I see now (especially when he gets to the person breakdown the x,y values of the vector graphics) how it all works (would work in reverse).
So I think I should be able to grab an SVG and get the x,y components from that (especially if I'm not drawing the art as I do it, though I'm sure there's a way to do a realtime trace of an SVG as well).
For the other responses, thanks for your links. Some of them weren't applicable as they were for spectrographic drawing (which I knew a little about from Aphex Twin), but wasn't what I was looking for, even though that's also really cool and maybe educates some other cool sound/art for people who stumble across it :) So thanks for replying!
posted by symbioid at 5:13 PM on January 12, 2022
Which helps understand the drawing process, and mentally I see now (especially when he gets to the person breakdown the x,y values of the vector graphics) how it all works (would work in reverse).
So I think I should be able to grab an SVG and get the x,y components from that (especially if I'm not drawing the art as I do it, though I'm sure there's a way to do a realtime trace of an SVG as well).
For the other responses, thanks for your links. Some of them weren't applicable as they were for spectrographic drawing (which I knew a little about from Aphex Twin), but wasn't what I was looking for, even though that's also really cool and maybe educates some other cool sound/art for people who stumble across it :) So thanks for replying!
posted by symbioid at 5:13 PM on January 12, 2022
Your main issue, I would expect, will involve working out how to turn an arbitrary SVG into a path that recreates the same shape as a single continuous stroke. Because until you have that, you've got nothing to feed into a Fourier transform.
A normal SVG drawing is just a collection of shapes, each of which defines some kind of relation between a set of X coordinates and their corresponding Y coordinates using the equations for straight lines and arcs and Bezier curves and whatnot. To convert those shapes to input suitable for a Fourier transform, you'd need to massage the equations that relate X and Y for each of their constituent segments into a single (albeit probably piecewise) function that maps time to X, and another that maps time to Y.
This is exactly the kind of problem that CAD software needs to solve in order to convert a design drawing into a toolpath for a CNC router, so if your problem were my problem then that's where I'd start looking for algorithms.
Once you have that continuous stroke, you might find that you can make interesting audio just by sampling it at regular time intervals and then treating the resulting X and Y coordinate streams as audio without even needing to involve Fourier at all. Because if you did feed such a set of samples into a discrete Fourier transform, then fed the outputs of that into the inverse Fourier transform in order to render it as an animation, the only thing you're gaining is that nifty chain of rotating arrows that visualizes the transformed version of the data in such animations.
The drawing itself is going to look very nearly the same whether it's the input to or the output from a DFT + inverse DFT pair of transforms - that's kind of the point of them - and of course the same would apply to any audio streams derived from the coordinates of the points along the drawing's stroking path.
posted by flabdablet at 8:42 AM on January 13, 2022 [1 favorite]
A normal SVG drawing is just a collection of shapes, each of which defines some kind of relation between a set of X coordinates and their corresponding Y coordinates using the equations for straight lines and arcs and Bezier curves and whatnot. To convert those shapes to input suitable for a Fourier transform, you'd need to massage the equations that relate X and Y for each of their constituent segments into a single (albeit probably piecewise) function that maps time to X, and another that maps time to Y.
This is exactly the kind of problem that CAD software needs to solve in order to convert a design drawing into a toolpath for a CNC router, so if your problem were my problem then that's where I'd start looking for algorithms.
Once you have that continuous stroke, you might find that you can make interesting audio just by sampling it at regular time intervals and then treating the resulting X and Y coordinate streams as audio without even needing to involve Fourier at all. Because if you did feed such a set of samples into a discrete Fourier transform, then fed the outputs of that into the inverse Fourier transform in order to render it as an animation, the only thing you're gaining is that nifty chain of rotating arrows that visualizes the transformed version of the data in such animations.
The drawing itself is going to look very nearly the same whether it's the input to or the output from a DFT + inverse DFT pair of transforms - that's kind of the point of them - and of course the same would apply to any audio streams derived from the coordinates of the points along the drawing's stroking path.
posted by flabdablet at 8:42 AM on January 13, 2022 [1 favorite]
This thread is closed to new comments.
I also found this device that can turn an image into a wavetable, but that's not really what you wanted.
I am sure that you can probably do what you envision in Max for Live, but I wouldn't really know where to start.
posted by ob1quixote at 10:07 PM on January 11, 2022 [1 favorite]