Best books/resources for learning audio programming

April 17, 2017 6:53 PM

I've been programming in Ruby (and other scripting languages) for a few years now—and hey, I've even been paid to do it for two of them. But my dream is creating music and audio related products and toys—think synthesizers, samplers, etc. I play with my OP-1 from Teenage Engineering and think "I want to make things like this."

I know I have to learn a lower level language, but what is my best bet? C++? Also, are there any great books/resources related to learning a language like C++ with a focus on audio synthesis, sampling, effects, etc?

Thanks in advance.

posted by defenestration to Computers & Internet (16 answers total)

I don't know very much about the intersection of programming and audio, but it's not at all obvious to me that you would want a low-level language for music and audio work. Here's a guy who's doing cool stuff with Ruby, for example.
posted by firechicago at 8:03 PM on April 17, 2017 [1 favorite]

Are you talking about programming embedded devices? Like signal processing sorts of things? I don't work in the area myself, but my understanding its that it's one of the areas where C is still commonly used.
posted by phack at 8:19 PM on April 17, 2017 [2 favorites]

Last I knew, VST plugins were largely C++, but that make just have been a function of whatever SDKs people used back then.
posted by rhizome at 8:46 PM on April 17, 2017 [2 favorites]

Web audio has come a long way in a few years. If you're thinking more on a software side of things I'd recommend looking into that (and with Electron it's now easy to make standalone non-web applications with the same tech). Here's a 30 mins conference talk on web audio with JavaScript (disclaimer - I ran the conference where this talk was given).

If you want to be creating and programming hardware, you'll need to look at electrical engineering and probably C, as phack says.
posted by jzed at 8:48 PM on April 17, 2017 [2 favorites]

I'd maybe start off with something like Reaktor or Max/MAP before diving into the programming side of things - it lets you build virtual instruments which is very much related to building real electronic instruments, at least conceptually.
posted by Jon Mitchell at 10:00 PM on April 17, 2017 [2 favorites]

Response by poster: I'm interested in all of the above, but just to confirm I am particularly interested in creating embedded devices with physical hardware. That is my long term goal.

Thanks for the recommendations so far. I think I'm going head down the path of learning C for that side of things. And perhaps play with JavaScript and web audio for the non-hardware side of things. I also plan to read and explore the code of Sonic Pi, as it's open source and written in Ruby, as far as I know.

Any educational resources—books, tutorials, videos, etc—that would be useful for those two different tracks would be much appreciated. Thanks!

Thanks!
posted by defenestration at 10:00 PM on April 17, 2017

I haven't delved into this sort of thing at all, but I've seen a few demonstrations / talks about Overtone, which is music processing in Clojure, and it looks really cool. Clojure isn't hard to learn if you know Ruby. There are some interesting talks online if you google "clojure music". It's pretty far from embedded programming, though.

I agree that for embedded programming you're probably best off learning C. The cool kids are moving to Rust these days, but if your goal is "make something" and not "learn a cool language" C is probably best, and once you know it you can conceivably use it for hardware interfaces that you control from a higher-level language.
posted by whir at 11:27 PM on April 17, 2017 [1 favorite]

Best answer: The reason for using C/C++ is, essentially, performance, and more specifically predictable performance.
Ruby, Javascript, Java, C# and basically any other higher-level language uses garbage collection for memory management. The problem with garbage collection is that, at essentially random intervals, execution of your entire program is halted while cleanup happens - for audio code, this isn't really acceptable as you need to keep feeding data to your audio hardware at a consistent, predictable rate.

That's not to say it's impossible - if your buffer sizes are large enough, and you can accept the latency cost of having a large buffer size (i.e., you're not going to be trying to synchronise with other instruments and/or you're not trying to play it in real time with a keyboard or something) then you can probably do something with Ruby/JS/etc. - but if you're after anything approaching real-time "performable" software, C/C++ is the way to go.
posted by parm at 3:28 AM on April 18, 2017 [3 favorites]

Response by poster: parm, that is exactly why I was thinking I should learn a lower level language. I want to create things that are performable in real-time, on embedded devices—physical hardware. Thank you for confirming my suspicion.

whir, I've actually been been messing around with Rust for a bit, and it's really cool. But I'm thinking C/C++ would make more sense, because my dream is to not only create/engineer these things, but to do it for a living. And my assumption is that jobs for doing that in Rust are not available the way the are for C/C++. At least for now.

Does anyone have any recommendations for a good books for learning C, perhaps even geared toward someone with experience in a higher level language such as Ruby? Bonus points if it touches on audio/music programming.
posted by defenestration at 5:33 AM on April 18, 2017

Best answer: Have you looked at Pure Data? That's what they use for the Pocket Piano and Organelle, which people seem to like, and they are definitely embedded software on performance-ready hardware.
posted by SaltySalticid at 7:00 AM on April 18, 2017 [3 favorites]

Response by poster: SaltySalticid, great point! I've wanted the Organelle for a while anyway, and that could be a nice place to begin experimenting. Looks like I'm going to be spending some of my tax return on that... heh.

I also want to begin playing around with Max as well.
posted by defenestration at 9:14 AM on April 18, 2017

Best answer: Seconding Pure Data. I know it works reasonably well with the Raspberry Pi because I attended a workshop where Miller Puckette (author of Pd & Max) gave a crash course in using Pd with the RPi years ago, and that was with the RPi model 1 B, I think, I'm sure the capabilities of the recent ones make it smoother. I was also impressed with Puckette as a teacher so it's a goal of mine to work through his Theory And Technique of Electronic Music, which covers a lot of synthesis with Pd (and hey, you can buy it on Amazon, or you can grab a pdf).

On top of that, if you decide you want to graduate from Pd's patch/flow interface, you can learn to work with libpd to do stuff in C, and I'd bet there are probably some bindings to other languages.
posted by weston at 9:51 AM on April 18, 2017 [2 favorites]

Best answer: If your end goal is embedded software then C/C++ is definitely the right way to go. However if you want to get up and running, there are plenty of options for higher level languages that use C/C++ audio processing libraries for the heavy-lifting signal processing.

Others have already recommended learning Web Audio API stuff but here's some other stuff I'd recommend looking into -

MAX/MSP - more of a GUI based visual programming language for audio processing, which can also work great with hardware controllers (plenty of Arduino / Raspberry Pi type projects based on MAX/MSP)
CSound - Pretty low level language for programming audio based on C. Usually used for building synths and sounds from scratch
librosa for Python - Great for music analysis. It's a set of C libraries with Python wrappers so very easy to get started. Check out musicinformationretrieval.com for some ideas
AudioKit for iOS - I'm currently learning iOS programming and have been using AudioKit which is a framework using C/C++ processing with a Swift interface for easy integration into iOS projects. There's also Xcode Playgrounds for playing around with code in realtime - great for experimentation and learning.
posted by TwoWordReview at 9:00 AM on April 19, 2017 [1 favorite]

Best answer: As for books. Two that I've used that I'd recommend are Principles of Digital Audio by Ken Pohlmann and Programming Embedded Systems in C and C++ by Michael Barr. I'm reasonably confident I wouldn't have my current job without having had those two books as resources at the time.
posted by TwoWordReview at 9:13 AM on April 19, 2017 [2 favorites]

Response by poster: Thank you so much for the insight and recommendations, TwoWordReview!
posted by defenestration at 5:55 PM on April 19, 2017

Best answer: No problem! I thought of some more stuff btw

CCRMA (Center for Computer Research in Music and Acoustics) at Stanford hold workshops every summer and last year they held one on Stompbox design using Pure Data. The notes for that course are on their Wiki so that should be a goldmine of resources and information.
posted by TwoWordReview at 8:44 AM on April 20, 2017 [1 favorite]