The notion of sound design as we know it has evolved over the last several decades to include musical, gaming, theatrical as well as cinematic applications. Dan Dugan, of the American Conservatory Theater (ACT) in San Francisco, was the first person to be called a sound designer. He devised ways of automatically adjusting levels to reduce feedback. In film, Walter Murch was the first person given the title of sound designer for his work on Apocalypse Now, a masterpiece of sound design and re-recording mix engineering. The helicopter scene is incredible! I have been privileged to collaborate with the great Don Preston in my group, TriAngular Bent. Don is an original member of Frank Zappa’s Mothers of Invention and the person who created the helicopter sound on his Moog for the film.
But even before the phrase existed, people have been using all sorts of mechanical and electronic methods to “design” new sounds. Luigi Russlo comes immediately to mind as the inventor of the intonarumori, a collection of noise-based machines for creating new kinds of sounds in the early 20th Century. Luigi Russlo Luigi Russlo Sound design is now accessible and possible for anyone with a computer. In this article, I will focus on some of the fundamental principles and approaches to sound design, audio processing plugins used by the pros, common applications and related career paths. Fundamental Principles and Approaches to Sound Design Purpose The function of a sound (or lack thereof) will dictate the methods used to create it in many circumstances. For example, if a sound designer is tasked with creating the sound of a fictional monster there is clearly no one sample that will suffice. Commonly, several animal recordings with various types of processing may be combined with synthesized sounds or even instrument samples. There are no rules for creating something that doesn’t exist. But there are expectations that are now embedded in the culture of media entertainment. Gunshots in films do not sound “realistic,” but rather they sound hyperrealistic with long reverb tails and perfectly balanced frequency spectrums. Even sports and nature are not immune to hyperrealistic sound design. Enhancing a Narrative One basic function of sound design is to enhance the narrative, whether it be for film, theater, gaming, etc. Some may suggest that sound is subservient to the image, but I prefer the word supportive. No one would deny that bad sound is one major hallmark of a bad film. While great sound might not be able to salvage a bad story, it will certainly make it less painful. Sound in gaming is a crucial part of the immersive experience. Sound design creation and implementation in the gaming industry is one of the most, if not the most, viable career paths for aspiring audio content producers. In this context, procedural or adaptive audio is used to realize an experience defined by user actions. While the sound design content is created in a similar way to traditional mediums, the implementation is non-linear and requires the use of middleware such as Wwise or FMOD to implement audio assets. The idea that sound needs to adapt to the actions of the user in realtime presents significant and unique challenges for designers due to the need to consider a dynamic spatial relationship between the user and the sound object. The more realistic the sounds as they relate to the imagery, the more immersive the experience. VR (virtual reality) adds yet another layer of complexity as head movements need to be considered as well. Sound for live theater might be considered the ancestor of modern sound design. Various mechanical devices have been used for thousands of years in theatrical productions. The Greeks invented various devices to create storm sounds that used brass balls and fabric draped over a rotating wheel. In modern theater productions, applications such as QLab or Ableton Live are used to trigger precomposed elements of music and sound design to enhance the theatrical action on stage. This could be as simple a dog barking in the distance, the sound of an urban soundscape or a gunshot. Musical Context In musical contexts, sound design can enhance a gesture or create novel textures not possible with traditional acoustic or electronic instrumentation. Using orchestral sounds combined with synthesized or sampled found objects is a way of creating hybrid sonorities that can be unique and unidentifiable while remaining linked to the real world in timbre and playability. Synth programming, with preset and sample pack creation is another possible income source for sound designers. While amazing instruments such as Omnisphere, Serum and u-he’s line of incredible synths offer enormous sonic variety and potential for composers, many don’t have the time or inclination to fully explore these products. This creates an opportunity for sound designers to fill the void. Sampling instruments are widely used in sound design since they can provide immediate access to many sounds from a simple MIDI keyboard or drum pad. Literally any sound can be mapped, layered, pitched, looped, sequenced and modulated using any one of a number of available samplers such as Native Instrument’s Kontakt, Sampler and Quick Sampler in Logic Pro, Sampler and Simpler in Ableton Live, etc. The market for third-party sampling instruments has exploded in the last decade for composers lacking the will or resources to create their own instruments, resulting in yet another opportunity for sound designers and virtual instrument makers. The use of real-time sampling, DJ gear, alternative MIDI controllers, custom electronics, circuit bent instruments and laptops in live music performance can arguably be considered a form of live sound design. You could say the relationship between musical improvisation and the performance of pre-composed material is analogous to traditional sound design and performance-based work. Sound for Sound’s Sake This phrase relates to the idea of art for art’s sake, a phrase purportedly coined by the French poet Théophile Gautier in the early 19th century “that expresses the philosophy that the intrinsic value of art, and the only ‘true’ art, is divorced from any didactic, moral, political or utilitarian function.” Extrapolating this idea we might consider creating or designing sounds simply for the pure joy of it. Taking the idea even further, we could imagine sound being freed from traditional musical relationships and form. Karlheinz Stockhausen’s idea of “moment form,” the work of John Cage, and Pierre Schaeffer’s musique concrète provide numerous examples that can redefine our understanding of sound and how it can be appreciated independent of familiar musical constructs. To say that sound for sound’s sake, without a corresponding teaching position, is a viable career path would be disingenuous. This is a creative direction that requires a day gig (at least for now). But these ideas have surely influenced contemporary sound design and mainstream music, and have gradually seeped into the consciousness of popular culture. Knowing the history of your medium is essential. With accessibility to landmark works that changed the course of music and our perception of sound just a few clicks away, there is every reason to explore. Sound Design Techniques Layering and Envelopes It is logical to propose that every sound we hear has a beginning, a middle and an end. In a sound design context this is a useful way of thinking about, deconstructing and constructing sounds. Similar to the idea of a four-stage ADSR envelope (attack, decay, sustain and release), we can consider each stage as just one component of a sonic event. In terms of amplitude, the attack, decay and release stages have a temporal component while the sustain portion is more about level. But things can be much more complex than using simple ADSR amplitude envelopes to construct a sound. Envelopes can be sources of modulation for any parameter in the signal chain including source generation, filtering and other modulators. Envelopes can have an infinite number of stages and can include looping algorithms, sustain points and rhythmic flexibility. While combining or layering several sounds with a shared amplitude envelope can yield amazing results, employing unique envelopes for each sound component can produce far more complex and evolving sonic textures or rich timbres not otherwise possible. Deconstruction and Reconstruction The example of FFT processing (Fast Fourier Transform) is worth considering at this point. It involves deconstructing a sound so that it can be represented by a collection of sine waves describable in terms of frequency, amplitude and relative phase. Once deconstructed, mathematical modifications can be made to particular components independently. This is how pitch can be shifted without changing duration and vice versa. Granular synthesis is a similar process whereby grains or little chunks of a sound can be treated independently in myriad ways such as pitch shifting, reversing and reordering. Consider the only pure sound in terms of harmonics (the sine wave), which includes just one measurable frequency — the fundamental. As soon as you combine just two of these pure waveforms you produce sum tones, difference tones and possible tremolo effects. Adding varying envelopes into the mix causes things get even more complex. This idea of achieving complexity from a few simple components also lies at the heart of simple FM (frequency modulation) synthesis, where two simple oscillators (a carrier and a modulator) can produce an astounding array of sounds and timbres. Synthesis Versus Sampling – Hybrid Approaches The combination of real-world samples and synthesized sounds is ripe with possibilities for sound design. Using the layering and envelope ideas mentioned above as well as additional processing, you can create engaging and evolving soundscapes and hybrid gestural events that incorporate both organic and electronic-based sources. This lends itself nicely to genres like sci-fi and gaming. Analog and Digital The resurgence of the analog aesthetic has been underway for many years now and shows no sign of slowing down. In 2020, vinyl sales surpassed CDs for the first time since 1986. Modular synthesis has been enjoying a renaissance since the introduction of the standardized Eurorack format by Doepfer in the late 90’s. And analog sounds are being used more and more for music and sound design. Having been raised in an analog world, the digital revolution seemed miraculous at the time and I never envisioned a return to old technology. But now, my studio is riddled with both analog gear and hybrid devices. After being immersed in digits for decades it was a breath of fresh air to hear pure continuous voltage again. Now I allow myself to explore both worlds independently and perhaps more interestingly, in combination. Both technologies have a particular sound, as do different instruments, and I see no reason to use one exclusive of the other. Spatialization One of the most fundamental aspects of sound design is creating a space for a sound to exist or creating a sonic environment. Two temporal processes to consider are reverberation and delay, both of which can include some sort of filtering, panning, and modulation. Putting aside surround sound and Dolby’s ATMOS systems for the moment, amazing results can be accomplished in a standard stereo field. Objects can be moved not just left to right, but front to back. Using binaural algorithms (typically restricted to headphone implementations) such as in Logic Pro, sounds can be located not just front/back and left/right, but behind, below and above the listener. Doppler effects can be applied to movement in the stereo field to heighten realism or create hyperreal motion. The contrast between sampled and synthesized sounds can be correlated in some ways with the difference between convolution and algorithmic reverb. The former uses impulse response (IR) files derived from samples taken in actual spaces while the latter relies on algorithmic computation to create spaces based on delay lines, filtering and other parameters. Convolution reverbs are arguably more appropriate for placing sounds in “real” spaces while algorithmic reverbs can be used to emulate vintage reverb technologies like plate and spring reverbs, or create impossible spaces that don’t exist in real life. Convolution reverbs and IR files can also be used to emulate vintage gear, speaker cabinets, amp circuits, etc. Algorithmic reverbs typically have more control over parameters and can offer extreme settings. Some engineers suggest cascading reverbs (placing them in series) to create smoother results. The creative use of reverb alone has enormous possibilities in sound design and it’s a well-known fact that everything sounds better with a little verb! Software There are so many great applications and audio processing plugins out there that are appropriate for sound design; it would be impossible to mention them all in this context. Below are just a few of my favorites at the moment (in no particular order), all of which I can wholeheartedly recommend. Playbox by Native Instruments This newly-released Kontakt-based instrument from Native Instruments is like nothing I’ve seen from the company before. It is loaded with randomization features which I love. In fact, the whole interface is designed around dice-like icons that encourage experimentation. There is an incredible number of preconfigured sounds, and users can select from 1301 presets or combine sounds by selecting from the 905 included samples. It has granular, lo-fi, phasing, delay, dynamic and spatialization effects, as well as modulation possibilities with two LFOs, ADSR envelope, sequencer and humanization. You can quantize the harmonic output to any major or minor key, change octaves and select half-time for tempo-synced processing if desired. The main page includes an XY pad that is assigned to the first two effect parameters based on the preset. The right side of the interface is populated with six bizarre 3-D textured dice icons which change to reflect the currently loaded samples. Categories include voices, noises, synths, instruments, bass and user-defined. NI Playbox In the Chord Matrix page you can program eight chords manually or by dragging and dropping a MIDI file. You can also drag and drop content from the matrix to a MIDI track in your session. NI Playbox 2 On the Samples page, you determine which sounds are activated by each key, and you can also drag and drop your own samples here! NI Playbox 3 The effects page includes all the audio processing effects and modulation controls mentioned earlier. Everywhere you look in Playbox you see the dice icons for randomization and discovering new sonic combinations of sounds and effects. Aside from the great onboard sounds, this device can breathe new life into your existing library of samples. Sound Particles Sound Particles Sound Particles is an extremely sophisticated standalone application for creating 3D audio. Inspired by the visual “particle systems” used to simulate things like fire, rain and dust, it is intended to assist sound designers in creating dynamic movement in 3D space. Projects can accommodate several simulated spatialization set-ups including everything from mono to Dolby Atmos 9.1 and Auro-3D (11.1). The software has been used in an impressive collection of films listed here. Although clearly designed for post-production in film, gaming and VR, I see no reason not to investigate the use of this software for advanced spatialization in music production, especially now that Atmos is becoming more accessible with native access in the latest Logic Pro version and elsewhere. The company also offers several audio plugins such as their Doppler + Air bundle, both designed to simulate distance and movement via frequency shifting and attenuation. As expected with any powerful software, the learning curve for Sound Particles is challenging. But as immersive audio gets better and better, this particle-based approach seems like a natural way to go. Falcon 2 by UVI NOTE: When this article was first published, there was an immediate comment lamenting the fact that I didn’t include Falcon by UVI in this round-up. I am the first to admit I have not tried every piece of sound design software currently available and I only write about things I have personally used. After the initial publishing date, UVI was gracious enough to provide me a copy of Falcon 2 to explore and I have to say it is an impressive piece of software. ADVERTISEMENT This instrument can be classified as a hybrid synth since it has so many approaches to sound design in one box including sampler capabilities, granular synthesis, wavetable synthesis, FM synthesis, additive synthesis, subtractive, synthesis, phase distortion synthesis and component modeling. It comes packed with 16 oscillators, 90 full-featured processing effects, 9 modulation generators, scripting processors for MIDI events, an analysis section that includes chord recognition and pitch detection and really way more than can be covered in this short description. With all that functionality, you would expect a bit of a learning curve. Happily, UVI has published several deep-dive YouTube videos that methodically explore this beauty. A quick perusal of the presets will convince you of the enormous potential this instrument offers. The multi-part and multi-layer approach guarantees the possibility of an infinite variety of unique sounds, textures and timbres. It is a sound design behemoth! UVI Falcon 2 Output This LA-based company has established a reputation for elegant interface design and superb processing and instrument plugins. Some of my favorites include: Portal – a granular-based processor. Thermal – offers saturation and distortion. Kontakt-based Hybrid Instruments: Analog Strings Analog Brass and Winds Arcade is a subscription-based instrument that allows access to a huge collection of loop kits that can be downloaded as desired. It employs an easy-to-use interface in which a two octave span of white keys trigger loop slices while the black keys will activate various processing and playback functions. There’s a lot more to it than I can explain in this article, but I suggest you check it out for yourself. Recently they launched Kit Generator, a function that creates a user-defined kit based on samples in the Arcade library or your own via a simple drag and drop. While other samplers can do auto-mapping as well, this feature takes full advantage of Arcade’s robust effects processing and playback manipulation, giving the user instant access and the ability to customize the instrument with a few clicks. Drag and drop one of your own tracks in and hear the magic. Really great! Output Arcade Native Instruments The ramifications of the recent merger of iZotope and Native Instruments remain unknown as of this writing, but hopefully they will combine their expertise in ways that benefit users through innovation. Below are a few of my go-to NI instruments and processors: Kontakt – the industry standard sampling platform that hosts hundreds of third-party libraries. Rise & Hit – contains user-programmable whooshes, impacts and transition effects. Driver – a unique saturation and distortion plugin with modulation possibilities. Kinetic Metal – a sample-based instrument capable of raw metallic textures with a steampunk GUI. Polyplex – a Reaktor-based percussion instrument with 8 voices and up to 4 layers per voice. NI Polyplex iZotope While NI’s forte has traditionally been focused on virtual instruments, their new partner, iZotope, has spent years carving out niches in the world of artificial intelligence and post-production. In terms of sound design these are my favorites: Trash – it’s ugly, it’s nasty, it’s deliciously angry. When you need to give a sound some hard-edged attitude, this thing is perfect! |
