Elements of Music as Elements of Games

1. Rhythm

2. Form

3. Timbre

4. Harmony

5. Melody

6. Dynamics

7. Texture

The goal here is to identify elements of games that function similarly, or identically, to each of these musical elements. I'm hesitant to counter the conventional wisdom that games need their own critical vocabulary in such an outright way as pushing the music-game relationship this far (melody in gameplay?), but it's not without purpose. My hope is that by studying and exploring the relationship between the two forms, game makers might be able to better learn from pieces of music the ways in which abstract meanings (which "point" at nothing, but are often the most profound) can be created and used in their designs. At the same time, I hope that players might tune into musical aspects of play that they otherwise wouldn't have. 

1. Rhythm

This is the most obvious parallel, rhythm being basically identical (I think? tell me if I'm wrong) in both games and music. Wikipedia defines rhythm as "the arrangement of sounds and silences in time", which I'll abstract further as "the arrangement of events in short stretches of time" (the arrangement of "long" stretches defining form). Mario's jumps, the spectacular 18th level of Portal (see video, starting at 8:30), Braid's time reversal, and on and on: sequences of game events regularly present us with unique, highly memorable rhythms. Player input contributes to game rhythm as does non-player object behavior and placement (the arrangement of matter in space is a powerful way of suggesting a rhythm; the relationship between architecture and music-- can we not dance about architecture?).

For more info: I write about game rhythm in my old thesis, Kirk Hamilton writes about it in his recent column, and I'm sure others have as well. Let me know if you're aware of any other good sources on this.

2. Form

The arrangement of events in long stretches of time. Obviously, there's some crossover between this and rhythm, with slow rhythms (chess) and quick forms (Warioware) moving at a similar pace, and becoming, maybe, indistinguishable (not that those two games are at all indistinguishable-- just that their pacing might be hard to categorize as either distinctly rhythmic or formal). Aside from its time sub-dividing, what seems unique to me about form, and musical form in particular, is the way it relates events (motifs) from different points in a composition to one another, creating new meanings out of the simple processes of repetition and variation. Musical form has a lot to do with narrative form, and has been used quite extensively in this guise. Ocarina of Time is a great example of a two-part form, with part 2 functioning as an extended variation of part 1. The Mario games do repetition and variation wonderfully, with old mechanics constantly being re-contextualized by a level's design.

There's so much to study here, and I hope to return to it in greater depth... maybe with a critique of Jonathan Blow's Raspberry, which I recently replayed--it's a game that does an admirable job of consciously creating its own rhythmic and formal language and developing it musically.

3. Timbre

This is the first element that seems to be pushing the relationship a bit-- in fact, quite a lot. Timbre, also known as "tone color," is defined by Wikipedia as "the quality of a musical note or sound or tone that distinguishes different types of sound production, such as voices or musical instruments." (Despite both being stringed instruments, a guitar and piano sound different-- they have different timbres). Unlike rhythm and form, which exist in games in much the same way that they do in music, timbre, as defined above, is incapable of existing in games, being a physical property of sound. Still, if we go backwards a bit, and pick apart our definition/understanding, we may find a fitting analogy/parallel.

I don't know anything about physics, but, using my fingers and common sense, it seems clear that timbre is a function of touch in sound-- certain tactile qualities manifesting themselves sonically. Pianos and guitars sound different because they use different types of string, one is hammered with felt while the other is plucked or strummed, one is much larger than the other, etc. So, while the sonic aspects of timbre may be untransferable to an understanding of games, the tactile element is certainly very relevant. Walking on different surfaces in Mario games (excuse my repeated use of these as examples), like ice, sand, and honey, there is a distinctive tactile experience that, if not a kind of timbre itself, is certainly very much like the played experience of producing different timbres. Bringing back the old analogy from my thesis, the mechanics of a game can be likened to an instrument in a piece of music; the feel of those mechanics (or how they interact with the world, i.e. honey), being the feel of instrument, is an experience which has everything to do with timbre. 

Watch this Derek Bailey clip, and go to a guitar to make all those sounds. It's fun--it feels great-- like walking through sand, ice, and honey.

4. Harmony

Harmony as defined by pitch relationships has no place in a gameplay analysis. However, there is, again, a broader definition which could theoretically be applied with success. Pitch harmony describes how fast one pitch is vibrating compared to another. A pitch an octave above another is vibrating twice as fast. A  pitch a perfect fifth (the first interval of "twinkle twinkle little star") above another is vibrating 1.5 times as fast. &c&c... So, pitched harmony can be described by simple (sometimes, though sometimes not: read up here) ratios.

The idea with applying this to games is that the speed of these "pitches" can be slowed down dramatically, with the "vibrating" units of time producing rhythms (or forms) instead of pitches. To do this, repetitions (of events) need to happen less than ~10 times per second as opposed to, say, 440 (this is how many times per second an A below middle C vibrates).

Though it's not interactive, there's interesting precedent in the visual harmony pieces of James Whitney. See the video below, and the whitney music box (this reveals the harmonic principles really nicely). He's got a good book on the subject, too, though it looks like it's sort of rare... some libraries ought to have it.

To my knowledge, no games have used this device consciously. However, it wouldn't be difficult to implement as a visual/spatial device, or one based on event timings.

5. Melody

I've got nothing to say here except that melodies in the traditional sense perform a kind of free movement within a harmonic space. So, shifting spatial/rhythmic harmonies could potentially produce a similar effect? This really is pushing this analogy further than it wants to go, I think... so, let's stop there.

6. Dynamics

How loud or quiet a sound is. Again, not applicable unless we consider the processes used by an instrumentalist to create a loud or quiet sound. This is generally controlled by the intensity of input. The speed of swinging a Wiimote, or how much finger is pressed down on a touch screen (this has been used to simulate velocity), or how quickly an analog stick is moved from point 0 to point 1 could be considered fitting comparisons. 

7. Texture

Wikipedia calls it "the way melodic, rhythmic, and harmonic materials are combined in a composition, thus determining the overall quality or sound of a piece." I'd add to that list timbre and dynamics.... so, everything except form; texture describes what's happening in a given section of a form--what sounds are present? what are they doing? It often changes throughout a piece:

It's hugely relevant for games, I think, in that it synthesizes all of these musical ideas into a whole: what objects are present? what are they doing? how do their actions (including the players') relate to one another?

Game textures parallel musical textures in fairly intuitive ways. I forgot if I'd read him describe it in this way, but Steph Thirion's Eliss is a great example of counterpoint in games, insofar as it has the player controlling multiple objects, performing a variety of actions simultaneously. Pikmin sort of feels like you're directing a Big Band. Fighting games are often thought of by skilled players as resembling musical duets or conversations...

The analogies could go on and on, but the basic idea is out there: a game's texture consists of what objects are present and what they're doing. This is about as simple as it can be, but there's so much that music has and continues to do with this idea, that it would be a shame not to listen.

***

I've not been able to go into much depth with any of these elements, though I hope I've made an alright case for considering them legitimate aspects of games (except melody). Of course, lots of games have most of these elements, and many have all of them-- it's because of this that I consider them pieces of music... though I don't consider many to be great pieces of music. I'm not sure what needs to happen to change this (though I have a sense it has to do with a shifting possibility space... more on that later), but I think that studying how music does what it does and how that relates to game designs should prove helpful.

And on top of this, how the actual sounds being produced by a game contribute to the musicality of the experience is huge... so, more on that in a bit.

Musical Play

In the fall of 2009, I wrote my undergraduate senior thesis on certain aspects of the relationship between games and music; the paper can be found here, though since writing it, I've not been very happy with it, for a variety of reasons. It's poorly written. The form reflects a messy marriage of my own goals and the project's official requirements. It's fairly boring. Here's a brief summary, which may not always be clear. 

1. Games are not just similar to, but, in fact, are music: non-aural music, in the same way that film, dance, and other time-based media are. I think I made this claim having just learned about visual music and wanted to run with that idea... One of the implications of this is that, once something is music, it's irresponsible in a way to ignore that. People that don't understand the musicality of film are going be lousy film-editors, non-musical dancers are going to disappoint... same thing with games, is what I was trying to point out.

2. As pieces of music, games function as both instrument and composition. A game's mechanics are its instrumental aspect, in that they set absolute boundaries, "walls" in the possibility space which cannot be ignored. A game's rules are its compositional aspect, artificial boundaries which can be ignored, but which, hopefully, ask the player to explore interesting/meaningful ways of playing that s/he might not have otherwise. This is one of my favorite parts of the paper, though I fail to address a lot of things that I wish I had spent time with... For instance, the fact that I've developed a kind of ethic which favors instrument over composition,  the absolute over the artificial-- and, eventually, freedom over form. Or the fact that, despite what I said about my freedom ethic, these concepts don't actually break down into so clean a binary as I might like-- that the grey areas are where some of the most interesting stuff lies.

3. A value judgment: I consider musical play to be more meaningful than game play. This is because musical play has only aesthetic goals, focused in the present, while game play has competitive goals, focused in the future. I've never really liked "games" proper all that much, so this is an undeniably biased point of view... still, consider those ideas about time, and ask yourself what you value. These types of play are psychological states in the player as opposed to design decisions, though the two ought to be related, I think.

4. Finally, I conclude with a lazy taxonomy of common game rhythms which basically breaks down to a distinction between the metered and the free. My original outline had a lot more going for it here, and I hope to come back to the idea and do it better justice.

My goal with this blog is to continue to explore these and other related ideas.