wombflash forest

Recorded Music as Object (and Process)

2011-11-07T14:14:00.000-08:00

1. When I play music with another person, at my best I have only one goal-- to play with them. It's the same with an instrument, which has its own unique properties that suggest rhythms, melodies, forms, etc. It's the same with all objects in all situations-- everything essentially functions as a collaborator. Anything can always surprise us if we allow it to-- love the collaborator and whatever surprises it might bring.

2. Recordings are used as functional objects in the lives of listeners; in practice, this diminishes their intrinsic meanings and elevates the meanings of whatever it is that they're being used for. To play a recording is to play music. There are no good recordings or bad recordings, only good or bad playings-- everything is useful in some context. (Everything is useful in every context; with all objects and in all situations, anything can be listened to, reacted to, played with).

3. Brian Eno explained an idea of his somewhere-- over the course of our whole lives, we as individuals only ever listen to one piece of music, a composite of everything we've ever heard. A long time ago, the sections of an individual's "one piece" would likely have been made up of mostly "complete" pieces of music-- an entire concert, an entire recording, etc. Now, most sections of our individual pieces are made up of musical "fragements"-- surfing the internet to find something to our liking, walking in and out of environments with music playing in the background, excerpts of music used to accompany other media, samples, etc. (In practice, fragments and repetitions).

4. Recordings are used as functional objects in the lives of listeners; how we play with a recording is more important than the recording itself.

5. Music software is lowering the barrier of entry to making music. Already, applications exist that give "non-musicians" expressive control of musical materials without their needing to practice a technical craft. One particular craft is still necessary, though, as it always has been-- one of focused listening.

6. Sampling is a process of playing with recordings as materials: one-way, asynchronous improvisation. At one point I felt that the most important use of this process would be an answer to the question "how can these I use these samples?"; now, I prefer to ask "how can these samples use me?"

7. How we play is more important than what we play and it doesn't matter how we play as long as it's play.

8. A long time ago, the sections of an individual's "one piece" would likely have been made up of mostly "complete" pieces of music-- blocks-- and most perceived meanings would have emerged from the completeness of the blocks themselves. Composers would have had a different experience, tending to listen to blocks on lower levels, the "complete" pieces as composites of smaller blocks, musical materials (instruments, compositional devices, etc.). Now that "complete" pieces have themselves become blocks, "nonmusicans" can become composers with focused listening and loving responses to the blocks at hand. The composites (pieces) which musical materials of the past have produced for us have become our own musical materials.

9. Now we can organize recorded materials, whatever they are, in such a way that both respects possible uses implied by their forms (to love our materials and the ways they surprise us), and respects our own inclinations, our own musical responses to the unique places we occupy in our "one piece".

10. (amor fati, "love of fate")

Soundtracks 1

2011-07-18T13:22:00.000-07:00

Video/computer games (most? all?) function similarly enough in time to the way that music does (particularly improvised music) that they can themselves be considered music, or, at the very least, musically meaningful play structures. I've thought about this idea enough now that I'd consider it a truism, and that it isn't apparent to everyone seems due, in large part, to the fact that game soundtracks have so far done very little to reveal the extent of this musicality.

To be clear here, I think there's a simple formula that can be followed, that has not yet been pursued with any seriousness (or playfulness), that will help begin to reveal the music that lies latent in games:

For every change of state in a game, there should be a corresponding change of state in its soundtrack.

As presented, this is hardly a novel idea except for the inclusion of one word: "every" (which can be substituted with "as many as possible" when dealing with technical limitations).

Recent games such as Mario Galaxy and Portal 2 (and more) have done some exceptional micro-studies in musical interactivity, with certain segments/levels mapping game events and processes to the soundtrack exactly the way that I'd like. However, they can't have mapped more than, say, 5% of game events to musical events, and as such, the musical interactivity becomes a novelty, rather than, as it ought to be, the articulation of a new expressive language.

It's as if, during a game of basketball, there was a 5 minute "experimental" interlude that accompanied all players' actions with a dynamic/improvised soundtrack by the pep band-- this would, without a doubt, be my favorite bit of the game, but it would also be a disappointment in that it didn't take that idea as far as it would stretch, to establish a new kind of basketball, an entirely new formal language, an improvised ballet of a sort.

Or, if this example is, itself, too novel, it's also as if early humans making vocal sounds, developing a spoken language of signs, for a moment produced sustained tones in unison, singing a perfect fifth--the beginnings of harmony--and then laughed it off, surprised at the unexpected beauty of it all, yet unwilling to continue to explore for longer than 10 seconds, unwilling to discover the language of music itself.

As pioneers of this new language-- musical gameplay-- games such as Mario Galaxy and Portal 2 ought to be celebrated; as ideals, however, they ought to be tossed aside as garbage, having done only a small fraction of the work (and play) that's necessary.

Elements of Music as Elements of Games

2011-06-13T17:40:00.000-07:00

1. Rhythm

2. Form

3. Timbre

4. Harmony

5. Melody

6. Dynamics

7. Texture

The goal here is to identify elements of games that function similarly, or identically, to each of these musical elements. I'm hesitant to counter the conventional wisdom that games need their own critical vocabulary in such an outright way as pushing the music-game relationship this far (melody in gameplay?), but it's not without purpose. My hope is that by studying and exploring the relationship between the two forms, game makers might be able to better learn from pieces of music the ways in which abstract meanings (which "point" at nothing, but are often the most profound) can be created and used in their designs. At the same time, I hope that players might tune into musical aspects of play that they otherwise wouldn't have.

1. Rhythm

This is the most obvious parallel, rhythm being basically identical (I think? tell me if I'm wrong) in both games and music. Wikipedia defines rhythm as "the arrangement of sounds and silences in time", which I'll abstract further as "the arrangement of events in short stretches of time" (the arrangement of "long" stretches defining form). Mario's jumps, the spectacular 18th level of Portal (see video, starting at 8:30), Braid's time reversal, and on and on: sequences of game events regularly present us with unique, highly memorable rhythms. Player input contributes to game rhythm as does non-player object behavior and placement (the arrangement of matter in space is a powerful way of suggesting a rhythm; the relationship between architecture and music-- can we not dance about architecture?).

For more info: I write about game rhythm in my old thesis, Kirk Hamilton writes about it in his recent column, and I'm sure others have as well. Let me know if you're aware of any other good sources on this.

2. Form

The arrangement of events in long stretches of time. Obviously, there's some crossover between this and rhythm, with slow rhythms (chess) and quick forms (Warioware) moving at a similar pace, and becoming, maybe, indistinguishable (not that those two games are at all indistinguishable-- just that their pacing might be hard to categorize as either distinctly rhythmic or formal). Aside from its time sub-dividing, what seems unique to me about form, and musical form in particular, is the way it relates events (motifs) from different points in a composition to one another, creating new meanings out of the simple processes of repetition and variation. Musical form has a lot to do with narrative form, and has been used quite extensively in this guise. Ocarina of Time is a great example of a two-part form, with part 2 functioning as an extended variation of part 1. The Mario games do repetition and variation wonderfully, with old mechanics constantly being re-contextualized by a level's design.

There's so much to study here, and I hope to return to it in greater depth... maybe with a critique of Jonathan Blow's Raspberry, which I recently replayed--it's a game that does an admirable job of consciously creating its own rhythmic and formal language and developing it musically.

3. Timbre

This is the first element that seems to be pushing the relationship a bit-- in fact, quite a lot. Timbre, also known as "tone color," is defined by Wikipedia as "the quality of a musical note or sound or tone that distinguishes different types of sound production, such as voices or musical instruments." (Despite both being stringed instruments, a guitar and piano sound different-- they have different timbres). Unlike rhythm and form, which exist in games in much the same way that they do in music, timbre, as defined above, is incapable of existing in games, being a physical property of sound. Still, if we go backwards a bit, and pick apart our definition/understanding, we may find a fitting analogy/parallel.

I don't know anything about physics, but, using my fingers and common sense, it seems clear that timbre is a function of touch in sound-- certain tactile qualities manifesting themselves sonically. Pianos and guitars sound different because they use different types of string, one is hammered with felt while the other is plucked or strummed, one is much larger than the other, etc. So, while the sonic aspects of timbre may be untransferable to an understanding of games, the tactile element is certainly very relevant. Walking on different surfaces in Mario games (excuse my repeated use of these as examples), like ice, sand, and honey, there is a distinctive tactile experience that, if not a kind of timbre itself, is certainly very much like the played experience of producing different timbres. Bringing back the old analogy from my thesis, the mechanics of a game can be likened to an instrument in a piece of music; the feel of those mechanics (or how they interact with the world, i.e. honey), being the feel of instrument, is an experience which has everything to do with timbre.

Watch this Derek Bailey clip, and go to a guitar to make all those sounds. It's fun--it feels great-- like walking through sand, ice, and honey.

4. Harmony

Harmony as defined by pitch relationships has no place in a gameplay analysis. However, there is, again, a broader definition which could theoretically be applied with success. Pitch harmony describes how fast one pitch is vibrating compared to another. A pitch an octave above another is vibrating twice as fast. A pitch a perfect fifth (the first interval of "twinkle twinkle little star") above another is vibrating 1.5 times as fast. &c&c... So, pitched harmony can be described by simple (sometimes, though sometimes not: read up here) ratios.

The idea with applying this to games is that the speed of these "pitches" can be slowed down dramatically, with the "vibrating" units of time producing rhythms (or forms) instead of pitches. To do this, repetitions (of events) need to happen less than ~10 times per second as opposed to, say, 440 (this is how many times per second an A below middle C vibrates).

Though it's not interactive, there's interesting precedent in the visual harmony pieces of James Whitney. See the video below, and the whitney music box (this reveals the harmonic principles really nicely). He's got a good book on the subject, too, though it looks like it's sort of rare... some libraries ought to have it.

To my knowledge, no games have used this device consciously. However, it wouldn't be difficult to implement as a visual/spatial device, or one based on event timings.

5. Melody

I've got nothing to say here except that melodies in the traditional sense perform a kind of free movement within a harmonic space. So, shifting spatial/rhythmic harmonies could potentially produce a similar effect? This really is pushing this analogy further than it wants to go, I think... so, let's stop there.

6. Dynamics

How loud or quiet a sound is. Again, not applicable unless we consider the processes used by an instrumentalist to create a loud or quiet sound. This is generally controlled by the intensity of input. The speed of swinging a Wiimote, or how much finger is pressed down on a touch screen (this has been used to simulate velocity), or how quickly an analog stick is moved from point 0 to point 1 could be considered fitting comparisons.

7. Texture

Wikipedia calls it "the way melodic, rhythmic, and harmonic materials are combined in a composition, thus determining the overall quality or sound of a piece." I'd add to that list timbre and dynamics.... so, everything except form; texture describes what's happening in a given section of a form--what sounds are present? what are they doing? It often changes throughout a piece:

It's hugely relevant for games, I think, in that it synthesizes all of these musical ideas into a whole: what objects are present? what are they doing? how do their actions (including the players') relate to one another?

Game textures parallel musical textures in fairly intuitive ways. I forgot if I'd read him describe it in this way, but Steph Thirion's Eliss is a great example of counterpoint in games, insofar as it has the player controlling multiple objects, performing a variety of actions simultaneously. Pikmin sort of feels like you're directing a Big Band. Fighting games are often thought of by skilled players as resembling musical duets or conversations...

The analogies could go on and on, but the basic idea is out there: a game's texture consists of what objects are present and what they're doing. This is about as simple as it can be, but there's so much that music has and continues to do with this idea, that it would be a shame not to listen.

***

I've not been able to go into much depth with any of these elements, though I hope I've made an alright case for considering them legitimate aspects of games (except melody). Of course, lots of games have most of these elements, and many have all of them-- it's because of this that I consider them pieces of music... though I don't consider many to be great pieces of music. I'm not sure what needs to happen to change this (though I have a sense it has to do with a shifting possibility space... more on that later), but I think that studying how music does what it does and how that relates to game designs should prove helpful.

And on top of this, how the actual sounds being produced by a game contribute to the musicality of the experience is huge... so, more on that in a bit.

Musical Play

2011-06-12T16:24:00.000-07:00

In the fall of 2009, I wrote my undergraduate senior thesis on certain aspects of the relationship between games and music; the paper can be found here, though since writing it, I've not been very happy with it, for a variety of reasons. It's poorly written. The form reflects a messy marriage of my own goals and the project's official requirements. It's fairly boring. Here's a brief summary, which may not always be clear.

1. Games are not just similar to, but, in fact, are music: non-aural music, in the same way that film, dance, and other time-based media are. I think I made this claim having just learned about visual music and wanted to run with that idea... One of the implications of this is that, once something is music, it's irresponsible in a way to ignore that. People that don't understand the musicality of film are going be lousy film-editors, non-musical dancers are going to disappoint... same thing with games, is what I was trying to point out.

2. As pieces of music, games function as both instrument and composition. A game's mechanics are its instrumental aspect, in that they set absolute boundaries, "walls" in the possibility space which cannot be ignored. A game's rules are its compositional aspect, artificial boundaries which can be ignored, but which, hopefully, ask the player to explore interesting/meaningful ways of playing that s/he might not have otherwise. This is one of my favorite parts of the paper, though I fail to address a lot of things that I wish I had spent time with... For instance, the fact that I've developed a kind of ethic which favors instrument over composition, the absolute over the artificial-- and, eventually, freedom over form. Or the fact that, despite what I said about my freedom ethic, these concepts don't actually break down into so clean a binary as I might like-- that the grey areas are where some of the most interesting stuff lies.

3. A value judgment: I consider musical play to be more meaningful than game play. This is because musical play has only aesthetic goals, focused in the present, while game play has competitive goals, focused in the future. I've never really liked "games" proper all that much, so this is an undeniably biased point of view... still, consider those ideas about time, and ask yourself what you value. These types of play are psychological states in the player as opposed to design decisions, though the two ought to be related, I think.

4. Finally, I conclude with a lazy taxonomy of common game rhythms which basically breaks down to a distinction between the metered and the free. My original outline had a lot more going for it here, and I hope to come back to the idea and do it better justice.

My goal with this blog is to continue to explore these and other related ideas.