I gave a talk last week in Montreal at MIGS, which followed this outline. The points I spoke about were more or less improvised within this framework/sequence, and I'm going to do the same thing with writing now, which is likely going to tunnel some ideas into a less conversational/more solipsistic hole, with things that i could write but might not say.... ohhh, etc.--
in any case-- annotations follow each slide, hoping to clarify them-- in general, hoping to to share some useful tactics.
I. Trying to find the IMAGE of the music/game identity
II. Lessons learned from specific games.
III. Future directions for research
RULE-of-THUMB / DOGMA ~~~~~ This is KEY ! The precedence for this statement is pretty huge, from the Pythagorean musical-numerical cosmologies (which consider the scaling categories COSMIC music, HUMAN music, and INSTRUMENTAL music-- only the last of which we'd still call by that name), thru all the hermetic tendrils that have flowered out of them..
Adam Harper has written on this here (http://rougesfoam.blogspot.com/2012/06/musical-radicalism-beyond-sonic-talk-at.html), and when the non-sonic image is kept in mind, the beautifully described 'progressive differentiation of Music Space' in his book Infinite Music starts to conceptually bind with the progressive differentiation of everything, more or less, a new Pythagoreanism for today, based on difference rather than identity? New geometries-- scale, paths, wiggles... Recalling some of the more old-fashioned understandings of what music is. Robert Fludd, old English alchemist, writes that:
"Music is the knowledge by which all worldly things are joined by unbreakable bonds and by which like is related to like by equal proportion in any object. This definition fits musica mundana, humana and instrumentalis"I like this! That music is the connective tissue of things, the principle of composition, assemblage considered broadly. The ground of things, insofar as Aleister Crowley's equation 0=2 can be read as ground...
But even without going into cosmologies, the simple existence of musical scores puts our belief in the primacy of sound in music to question.
Old men who are into musical aesthetics are very concerned with The Musical Work, which is this more or less wholly computable string of information that we are given in the score^^. There's the work and the performance, which are tangled but discrete, and the Work somehow manages to exist independently of sound-- this, regardless of whether it's intended to eventually guide the production of sound or not.
So, this is the music design TACTIC that the last hypothesis prepared the ground for. Just like we were starting to read Kandinsky's picture as a score, and just as we could do the same with dance notations-- it is possible to read ALL GAMES as dynamic scores already complete with the necessary time-structures, rhythmic information. The picture above is a clipping from the mario 64 manual, showing a handful of the core jump-mechanics. Anyone who has played can recall the different rhythms of different jumps. The triple jump, for example, where the rhythm is elastic-expansive, air time increasing with each additional hop.. Rhythm looks something like J - - , J - - - - , J - - - - - - - ..... Where "J" is for jump, and the dashes are airtime. It would be possible to create a spatialized notation of interactions in a game in this way, even if a bit absurd, as we would quickly require many more than the 2 dimensions that the page allows for, if we wanted to account not only for the time-structures of isolated interactions (which may often be accountable for using only 1 dimension, the path they follow), but also the more important combinations of mechanics that emerge in play, which will require a stacking N+1 dimensionality.
In any case, just based on the sequence of events and processes in a game and how these relate to the broader space of all possible sequences --from this, we are given the new 'meter' of game design, which has little to do with the evenly divided 4/4, 3/4 etc of much linear music-- rather, composed of metric 'downbeats' which are placed seemingly arbitrarily, by a kind of willful chance, the player's activity.
Music design takes this basic temporal architecture of any game, and 'hugs' it with a material-vibrational SKIN which is called the 'soundtrack.' This is just like a 'skin' for Winamp or whatever, in that it's at least in theory totally replacable -- the game organism can fully survive a skin graft without suffering any pain.
The skin needs to 'eat up' two concepts/disciplines to be counted as One -- sound design and composition. Musicality should exist in the the haptic-responsive aspects of the sound design as much as responsiveness/touch/immersion/nonlinearity should exist in the compositions.
TOUCH is the thread that holds these components together.
For this reason I add "game feel" to the list of things music design ought to be wholly tuned into. Game feel describes the concept of input-microrhythm, more or less, that Steve Swink has written about in his book. It is the game's time-architecture-- but zoomed in deep, where a whole rhythmic composition can unfold in 1 second or less-- how does the ground respond that is covered in honey? in ice? How quickly do we slide down a sticky wall in Knytt?
The game feel is like the musculature of the game-organism, which, being so close to the surface of the skin itself, makes itself known, haptic/visual, through the skin, and acts as a medium between the external world and the hidden internals, like the skeletal frame, which corresponds to the macro time-structures considered broadly
To treat ALL of these components musically can send us down a sometimes confusing path. We are trying to integrate the a meaningful aesthetic of both pieces of music and of instruments as if these were One Thing. Something that has beautiful sequence (regardless of the order of seqence) in addition to beautiful TOUCH/response.
Beautiful TOUCH has not been often acknowledged as one of the most important parts of music, because it is always tuned into from the PLAYER's perspective, much moreso than the listener's (even if listener-projection into the player is a very real thing). But the player knows well the importance of touch, and that, indeed, there are countless pieces of music that, while beautiful in their touch, for those involved in playing, did not seem so to the audience members who were not implicated in the causal source of the music in the same way (The opposite is also true-- beautiful sound-affects, ugly touch-- and this is especially true of much computer music today).
This is one of the greatest challenges of music design in light of musical developments considered broadly. To integrate an aesthetic of immanent touch, and necessarily transformation, into the existing aesthetics of progression, sequence, etc. Perhaps this is something that can only be done in videogames, or in other software spaces considered broadly. Spaces which, are they to become compositions, need to integrate something of the elastic-sequential aesthetics that videogames have really excelled at.
This is the question: can we come to terms with an understanding of instruments and compositions which are not at all describable in terms of a simple one-directional hierarchy?
As is the case today, instruments are used in compositions, and not the other way around. It is much more interesting, it seems, to ask how compositions can be used in instruments...
Scrubbing through samples is a basic way of practicing this idea today, that anyone can do. The instrument is the sample-space, which is the linear strip of information from the beginning of a piece to the end-- but the the instrument's haptic aspect is its capacity to move through this space, not in a straight line, but drifting from point to point, triggering events, new sequences, recombined as parts from the old dissolved whole. The material that is sampled is the composition that is used as a component part of the sampler-instrument.
Douglas Hofstadter's concept of the Strange Loop, or tangled hierarchy, is predated by the alchemical ouroboros (above^^), the snake eating its tail, and I believe this will prove to be a very powerful conceptual image we might want to consider carrying along with us to navigate these problems..
Once the strange loop is taken for granted, there need not be any difference between an instrument and a composition, because we will naturally assume that any instrument has its compositional aspect, its time-structures, and that any composition will have its instrumental aspect, its degrees of freedom, or haptic capacity to be played.
Even if we do this, it is impossible to ignore that they are both played, and it is hard not to be curious what is this PLAY that music and games have in common.
Some would say this-- that the shared use-word is deceptive-- that playing music and playing games mean totally different things.
& I do think there's something interesting to tunnel into here, namely the difference between aesthetic play with its unspoken Many goals which may converge into an unspoken One-- and game play with its explicitly spoken One goal, which may be partitioned & micromanaged in terms of a manageable Many...
But-- I don't think this is fundamental. Because I think many games are playable from the aesthetic point of view as much as the gaming pov, and that many pieces of music are likewise playable from the gaming point of view as much as from the aesthetic pov. Exploring these distinctions is for another time
It is enough to say that there is something that is played which is in common between the forms..
So, then, they are both to be regarded as PLAYSPACES, spaces where play happens...
Or, to be more descriptive-- as SHIFTING POSSIBILITY SPACES.
Shifting Possibility Spaces is my best attempt at describing the structural-materiality of this form that game spaces and music spaces are both part of ...
Playspace super-set (space of all possible playspaces??)
A possibility space is fully spatialized. A shifting possibility space allows for the immanent flow of time to enter its description.
Meaning, it can begin to account for the NOW in the space-- the possibility space is ALWAYS an contingent thing, which is not describable from the outside-- which is immanent to our situation in the sapce, contingent on the flow of time, always destroying and recreating itself..
SHIFTING possibility spaces attempts to put TIME back into the possibility space idea, which is too often satisfying with mapping of time onto space ("time is just another dimension of the space").
That SPS will also spatialize time is a probably a necessity and almost certainly risk-- but to keep this in mind early on, the EXPERIENCE of time, local to the player's experience-- maybe we can avoid some of formalism's pitfalls, even as a new aspect of gamespacetime is given quantitative description.
So what does this immediate experience look like? Constantly changing, of course, but changing around relatively fixed grounds, which are the conventional mechanics/rules/boundaries/goals/virtualities that are used to describe structural possibility spaces as such. For instance, you probably have a wall near you right now, which would be difficult to break through, and for all intents and purposes, it is a fixed boundary, even though you could smash it if you got a sledgehammer or whatever..
A new way of describing these relatively fixed properties of a space may be in order, one which can account for game mechanics, rules, instrumental resistances, etc.. i've been attracted to some of the language in the chaos sciences of emergence/complexity/etc, which seems ripe for reappropriation in the context of PLAYSPACES (PLAY is the entropic elephant in the room in all of that, if you ask me..) .. attractors, topological invariants, phase transitions.... but im getting ahead of myself, just a quick mention if you're keen to follow clues and cruise down those avenues, from the local POV instead of the global...
The kinds of spaces we want to imagine, then, are composed of these situational objects (like KOKIRI VILLAGE, or HYRULE FIELD), which are the fixed things of the Idea, and which condition our possibilities as we travel through...
But more importantly for our purposes, these situations are composed of necessarily context sensitive events that act as catalysts, transitioning the space into something totally new. I remember I learned this concept from Conker's Bad Fur Day when I was a kid, explaining the SHIFTY nature of the B-button, which would respond differently based on the situation..
These catalysts are sometimes discrete event-'clicks' like the B-button, but they are just as often rhythmical or tonal/pitched, repetitive, dispersed across a time-field, such as the 'flocking' mechanics that can happen in improvised music, where a little tendril of ornamental excitement from one player might be mimicked and amplified in the others to the point of phase-shift, where the improvisation was once moving in a fixed rhythmic-tonal space, now it's exploded into free-rhythm/non-counted pulses, non-counted tonalities, with its own new set of possible relations/feelings.
With the SPS idea formalized, it should be possible to describe the spacetime of a given situation, which is curved by the objects that populate it, just like our spacetime.
This is done, first, by simply identifying the objects and processes at play (which OOO counts as objects, too, and indeed as long as they are functions this is the case).
Second, by PLAYING them, and working out internal relations (music) from this experience, bottom-up, local SPS, as opposed to top-down, global PS / Universal Set.
The way we each, individually, choose to engage objects in spacetime, describes the curvature of that spacetime, and it will be DIFFERENT for each player, because we are attracted to different things with differing degrees of intensity. Thus, a probabilistic description of the gamespace is doomed, as it attempts to map a sequence of different playings onto space and to then divide this space into a statistical average, forgetting the attractional & repulsive vectors of experiential time which qualified the space as such in the first place.
But is not THE theoretical answer to shifting possibility spaces, by any means. This should be obvious! This is not 'code-level formalism', even if it uses concepts from code to articulate the structures of immediate experience.
This is part of why SHIFTING possibility spaces are used in contrast to straight up "possibility spaces" -- as long as the space is forever shifting, the particular instance of it that we are experiencing right now cannot be counted as a mere repetition, and is always a unique natural occurence. We must tune into the play experience, to experience even the same computational "game state" as two totally different things when we encounter it at two different times in our life... Allow our body to be the medium...
But for now let me just point you to a couple books:
I read David Byrne's new book recently, too, and it is less explicitly theoretical, but comes from a similar point of view, that all music is contextual, situated, and describes this position with a kind of everyday ease that some might find lacking in Harper's more coded/ scholastic style. Bryne's objects are VENUES, MONEY, SOFTWARE, STUDIO, ENO, OVERDUBS... etc ! These will curve the experience of spacetime just as much as anything else.
This is the most simple AND the most complex tactic, according to the relative complexity of the gamespace itself.
In movies, the mickey-mousing is not involved in turning the movie into a musical instrument.
In games, mickey-mousing always serves this function. Because we are in haptic contact with the game, when events are given musical skin, we become hyper-attuned to the possibility of playing those events, and this is how a musical instrument is born in the first place..
The Assassin's Creed example that I put up here last year does this, where only footsteps and murder are given corresponding musical elements. And murder is only a chord change.
It was a rhetorical question all along!
I've found the concept useful, and will probably keep exploring it, but the point is not that this is a fact, an objectively TRUE proposition.. This all depends on how you want to understand games and music--
Rather, the point is that it is a useful one if you are interested in doing things that play with the ideas of games and music existing in any sort of pairing, and that it is useful for moving past this too--
Videogames are called games by habit, but this habit has put us in a funny place, because the structural requirements of the game theoretical GAMES we are used to calling games (which can be played optimally) are by no means a material requirement of this medium.
Even when games are apparently very game-like, it is not that they are actually games, with rules that we follow, etc. We are led through a system of bifurcations in the computer, of branching paths, sometimes the paths insanely dense with branches, and we are given end states every once in a while,which tell us that we've lost, or that we've done well, or whatever.
If we choose to follow the rules that are suggested to us, then the game feels very much like the optimizable games that we have known.
But if we don't choose to internalize the rules, these screens often can feel absurd, out of place.
The game is IN US. If we want it to be a game, it will be... but if we want otherwise....
Then we can just drift.
And the fact that free-and-easy wandering is possible AT ALL in videogames, no matter how much we're told not to-- the fact that MOVEMENT cannot be avoided -- this seems to me to suggest that the drift is a more fundamental aspect of videogame materiality than any sort of relationship to game theoretical optimizable games.
We can move or play in videogames-- not much more can be said definitively. I am interested in a formalism (yes! all the better since everyone is jumping the ship, it seems) that builds from this premise, that regards this movement in much the same way that musical movement is regarded, which has meanings, but meanings which are unspeakable, which are living in the material itself, and which mean very little when divorced from the context.
Game-naming politics will go on, but I'd like to try to imagine a future where things have settled down and either everything or nothing is allowed to be a game-- where the fact of playing takes precedence, and the materials that are playing back-- and that the game is still regarded as a conversation like Chris Crawford has said, but that it emphatically one of mutual receptivity, rather than one of control/persuasion--
I like to imagine that a point of view from along these lines could be regarded as more realistic than that of the GAME OBJECT image and its corresponding representational-boxes sculpted by the designer from the top-down/Universe who has God's perspective, where all the shifts are part of an unchanging whole in His control.
We're still living in the age of the music object, and there will be more of this still, but the sooner that we can respect that OBJECTS ARE SPACES, I think, and that we can play spaces-- the sooner we'll be on our way to allowing all the connections that are possible to be forged between these concept-groups.
SPS operates on the hope one day maybe it wouldn't be very strange at all to talk about games and musics and all other PLAYSPACES as one substance-- composed of many, but all of which can easily speak to one another and listen, because of their shared structure in time, which is concerned with immanent possibility and its contigency on the particulars of the situation which are ALWAYS going to change, even if some more gradually than others.
New Wholes from Mush.
I've learned a lot more, too! This is just a little brain-dump, trying to connect design pragmatics/particulars to the theory that I've covered up to now.
Panoramical's 18-dimensional phase, controlled by 18 parameters on a MIDI controller or other, seems to me to be a PERFECT starting model of these ideas, and how they could be connected both to concepts in playsapces that are so easy that infants (infantile!) or animals could play them (all you do is touch, slide), and to concepts in math (the 18-d space itself), and how, building from here, mathematical concepts might be used to enrich N-dimensional spaces in such a way that animals can still play them.
Panoramical is ready-proof that a HIGH-DIMENSIONAL system is not really so confusing when we encounter it locally, knob to knob.
Even without connectivity between dimensions, a high degree of complexity is possible, the local states of which are determined by the point in phase space represented by the current values of the parameters, and the line which leads up to that point, which dances in some or all of the 18-dimensions. The relations between dimensions are the sorts of harmonic relations in this space, pointing to a connection between an SPS of this sort, where all is given in advance, and say, the piano keyboard as SPS---we would not consider it to be a 'dynamic' game, the piano, but the harmonic combinations we channel through it alters our own sense of possibility, and indeed when we have been playing on the white keys for a while, stratifying habit, the black keys do not attract our fingers so strongly-- habit, too, is an object which curves spacetime.
"Architecture is frozen music"-- via Goethe and others
Now, it is possible to create these architectural spaces that are rather LIQUID than solid, as Fernando Ramallo has done in the visual environments, and it becomes interesting to plug this new empirical evidence of liquid architecture back into the equation, which now reads:
"Architecture is music" (which, I guess, can be SOLID (frozen, traditional), LIQUID (videogame), or GASEOUS (4chan?).
I have no idea how to do a dimensional model of Dyad-- there is a lot going on!
Even without being able to wholly count the full dimensionality of the system at play, though, it was possible to tune into all the micro-rthythms of the game, to separate them into Classes of events, interactions, etc., and to put together a list of 'homework' to get done, all of the parts needed to adequately account for the progressive differences at each moment of playing...
What was KEY with Dyad, was being comfortable producing TONS of stuff.. WORK WORK. And to be happy doing this, to treat the work AS PLAY.
To not judge the work, but to just get it done. There was some judging, to be sure, but it was mostly playing, with the belief that it didn't matter so much what the particular content or SKIN of the game's soundtrack was, but rather than the skin hugged the muscles/game feel nicely..
Much of the Dyad work with Shawn McGrath is the most intensive 'music-organism' shaping I've done, with lots of attention to detail, little volume fluctuations at every point of played contact-- so many details, you zoom into one and lose track of the others, and what is achieved is a strange hyper-intensive messiness/ornamentation which gives particular affective potency to different mechanics in different situations, such that the original CLASSIFICATION of them into groups becomes more difficult, each particular is its own thing..
What I want to emphasize in Proteus work, though, which is very much reflected in the existing rhythm of the final product, is the GRADUAL work that went into it.
Slow work. Non-work.
Ed Key had been working on it for a year before I came on board already, but even once I joined up, a lot of time was spent discussing themes, possibilities, this sometimes more than actually putting in any new content.
The shared mood that was created was the PLAY of making the game, and there was work, too, but this, at its best, amplified the play, the non-work, rather than negating it..
The WORK is key, too, but the energy for work was nourished by a taste for non-work ~~ it will be important to nourish work on videogames in general from things outside of those games.
Some of the most powerful experiences I had with Proteus were the early builds I played, before I had put in any music. I just loaded up the game, and played in silence for a few hours. Imagining the form of the music, the mood of it, the structure of its possibilities, even if I didn't imagine any musical themes in particular. This brings us back to the Kandinsky painting from earlier-- and the possibility of listening with other organs than the ears.. Listening with the eyes, listening with the fingers/touch-- these have felt like KEY tactics throughout all of this work, and by no means am i a clinical "synesthete"-- I do think this is a kind of mood or way of playing that can be entered into by trying to amplify receptivity.
And thus, the non-work. Being receptive cannot be a strong-willed WORK because it requires a silencing of the will that is that active agent which allows work to happen in the first place..
It is possible, too, to balance work and receptivity, and I have managed this on a few occasions-- but I have not figured out any consistent method for doing so.
Where to go with these ideas?
I do think there will be a strong role for videogames to play in our lifetimes (RE: Ludic Century), but I'm certain that they're still not there, having not opened up to inspiration from the space of all playspaces, and the possibility of finding mechanism-independent structures in these that can be computed in videogames without full loss of meaning (with NEW meaning, at least, where the old has been lost).
So, i'm interested in looking at these playspaces outside of games, but then-- slightly contrary to what Kazemi wrote-- to attempt to integrate them into a zoomed out framework of shifting possibility spaces in general, such that what is 'outside' of games is not thought of as being essentially outside, but rather accidentally so.
I listed some examples of these other playspaces above, a list that I've not really seen any attempt to integreate into the 'ludological' framework.
I mentioned earlier the desire for an SPS formalism-- this is probably quite an unpopular desire right now, at a time when games-formalisms are being rejected all over the place, but it feels to me necessary in some way. I see the shadows of the the canonic game thoeretical formalisms even in those games that critique the dominant formal strategies.
It seems to me that it's not a question of IGNORING the existing formalisms, but of LEARNING from them, and DESTROYING them in LOVE, blending them up (like we DESTROY a mango for a smoothie we want), in order that they might be recombined, particle by particle, into a formal framework that does NOT stop at game theory, but which allows for analyses of all playspaces, whether this be done musically, mathematically, etc.. almost certainly pseudoscientifically.
So, lists of other kinds of playspaces ought to proliferate, and we ought not be afraid of the potential this category has to blossom into a new kind of everything (this is what the proper meaning of ludic century would blow open into, in my opinion).
All these practices involving free movement -- PLAYSPACES -- the question is to find the practices that we LOVE and VALUE the most, and to NOT limit these to videogames-- and to immerse ourselves in these practices, to learn from them what we can, and the possibly, if we feel the desire to do so, to bring back our love of these things to games. To count aspects of the processes in such a way that they can be computed with-- but to not disrespect that thing we came to love in the first place.. Not to gamify it, but rather to learn from it what a game actually is, to learn its pattens of movement, the parts of the body and social milieu that it engages, et etc.
So, with this I become a dogmatist again-- shifting possibility spaces as a 'fixed' idea to explore the interrelations between these categories, a ceaselessly transforming substrate or matrix on which apparent differences can be thought and combined/reconciled in action.
Is there an SPS Realism that we could imagine? It doesn't seem too far-fetched.. I can't think of anything that does not fall under the umbrella.
Now, what is useful about this is precisely the possibility of dispensing with the graphical and other shallow 'realisms' that games are obsessed with today, and to tune into their rhythms, and the shifts of context that they employ, and the relations of these shifts to those that we experience in life.
From this perspective, it could be possible to design wholly 'abstract' games, with no representational elements, that are nonetheless SPS-realistic, having something to say in their time-flows that reflects the time-flows and harmonies of our lived experiences..
haaaaahhha, but it's TRUE.
The question for videogame-pragmatics is to stop trying to read games as art, and to start trying to read arts as 'games' (broadly considered), as played things, from the creator's POV (conscious or reconstructed fiction), or the viewer (tho only insofar as viewing is creating)-- to find out what these things mean, not as things but as active ways whose gravitational pulls we enter into-- as flux-worlds, chaos-cosmos, process-organism, shifting possibility spaces.