Real-time interaction benefits the investigation of systems whose behaviors are observable in what Otto Laske has termed "conscious-time"[1]. In conscious-time, one is interested in being able to observe features which evolve over a duration of a few seconds and, most particularly, in response to gestures and actions taken. The kinds of events which one may observe are, by necessity, temporally linear and non-reversable. Direct manipulation displays facilitate this mode of interaction.
Non real-time interaction benefits the investigation of systems whose outputs are not temporally linear and in which events occur over what Laske has termed "interpretive time." As Laske notes, "the illusion of lasting time is created by memory through interpretations of events on a high level of abstraction" [1]. Programming displays facilitate these kinds of interactions since they allow a composer to hypothesize musical structure as outside-time.
As a software system, Orpheus is an experiment in designing a possible task environment for the composition of acoustical and musical structures. Toward this end, it attempts to join real-time interactive tools (for the design of sound morphologies) with non real-time tools (for the organization and deployment of sound morphologies across different levels of time). Moreover, it attempts to join direct manipulation interfaces with programming interfaces in an effort to capture different moments of compositional design.
This paper describes some of the features of the system, as well as detailing what the author views as some of its shortcomings while emphasizing the authors strong feeling that composers must be involved in the design of computer-based task environments for musical production and research, not so much as a way of guarenteeing the preservation of historical musical paradigms, but as a means toward retarding the rate at which music software systems surrender to those historical musical paradigms.
While the system is designed primarily to investigate control interfaces to synthesis algorithms it can, for precisely this reason, be a useful tool in the development of synthesis algorithms, at the source code level.
Orpheus is built on a Model/View architecture. It provides displays for direct manipulation and real-time interaction with synthesis algorithms. It also provides text-based editors and a command line utility for interactively sketching and instantiating acoustical and musical data structures.
Using the integrated control display tool, one is empowered to create different interactive "views" into the underlying system. A "view" constitutes a selection and grouping of parameters. Figure 1 depicts such a view. Here parameters N1, N2, P1, P2, B, R, S, and L are selected into the view. Groupings of parameters are formed through the linking of one parameter to another. Parameters "N1", "N2", "S", and "B" constitute one such grouping. To engage this particular grouping, one might click on the node labelled "N1" and drag it around, observing acoustical feedback to one's movements. Movement of node N1 engenders movement of all nodes to which that node is connected (depicted by lines in the diagram): N2, S, and B. Each connection is defined by a "weight" according to which the movement of one node will effect the movement of its attached node. So, for instance, movement of the node labeled "N1" would cause movement of nodes "N2", "S", and "B" by weight factors of -2.0, .5, and 1.0 respectively.
Within the integrated control display, one can add connections, remove them, or change their weight. Any such display can be made to be persistent: they are saved as "sound configurations" into files with the file extension .cfg.
Temporally-bound sonic structures unfold as a consequence of dragging different nodes around. Any one such structure traces a "path" followed by the movement of a node. The integrated control display allows the composer to save paths for future retrieval. In addition, if when investigating the behavior of a particular grouping, one finds a particularly interesting region within the parameter space being tracked, one can use a zoom tool in order to enlarge the view and zoom in on that area.
Through
the combined use of the sliders display and the integrated control display,
the composer investigates aspects of the behavior of the underlying synthesis
algorithm through direct manipulation. Moreover, s/he can create different
views and save different paths within any such view for future use and
reference.
4. Acoustical and Music Data Structures
Currently
Orpheus defines three primary data structures. These are (1) the
SoundObject; (2) the EventSpawner; and (3) the StreamBuilder.
4.1 Sound Objects
Saved paths can form the basis for the definition of a particular acoustical 'prototype' or 'template', from which events are generated and projected in time. Within the current version of Orpheus, such a template is called a "SoundObject." The term is purposefully reminiscent of Schaeffer's term: it refers not to a specific acoustical artifact per se , but to a model according to which many artifacts might be generated . A SoundObject forms the basis upon which actuated variants are produced.
A SoundObject includes another data structure called a SpawnFilter. A SpawnFilter determines the constraints according to which acuated acoustical events are spawned from a SoundObject.
A SoundObject also defines aspects of the auditory environment in which spawned events are to be produced. This is accomplished through an Environment data member object. Depending on the particular circumstance at any given moment, the Environment member of a SoundObject can constrain the unfolding of other simultaneously occuring SoundObject events. In this regard, the SoundObject views acoustical events not as isolated occassions, but rather as parts within a larger whole, in which the larger whole can influence the structure of otherwise homegenous parts.
The
SoundObject data structure is depicted in figure 2.
4.2 Event Spawners
Within Orpheus, a composer can generate more-or-less traditional "playlists" of SoundObjects. A more compelling tool however is the EventSpawner. By defining an EventSpawner, a composer articulates the conditions for the unfolding of events in time by specifying the unfolding of data. An EventSpawner links three data structures: the SoundObject, a SequenceProducer, and a ProcessObject (figure 3).
A SequenceProducer generates tokens on the basis of which events are generated. SequenceProducers can be of many different types: they can effect the output of particular grammars; of stochastic systems; or of iterative systems, such as non-linear chaotic systems.
A SequenceProducer generates structures that are of local scope: they apply to currently computing aggregate of events. ProcessObjects propagate higher level structures in that the output of a Sequence Producer is combined with iterations generated within a ProcessObject. For example, a ProcessObject might engender the gradual decay of particular acoustical features. This could result in event aggregates in which each successive aggregate is quieter, more sparse, or more disparate than the previous one.
EventSpawners spawn events whose morphology derives from a SoundObject. First, it determines the number of events for a particular event aggregate. Then, it tells the SequenceProducer to generate that number of tokens. These tokens are combined with output from the ProcessObject to produce an input to the SoundObjects spawn filter. The result is the generation of a sound event which is placed on the system scheduler for production at some specific point in time.
When that time arises, that event is computed at that time (it is not computed in advance). This allows for greater flexibility than would be the case if all events were computed ahead of time. Its cost however, is greater taxation of computing power and possible audio dropouts.
StreamBuilders are data structures for aggregating streams of events. A StreamBuilder contains from one to three EventSpawners. It schedules the execution of event aggregates in much the same manner that EventSpawners schedule the appearance of individual events. Moreover, StreamBuilders allow for interaction between constituent EventSpawners and, as such, of unfolding streams of events.
Within a particular StreamBuilder, simultaneous streams are mutually determinative. One way in which this mutual determinateness is manifested is through activation of a SoundObject's Environment object. When an EventSpawner is launched, it is given a value which determines which of the EventSpawners is "primary". Which ever EventSpawner is primary is given a pointer to the other EventSpawner's SoundObject. The Environment of the primary SoundObject is used, in this instance, to constrain the synthesis parameters of any other simultaneous unfolding SoundObjects.
This
situation is depicted in figure 4. Here a single StreamBuilder has launched
two EventSpawners, ES1 and ES2. Since ES2 has been given primacy, the Environment
of its member SndObject (SndObject__2) acts as a secondary filter to the
SpawnFilter belonging to SndObject__1. As a consequence, the events of
which SndObject__1 is a prototype are constrained according to the Environment
specified for SndObject__2.
5.
Musical Form and Data Structure
One of the
principles I have sought to demonstrate in developing Orpheus, is
the principle of the permeability of musical form[2 Koenig]. By this I
mean that generative structure is not top-down. Rather, while aspects of
the global framework may influence the evolution of events in time, the
reverse is also true: that the particularity of the unfolding of events
determines, in part, the eventual unfolding of a higher level framework.
This is currently
accomplished through the ProcessObject. Each time it is used, the ProcessObject
is left with some, oftentimes minimal, trace of the data object that used
it, be it an EventSpawner, or a SoundObject. The ProcessObject can be initialized
with a minimal kernel structure. Then, as it is used, that structure is
gradually fleshed out during the process of its use.
6. Using
Orpheus
In addition
to the direct manipulation displays described above, Orpheus presents a
text editor for editing various data structures, and a command line interpreter
for initializing and launching data structures and processes.
Figure 5 shows
an example of the command line interpreter with several commands. The first
command instantiates a SndObject called "so" from a sound configuration
displayed within an integrated control window. It will be remembered that
within an integrated control display, particular "views" can be defined
and saved to disk. Moreover, any such view can have any number of "paths."
Here "so" is instantiated from path #4 of the sound configuration called
"saw1."
The next command
causes the Sound Object to play itself at twice the duration. In addition,
it multiplies the value of its pimary node by 1.2. Remember that the primary
node is the node which you dragged with your mouse, and whose movement
occasioned the movement of any parameter nodes attached to it.
7. Conclusions
and Future Developments
This document
serves as very sparse introduction to a fairly complicated software project:
many details have been left out for the sake of brevity. Part of the reason
is also that the project is in a state of constant flux.
The software
is better tailered to the composer who is interested in investigating non-standard
approaches to music composition and sound design. It is, as such, not particularly
usefull as a "production" tool.
In the future,
I would like to add support for VSS, an authoring tool developed
by the Audio Development Group at NCSA[ ] or for Perry Cooks STK synthesis
library. Currently, it is built around a single synthesis algorithm which,
though rich in its behaviors, some may find lacking in variety. I would
also like to add support for handling network messages so that it might
perform in a live performances environment. I would like to continue enriching
the data structure layer in order to incorporate a variety of procedural
scenarios. Finally, I would like to build a strong language interpreter
into it, such as ELK.
Source
code, executables, and examples are available on-line at http://duracef.shout.net/~mhamman/projects/orph.html.
9.
Acknowledgements
My thoughts
in the development of Orpheus have benefited from many other similar
projects. These include Modalys
[4], Foo [5], Bol Processor [6], and Kyma [7].
Other influences include the work of Roger Dannenberg, research within
the ACROE group, and the research and tools within the Audio
Development Group at NCSA. Acknowledgement is also made to Camille
Goudeseuene who provided software development advice when it was most needed,
and who was a partner in the development of AREAL, the real-time scheduling
library on top of which Orpheus is built.
10.
References
[1]
Laske, O. E. "Considering Human Memory in Designing User Interfaces for
Computer Music." Computer Music Journal 2(4).
[2]
Goudeseune, C., and Hamman, M. "A Real-Time Audio Scheduler for Pentium
PCs." Proceedings of the 1998 ICMC . San Francisco: Computer Music
Association. 1998.
[3]
Koenig, G.-M. "Composition Processes." Lecture delivered at the UNESCO
Workshop on Computer Music. Aarhus, Denmark. 1978.
[4]
Morrison, J. D., and Adrien, J.-M. "MOSAIC: A Framework for Modal Synthesis."
Computer Music Journal 17(1), pp. 45-56. 1993.
[5]
Eckel, G., and Gonzales-Arroyo, R. "Musically Salient Control Abstractions
for Sound Synthesis." Proceedings of the 1994 ICMC . San Francisco:
Computer Music Association, pp. 256-259. 1994.
[6]
Bel, B. "Migrating Musical Concepts: An Overview of the Bol Processor."
Computer Music Journal 22(2). pp. 56-64. 1998.
[7]
Scaletti, C. "The Kyma/Platypus Computer Workstation." in The Well-Tempered
Object: Musical Applications of Object-Oriented Software Technology ,
ed. S. T. Pope. Cambridge, MA: The MIT Press. 1991.
