Research Blog – IV

Although it seems I’ve been a bit relaxed with my blog posts this fall, the last months have very productive and I feel like I’ve made some great progress in my Master’s work. The spring was a very active period for me in terms of performing, and while deepening my knowledge of what I already knew, I also gained a better sense of the gaps in my knowledge about synthesis, DSP, and programming. I tried to spend my time this fall addressing those areas, both in my practical work and in my lessons with Eirik Arthur Blekesaune and Øyvind Brandtsegg.

One of the main things I’ve been focusing on this fall is deepening my understanding and practical use of algorithmic composition. At the end of September, German improvising vocalist Mascha Corman asked me to perform a concert with her at the end of October in Bern, knowing very well that I wouldn’t be able to be in Switzerland at the time. She wanted me to create a SuperCollider program that would run independently, listen to her performance, and then process and synthesize sounds based on her input. Her additional requests were that it perform differently each time and not become predictable, but also that it maintained some sort of structural integrity. Essentially, what she was asking for was heading in the direction of an Artificially Intelligent program that could consistently interact and respond to repeated behaviour without becoming predictable.

At the moment (and for the foreseeable future), I’m under-equipped to create such a program – and certainly not within the timeframe of a month! What I did instead was create a program that generates nested patterns of events in order to create event phrases. These phrases are then distributed over the length of a performance to create a larger form that should (presumably) contain an inner logic, even if inaudible. Each event triggers the real-time processing of a recorded sound buffer or the performer’s live input .

This approach was a pretty quick-and-dirty solution for the given deadline. The performance went okay – Mascha was quite pleased, and many things happened in the performance that were unexpected (as she wished) but implied a certain sense of interaction/machine listening that she desired. These were essentially coincidences – there was very little machine listening happening in my code, but her perceptual experience (and that of members in the audience) was very interesting for me. As I’ve been exploring algorithmic approaches to composition this fall, the idea of musical semantics keeps arising, and I find it very interesting to see how people respond to randomness – sometimes listeners can assign a great deal of value to elements that have no human agency behind them whatsoever, and I find this fascinating!

At the same time I was making this program for Mascha (which I called GHOST), I used the core of the piece – the form-creation algorithm – to create a purely acousmatic piece, using just processed sound files as material. The nested-pattern-generator (below) assigns a number to sound files and processes contained within a Dictionary[~sounds], and then arranges the numbers into groups or phrases of patterns, sometimes with different lengths, occasionally repeating phrases, etc. I called this piece “pyramidg,” after a built-in sorting algorithm in SuperCollider that was originally at the heart of this piece. After some modifications, I’m now just using the “pyramid” sorting method, which you can see in line 11 (I have to acknowledge that Eirik helped out immensely with the creation of this part of the code):

(
~makeSegments = {arg numSegments = 20;
	var pSteps, result;
	pSteps = ~sounds.keys.asArray.sort;
	result = numSegments.collect({arg item;
		pSteps.scramble.copyRange(0, rand(pSteps.size - 1));
	});
	result;
};

~segments = ~makeSegments.value(5).pyramid(9).sputter(0.25,25);

~makeScoreFromSegments = {arg segments;
	var result, list,start;
	var startTime = ~startTime + 10, nextSubsegDelta = 0.0;
	var addDeltaTimesToSegment = {arg seg;
		var times = ({ exprand(30.0,125.0)} ! seg.size).sort;
		times.put(0, 0);
		[times, seg].flop;
	};
	list = segments.collect({arg it;
		var subsegs = addDeltaTimesToSegment.value(it).collect({arg jt;
			var delta, segnum;
			#delta, segnum = jt;
			[delta, ~sounds[segnum][\start].value]
		});
		subsegs = subsegs.collect({arg subseg;
			subseg = [
				subseg[0] + startTime + nextSubsegDelta,
				subseg[1]
			];
			subseg;
		});
		startTime = subsegs.last.first;
		nextSubsegDelta = exprand(30.0,45.0);
		subsegs;
	});
})

Mascha and I agreed to try to develop this project further (with some more time to experiment, of course), so I went to Bern at the beginning of December to expand upon the GHOST code. We tested a few of the machine listening Classes in SuperCollider and discussed in more detail the kind of performing software she would like to engage with. In the last days, I have begun creating the new version of GHOST (which I’m calling EIDOLON), which has very little to do with the original code but may actually come closer to Mascha’s original request. The heart of this beast is the \analyser Synthesis Definition:

(
SynthDef(\analyser, {
	arg inBus=0,frames=1024,thresh=0.3;
	var in,amp,silence,freq,hasFreq,chain,onsets,density,meanIOI,varianceIOI,time,trig;
	
	in = SoundIn.ar(inBus);
	amp = Amplitude.kr(in);
	silence = DetectSilence.ar(in,0.01);
	# freq, hasFreq = Pitch.kr(in, ampThreshold: 0.02, median: 7);
	
	chain = FFT(LocalBuf(frames),in);
	onsets = Onsets.kr(chain,thresh, \rcomplex);
	# density, meanIOI, varianceIOI = OnsetStatistics.kr(onsets,2.0);
	
	time = Sweep.ar;
	trig = Impulse.kr(density/3);
	
	SendReply.kr(trig, '/analysis', [amp,silence,freq,hasFreq,onsets,density,meanIOI,varianceIOI,time]);
	Out.kr(~ampBus,amp);
	Out.kr(~silenceBus,silence);
	Out.kr(~freqBus,freq);
	Out.kr(~hasFreqBus,hasFreq);
	Out.kr(~onsetsBus,onsets);
	Out.kr(~densityBus,density);
	Out.kr(~meanIOIBus,meanIOI);
	Out.kr(~varianceIOIBus,varianceIOI);
}).add;
)

This part of the code analyses an incoming signal in the following ways:

amp – follows the amplitude of the incoming signal

silence – sends a trigger whenever amplitude falls below a threshold

freq – follows the pitch of the incoming signal

hasFreq – sends a trigger whenever pitch is detected

onsets – sends a trigger whenever an onset is detected

density – amount of onsets within a certain time window (2 seconds)

meanIOI – the average interonset interval in the time window

varianceIOI – the standard deviation of the interonset intervals

This collection of signals is used in two different ways: first, all of this generated data is sent to control-rate busses (Out.kr) to be used as control signals for various processes to be applied to the input signal. Second, the SendReply.kr class is sending all this information via OSC to a “Listener.” The Listener takes all the incoming analysis data and then runs them through a list of conditional statements; these statements initiate and terminate real-time processing and algorithmic patterns of synthesized sounds, along with controlling certain parameters (global envelope durations, for example). The Listener also keeps track of data sent in the previous OSC message, with the intention that I can create conditional statements that react to directional tendencies of the data. For example, if the last amplitude value was much smaller than the current amplitude value, we can assume the input signal is crescendoing – and an appropriate response can be triggered.

Though I’ve only been working on the EIDOLON for about a week now, I haven’t run into any major problems yet. I’m trying to create a large library of processes that the Listener can choose from, and exploring how these processes react to the incoming control data is consuming most of my time at the moment. My plan is to have a functional version by the beginning of January, and then I should be able to begin testing it with performers in Oslo. Mascha is coming to Oslo in February to perform with this program at Kulturhuset, and I’d like to have a working version I can send to her for testing as well.

After finishing an early version of the EIDOLON, I’d like to test it quite a bit in order to deal with all the bugs, but I also want to see how it will respond to various instruments and performers and I want to get to know their experiences performing with it. Will it be “convincing?” Will the outcome be musical? I’ve also been thinking about how it could be possible to develop a version with multiple inputs, which would present several problems to consider: do I analyse the inputs separately or globally? Are processes applied only to input sources that trigger them? I feel as though trying to create an EIDOLON that would perform with two or more performers would dramatically increase the amount of CPU and logic necessary to process the incoming data…but it could be interesting to explore!

For what it’s worth, here are the performances I’ve presented this fall since my last blog post; many of these concerts consisted of improvised music, but the two performances at the end of October were my first experiments with algorithmic/generative pieces:

17.9 «Metamorphic Songs» w/ Unni Løvlid et al. @ Ultimafestivalen

26.10 pyramidg @ Norges musikkhøgskole

30.10 GHOST w/ Mascha Corman @ Café Cairo, Bern

16.11 Abelseth/McCormick @ Lillesalen, Oslo

7.12 w/ Mascha Corman @ House Concert, Bern

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s