< Back to IRCAM Forum

Too far behind

Antescofo
#1

I’m encountering a latency problem which I don’t understand. The metric/rhythmic context is quite simple, just noires and doubles-croches, the tempo is rather slow (69 bpm), all notes, non legato, with some neglectable microtonal alterations, are played ordinario by a flute (currently a synthetic one, because the real flutist lives in another town, however fairly accurate). Nevertheless actions fired by Antescofo are always about 60 ms behind. This is of course much less than the extreme latency I got some time ago because of lacking calibration – but still far too much for what I need here, as this is within the limits of human perception for rhythmic discretion. In the passage which I try to realize 21 sixteenths should be recorded to a sample buffer and played back after a delay of three sixteenths at a rate which fits the canon follower to the remaining 18 sixteenths of the leader (see the notation of the expected result in sheet.tif). But the beginning of the first note of the diminution version is truncated because the recording obviously started too late. I tried to relocate the actions here and there (while adjusting the delays) but that didn’t help. I also tried diverse combinations of I/O and signal vector sizes what didn’t help either. The analysis parameters of Antescofo are set to the default (nfft=4096, hop=512). I introduced clicks to see when Antescofo detects an event (“debugclick” in the attached excerpt from the Antescofo score). The click always appears about 60 ms after the onset of a note.

late-click.tif (41.7 KB)

#2

Hello,
could you post the synthesized audio file, so that we could reproduce the problem?
Thank you very much.

#3

I’ll send you the audio this evening.

#4

Philippe, I uploaded the audio to a cloud and sent you the link per e-mail. In the meantime I changed a note or two (just for musical reasons), so I also uploaded a recent version of the Antescofo score. Regards, Kai Yves

#5

I experimented a little with the analysis parameters. Setting the window size to 2048 reduces the latency by about 20 ms on average. Some events, however, are detected later than with the larger default window size of 4096; this is probably due to the position of the window hop in the audio stream. Decreasing the window size even more (e.g. 1024) leads to a lot of event misses. Cutting down the number of harmonics used for spectral match (e.g. from the default 10 to 5) doesn’t seem to be a choice because it changes almost nothing (except for eventual event misses). Thus,

antescofo::analysis 2048 512  
antescofo::nofharm 10

seems to be the best setting for flute. Still there are many actions which should be triggered on note detection but are late by more than one FFT window size (about 46 ms at a sample rate of 44.1 kHz), often 70 ms or more.

Are there other parameters concerning analysis which could be tweaked?

#6

@kyl:

I just found some time to look at this issue.

At the beginning of your score, you set variance to 0.3 which is also the default value. As a reminder, the variance parameter in Antescofo is in semi-tones and defines the pitch sensitivity of Antescofo.

For the synthesized audio you sent, if you just reduce this to 0.2 you’ll reduce the latency to almost half (i.e. roughly 20-25ms) and without necessarily changing other parameters.

Note on using synthesized audio for simulation:

Synthesized simulations of instruments are usually poor in spectral content. Antescofo is designed to perform in tough and rich situations. Having poor audio can degrade performance and requires tweaking. For example, in your audio simulation Flute harmonics enter step by step. If you look at the Spectrum of your flute simulation in AudioSculpt, you will notice that 3rd harmonics enter at about 30-40ms after the fundamental. This is rarely the case on real instruments!

My guess is that if you use a real flute, in your case, default values will work perfectly and you wouldn’t notice the latency you observed even with default variance.

Note on using the Variance parameter

High variance increases uncertainty and leads to more latency. Less variance requires your audio to be more precise (pitch wise) and leads to more reactivity.

As a consequence, on instruments with keys, (where pitches are more or less always tuned) you’d want to avoid using high variance. On the contrary, on strings you wouldn’t want to use very low variance (for obvious reasons).

From experience, some wind instruments will warm up during performance and get slightly detuned (especially if your piece is long). This is when variance is useful. We have lived this many times in performances of … Explosante-Fixe … by Boulez following the flute non-stop for 40 minutes.

When you use micro-tonals (which is your case), you’d want to reduce the variance so that Antescofo can distinguish between adjacent pitches.

In practice, we have found that the default value of 0.3 is good enough for most situations.

Let us know if you have any other issues and sorry for my late response.

#7

Arshia, thank you very much for these valuable insights. It’s evident for the ears that a synthesized flute is a rather poor imitation of the real thing, but I didn’t imagine that the coarse spectral transitions could be problematic in score following. Regarding variance, it’s puzzling at first but plausible then that it also affects latency. This leads me to another question: Could it make sense to change the variance value for a section of a score or a passage, according to the musical context? Generally, I think, it’s better to tolerate eventual intonation errors than to risk missed notes; but sometimes low latency is more important.

#8

@KYL: It totally makes sense to change the variance value even in the middle of the piece! For example, if you have specific microtonal sections, you’d rather want to tighten the variance only on those sections and leave more tolerance elsewhere.

Note that changing the variance inside the score, will change the variance of events from that point on in the Antescofo score.