< Back to IRCAM Forum

Antescofo Midi capabilities and reliability

Antescofo
#1

Dear all,

I am considering using Antescofo for a doctoral project that combines a live performance of Hindemith’s Ludus Tonalis with real-time video and generated graphics. I am exploring the possibility of relying on Antescofo’s score-following capabilities to track the live performance, and I would like to know whether the system can handle a highly polyphonic and virtuosic piece such as this one.
I could also consider performing on a Disklavier to ensure an unambiguous MIDI input, but I am unsure whether Antescofo can manage multi-voice MIDI tracking reliably.
Thank you for any help or suggestions you can provide.

#2

Hello,

We have several examples of piano following for highly virtuosic pieces. I am thinking in particular of Ravel’s Concerto for the Left Hand. And the Metronaut app for iPhone and tablet uses the same tracking engine and offers many piano pieces.
So it is definitely worth trying. You can transcribe an excerpt of the score to see how the system reacts. It is possible to automatically convert any score to be followed (in MusicXML format or in midi format) into an Antescofo score; see: Editing the score - AntescofoDoc

A few tips:

  • Antescofo can struggle in the low register and when the sustain pedal is used extensively. These are exactly the types of passages you should test. You can increase the size of the buffers used for the FFT, which provides the spectral information needed by the follower. Another source of difficulty may come from sequences of very dense chords sharing many common notes. The system can have trouble distinguishing two consecutive chords, especially if the pedal is down. However, Antescofo also relies on temporal information, so tracking can still be correct in such passages. Proper testing is essential.
  • Good microphone placement is important. At IRCAM, sound engineers typically use two microphones above the strings: one on the bass side and one on the treble side. The two signals are mixed, but no processing should be applied (in particular, we insist, no compression).
  • Proper calibration of the system is crucial. The example patch shows how to monitor calibration, and the levels should ideally “stick to the ceiling” without constantly saturating.
  • You should not expect perfect note-by-note following. Antescofo may miss a note or a chord, but it can catch up on the subsequent events. Its synchronization and recovery strategies compensate for local errors and still ensure correct alignment of the electronics, even if note-by-note tracking is not 100% accurate. For contemporary pieces including passages or playing modes that completely break the follower, it is often possible to switch automatically to manual mode (where a stage manager advances the score by hand) and return to automatic tracking once the problematic section is over.

Arshia Cont, the father of the score following system, is watching this forum and will no doubt have additional useful advice for you.

#3

One more remark:
Antescofo has been used several times to drive or send information for lighting control, process automation, or live image generation. See for example:
https://youtu.be/tJKJ8RcN-7Y?si=xQHuF82wzhvEd922
https://www.youtube.com/watch?v=eGV5aLhgBqA

In this example by José Miguel Fernández, Antescofo does not follow a score but instead tracks the gestures of Philippe Spieser, who is wearing motion sensors. These data are used to control the electronic synthesis, and Antescofo sends information to Processing-based visual processes that generate the images : https://youtu.be/U9ZhU5o7kk8?si=IrxfYcLHTdv4M4S8

#4

@steinway95 I just listed to Hindemith’s Ludus Tonalis which I didn’t know! Magnificant piece!

I do not see any major difficulties on this piece for Antescofo. I would say the major difficulty would be to get the Sheet Music into Antescofo. For that to happen smoothly, you should try to fetch a MusicXML version of the score and use our antescofoConvertor that directly converts MusicXML to the Antescofo Format… and then test!

The Polyphonic MIDI following of Antescofo is currently under development for Max/Pd (it exists in the sister App Metronaut but you probably won’t find the Hindemith there).

Since you are not using any real-time audio, using a mic should just work!

If you run into any issues, this Forum is the place.