< Back to IRCAM Forum

Sylseg, ircamdescriptors, segmean -- can they work together?

MuBu
#1

I’m trying to use sylseg and ircamdescriptors together, and can’t quite figure out how, or whether it’s even possible. The basic goal is to segment an audio file into syllables, run one or more ircamdescriptors on the same audio, and then get mean descriptor values for each of the segments as defined by sylseg, so what I would ultimately have is (preferably) a single track looking like this:

[time] Duration SyllableMarkerType Descriptor_1Mean…Descriptor_NMean

The problem is that these don’t fit nicely into a chain — Since sylseg looks directly at the audio, as do the ircamdescriptors, I would have to do them in separate mubu.process objects, but then the data would be in two different tracks, and I couldn’t use the sylseg marker values to get mean values from the descriptors. I think what I would need is something like

seg_drawing

But I have no idea if this is possible. I dimly remember some experimentation with a syntax that allowed splits like this in a chain, but don’t know if anything came of it… Any suggestions will be most welcome!

Thanks,

\M

#2

Hi Matt,

there are 2 possible solutions to this advanced and very welcome use case:

  1. the immediately working one can set @chop.segtimes from a sylseg’ed track, similar to p onseg-with-descr-then-edit-and-resegment, just from a different track, in the howto-segmentation or like in catart-mubu-tutorial 5b resegment.
  2. the pipo-graph based solution you propose needs the merge to be able to handle unsynchronised streams (because you want the labels). This still needs some work. (The intermediary solution not to have sylseg output data, just markers also needs proper synchronisation, since sylseg buffers all input until the end of a phrase and then spits out all markers at once.)

Best!

#3

Hey Diemo – option 1 absolutely does the trick! I don’t even know if option 2 is worthwhile to pursue. I had a feeling there would be some kind of synchronizing issue with my little drawing, and using @chop.segtimes works just fine!

Thanks,

\M

#4

Is there a size limit for the list you can use for @chop.segtimes? After about 10K markers, I’m getting crashes. I know it’s a ridiculous number, but the customer is always right…

\M

#5

There is Max’s limit of 4096 list elements that surely plays into this.
Guess you’ll have to pre-segment by sylseg’s phrase output, then do syllables, then concatenate again (on export).

#6

Oh, I can happily generate and segment with 30000 markers, I must have missed the augmentation of the list length restriction. So the problem is not with pipo.chop. I’d rather look at sylseg.