< Back to IRCAM Forum

Using segmentation and knn ml but performing resynthesis on an external hardware device

MuBu
#1

Dear friends,

I have a question for the forum.
My goal is to perform segmentation of an audio file by onseg detection using the usual Mubu modules,
exporting the segmented audio files, then using knn classifcator index to point to those file following some strategy (let’s take the shaker example in the main MUBU overview page). I just want to load those
files in another workstation (hardware, external) and using the knn index to address the files sending the parameter via OSC. Now the question is: 1) how do I export the segmented file in the proper order and so I can use the knn properly as index in the other device. For instance exporting the proper sequence of audio segments from the Shaker example in the Mubu Overview. Any suggestion would be really appreciated.

All the best to you,

Federico Placidi

#2

Hi Federico,

a lot depends on what your external HW player is capable of. Is it running Max/PD, or is it a sampler?

A general way would be to save the segments from onseg into individual files and name them according to the id also used by knn (a buffer/marker index pair).
Another possibility is to derive a linear index from your corpus, either by counting segments across buffers, or by combining index = bufferindex * maxsegment + markerindex.
That id/index is then sent over OSC and can be mapped to one saved segment file on the external player.

For the splitting and saving you can adapt catart-mubu-tutorial 2a analysis to also use the buffer index or just the linear index in the file name.

Best!

#3

Hi Diemo,

Many thanks for your suggestion.
I would perform the resynthesis on Kyma.
The sampling/granular object, would load a folder of sound files in terms of index. So for example if you have 50 segments in the folder, the very first one would be index 1, and so on up to 50, in a linear fashion. So I was pondering if using the index coming out from mubu.knn would be enough to match the same index (segment) in the external hardware.
Many thanks !

Federico

#4

you can make it so if you load the saved individual files after segmentation also in mubu (and redo the descriptor analysis on the whole file), then the bufferindex from knn will correspond to the index in kyma (under the condition that both load files from a directory in the same, hopefully alphabetical order).