< Back to IRCAM Forum

Structural problems with the SDIF file written by MUBU

Hello,

I have an SDIF file containing the motion data of a cellist with its instrument (https://we.tl/t-NcJSFjf0X2).
This file has been written from a MUBU container containing 2 tracks :

  • markers (the X,Y,Z coordinates of each marker) with the info table
    SDIF.StreamId 1 SDIF.FrameSignature XMMK SDIF.MatrixSignature XMMK SDIF.MatrixName Markers
  • rigidbodies (the X,Y,Z,Yaw,Pitch,Roll of the bow and the cello) with the info table
    SDIF.StreamId 2 SDIF.FrameSignature XMRB SDIF.MatrixSignature XMRB SDIF.MatrixName Rigibodies

I’m facing two problems with this MUBU-generated SDIF file :

1/ I cannot read the SDIF file with other softwares.

Actually, the frame/matrix SDIF structure seems inconsistent regarding the second stream (XMRB).
I tried the SDIF-Edit program (https://github.com/j-bresson/SDIF-Edit/releases) and PYSDIF3 Python library (https://pypi.org/project/pysdif3/)… Both give the same error at reading :
*Sdif* Warning 20: SdifFile: data/cellist.sdif (byte: 1224=0x04c8=0002310), TextFile: (null) FramH : XMRB Size: 0x0050 NbMatrix: 1 NumID: 2 Time: 0.0020466 MtrxH : XMRB DataWidth: 0004 Rows: 2 Columns: 6 --> Matrix is not in frames components : XMRB

2/ I cannot add a 1NVT table in the SDIF file.

Actually I’d like to get the matrix column names of my 2 SDIF streams and I thought to do it by adding two key/value properties in the 1NVT table produced by the MUBU container… Something like that :

  • marker_names = « head_x, head_y, head_z, etc… »
  • rb_names = « bow_x, bow_y, bow_z, bow_yaw, bow_pitch, bow_roll, etc… »

But I don’t know how to do this with MUBU (whereas it was quite straightforward with the ftm.sdif.write object).
I tried to add my key/value pairs in the info table of the buffer containing my MUBU tracks, but without more success… This buffer info table doesn’t seem to be exported as a 1NVT table in the resulting SDIF file.

Please would you have some suggestions regarding these two MUBU/SDIF issues ?

Best,
Jojo

1 Like

Hi Jojo, what a great power-user question!

Ad 1: the file wrongly defines types which use the XMMK Matrix in the XMRB frame, instead of the XMRB Matrix, although it is written correctly with #2:XMRB/XMRB. Can you post an extract of your patch that shows how the SDIF.* info is set? You can also post the .mubu you get with writeall.

Ascii chunks of file /Users/schwarz/Downloads/cellist.sdif:

1TYP
{
  1MTD  XMMK    {x, y, z}
  1MTD  XMRB    {x, y, z, yaw, pitch, roll}
  1FTD  XMMK
        {
          XMMK  Markers;
        }
  1FTD  XMRB
        {
          XMMK  Markers;
        }
}

Data in file /Users/schwarz/Downloads/cellist.sdif (6108992 bytes):
11397 XMMK frames in stream 1 between time 0.002047 and 94.963730 containing
   11397 XMMK matrices with  34 -- 34 rows,   3 --  3 columns
11397 XMRB frames in stream 2 between time 0.002047 and 94.963730 containing
   11397 XMRB matrices with   2 --  2 rows,   6 --  6 columns

Ad 2:: I thought any other entry in the track info table will be written to the files’ Name–Value Tables, but in fact, this happens only with track info SDIF.FrameSignature 1NVT, but then no data will be written…
The workaround here is to create another track specifically for the NVT info. I suspect this limitation is because NVTs can not be formally associated to a stream in SDIF.

Best, Diemo

Hi Diemo, thanks !
Sure that powerful technologies inspire great questions :slight_smile:

1/ : I send you the extract of my patch which sets the SDIF.* infos of the tracks (I’ve a little bit of Odot inside). As well as the cellist.mubu file obtained by a writeall command on the container : https://we.tl/t-oB6yHXPkjx
I checked again my track infos, and I don’t understand how the generated 1TYP struct can reference the XMMK matrix within the XMRB frame. It’s weird… By the way, what tool did you use to get this raw ascii chunks of the SDIF file ?

2/ : I also tried the SDIF.FrameSignature 1NVT in a track info, but in this case actually, we cannot preserve the frame/matrix structure of the stream. Ideally, it might be nice to store additional key/value entries of the track as NVT infos of the SDIF file (especially to keep the column names). I will try your solution of additional empty track dedicated to the NVT fields. Will I be right if I format the NVT infos in this way ? :
SDIF.FrameSignature 1NVT
marker_names "head_x, head_y, head_z, …"
rb_names "bow_x, bow_y, bow_z, bow_yaw, bow_pitch, bow_roll,…"

Cheers,
Jocelyn

Hello Diemo,
Did you get my little patch and could you open the cellist.mubu file ?
I’m still hoping that a solution may be found regarding the SDIF structure written by a MUBU container with multiple tracks…
Do you think it would be too much work to adapt the MUBU code for taking into account this functionality ?
I planned to use it quite a lot in my architecture, as SDIF is a powerful way to keep synchronized informations of the real-time performance.
Or should I forget this format to prefer individual text files (one by MUBU track) ?
Best,
Jocelyn

Hi Jocelyn,
sorry, I missed the wetransfer. You can attach files to posts here directly.

querysdif on the command line.

these are in the STYP header and wind up as mubu column names, but yeah, additional info can go into the NVTs as you’ve shown. You might want to use just space as separator, without comma, then Max can parse the value string into a list.
Best

Hi Diemo,
Thanks for your reply.
My .mubu file is too big to be directly attached to the post.
So I provide you with a new wetransfer link : https://we.tl/t-pIgvlSqn7p
Jo

Hi Jocelyn, thanks for the patch, I can confirm that there is a bug mubu SDIF export user-type definition when there are more than 1. The data is exported correctly, though, but readers will complain.
There are 2 possible workarounds until we fix this:

  1. edit the SDIF file and change 2 bytes in the STYP chunk: from
1FTD  XMRB
        {
          XMMK  Markers;
        }

to

1FTD  XMRB
        {
          XMRB  Markers;
        }

(no one cares about the matrix name). This could be done simply with a text editor or with a sed command

  1. use sdiftotext / texttosdif and modify the text representation in between

BTW, do you have documentation about your cello mocap project, which is very interesting?

Best,
Diemo

Hi Diemo,
Thanks a lot for the trick, it works :slight_smile:
I didn’t find the texttosdif command in the SDIF distribution, but a tosdif which makes the job.
Just an additional remark about the contents of the SDIF file generated by MUBU : I notice that a lot of frames have the same timetag, whereas their matrix content is different. For some marker data, I counted until more than 10 occurrences of the same timestamp… Is there a reason for these repetitions or is it something that might also be corrected for the next MUBU release ?
Regarding my cello project, I’m interested in understanding how the cellists’ sound features connect to their motor control. To this aim, I explore some timbre and space deformations to assess how the musicians adapt their movements in real-time. I didn’t get some experimental results at the present time because the technical architecture is quite complex to set up, but you will find some useful details of the latter in my proceeding for the SFA 2020 congress.
Cheers,
Jocelyn

Hi Jocelyn,

I see that, too, in the .mubu file. Can you please check if the timestamps are different in the mubu container right after recording?

Best.