< Back to IRCAM Forum

Gesture recognition with audio using xmm

Hi, I am trying to put together a patch for gesture recognition using xmm as I want to track the movement temporally with audio. The movement data is recorded with a Riot Bitalino. At the moment I have Frankensteined a few examples from the Ircam and Mubu libraries together to do this. The gesture recognition seems to be working OK, but the audio is not great, it does not seem to follow my movement. I’m not sure if my parameters are not optimal or what could be the issue. Hopefully someone here can help or point me to an example patch somewhere…I’m a bit new to this. I’m afraid I can’t attach my patch because I’m new to the forum and it’s not allowing me to upload any files. Also, is there a way of doing this with 2 sensors (for example, training the gesture with 1 hand but then using 2 hands to play it)?

Hi, I don’t understand well what you’re trying to achieve with the audio: using audio descriptors for recognition or playing audio based on the recognition ?
for the patcher: you can copy-compressed in max and put it in the message ?

Hi there, I am trying to play audio based on the gesture recognition. Here is the copy-compressed file. Thanks very much for your help!


----------begin_max5_patcher----------
4864.3oc68j9iipjeetm+JPshTNTONTm.4SypMJIRqxpHkUIeXeQsvlx17FL
XA39XVsu+1Sc.Xvt.WfKr6IoG8dz1bU0u6yp7e4KO73xr2XEO57O47mcd3g+
xWd3A4oDm3gpu+vi6BeaURXg71dbGqnHbC6wmTWqj8Vo77z5yjdXW1gxDVo7
9AUmccVZYZ3Nl7d+c4wgI02+9vxUaiS27bNaUoZl.8oK7HO4PbQKbexAFrf+
EHXgqy+S0CoFgx22yTOwiO1bIwLZUVRVt5Jth2vwCseIwQxYS1xe8qdvVS+3
z5YOr0ruH9GxACveEUmdSdXTLKUNqqO2xMcFb.B4QEisGE3BAhOQcgPJp6TY
4l0wIIxm74Zn53qWyMIOBthQQdDV+Bn9jmN4OC9f0OGLvEDHGYjafuKV8I9o
.8+BBS2jHAPnm6BWc2vgxrNn1EdAZts84Y6yxKiyRU2DJPbO+0u7EwgmtR1Y
hMYmw92Z1Y2OYm+jctE6L1lrytjaL6LM3S14+eI6bJ6UN8+Lt4bNCFygOBax
4765YsgSg0l.8ErgdXuEXnOw2OP8Oee9ICjb6Cym+Tad8VbvD5n3fsMpBNC3
Hn643HL5JvQTu4FGs6PRYbQRbDKudnJ3iR46JNdzfHoUgoQw7CrFwRenum.p
ER3AdtpOg8H.+thWY4BYkvFYg5gVNQ5pLD4KUO.8Pn.wGvDeD1q6qa.ZRkey
.HT7Upq4zgFnCMRsVhG7cwC1FC1QMKnRy9wCZo99G44x47hkr7mYogKUjF2i
jKNqAmn1x7QmYCmIIJrLrswsNbS.KZTcUBKLeXCq5nUD4rF6RE1MQDoDCzLq
mswWD8RKVD9xYgQgIIiFBoPuE.HG3n9b09bcDH2p+A4v8TAXLZ1A3WyiKYSA
hIddRANcvKchvKhdSHvN+qL1hesfqZZz.MVBzX0e7wSENI2F5pg.5nrEhoPg
PbEJvHODzgBBP2UuClAGoPAJKQ12SJj6cEWAlCjj0ckxCbWQR6xhXIy.hhG4
y4HJe2qAQE7oOmSzmyFt2elc57CqOmqx1sqUxEZDv9Sw6XN+GUJr4DWG8hYt
WiXF2AFIL5pbUyUOtCpObMfNQGnM0wrOONsTXMeXPWC.5AkVdHPI7Q7Oyik1
1YzaS1lwODFEs7v50rIDCgq2Beud7vlDLUOw.ytmXxPllhG1XBreHV490DfX
nu0f3dXVSxBiDHCtXzngYtBdAbgTV37lLPhmcN4C64J.YCCfha9WOTTFulqx
u1tzj75lRCTwWVIHiwiwu6yx.c.7oNG6wVADcsIgFnOIzHoQ7iG6lY1tYBF8
TqCVLE0COG5KuzFMatGYj18FmP584r8rzHmWYwa1VVLZIcuffEHo1MWoTdvT
k0APf0D1ujFMvBw+wkY7c8nAhOvckDSkEDfeJOguajpyfBvPDPv..A.BV5oG
lBHhO3RTOtKt5obg97mpdfShSYqxNnHsX6E6dCNG6KcD.PH0dELVErMp12FG
EwR62QP6EVQe9G9eq3.cxV67G9i+QmBt41UasnOhMXMBwSj5CNwTh7B5QoI7
wYFQbg3q.yT7UVLPpinTObKTJzeAY1Cj5nG5VNxJ3OegVkD+cVRLqnzhhKTf
T2Bg.TkLGqbbouXp.fYOnp9bgCby7dqBkn7cCMJKcS02M+QTZQ3m9t8+M8ca
X49sYYQiNgBUQnQ4x0DO.F3VkVx.NYfGjh2.h5T5iysBMtWobqBkrHm+8+ke
+u2hZ0v9ndA6.vP9C..fOcH3RNDTgdITWU53jEJ126FjW0Yyc.NKxD7G3qPk
GAtydE9+ommoRSTMOipDa2TdFjsyMO3CqGjCV7qilSbRx1bIiKnKSRUNJALJ
stOMT+UouxoHK5D49jv2c9a.iuiQjdc4fAxPvmb5bIdVKctbuRDdtLQ.oJks
Pb8w9AjXgeO5fErIr+yVFcKyCimPKg.vs.eDbpzwgKe8I9fCCv73lnXOWhuO
T5VpImZHW0svqTevCZbzWAKWzKeq.k8ELfL9o5CFEL..SDAAfBf.B0WZlfhQ
.jGjfCpBJf+1DyBDoZVQt4wIPlhx8C6V1WI4fWTaMVIAn5wuKqrdIGv02sa2
Ww+hxvRVwTTkWiGTJwISUUNFN2Ula2gkGV71tchJI6HbQvY25Uqb9VEnCcc9
VNaygjv73eHy3gHtSw+qGkPtLJQkmYDFLBC4CYPG6dyx0+WuT4KGU7i9npxW
6ovEUk1b7MNGEd6y7dehLpR4GmFwd6hxMiKk6351okp5zR2IhsvydyNw9QT3
pea3LMjjshCtWvMp5J7JZtHgLiqrWocakQ71bAXqoqHI9E1hMbWeZ.hWByan
Tme0tfXc0oVsMLMkkzgjeRCYYDviTwGoZTE7fD7h3MoBtHk5h0bA2RwGRhKz
6dIEMj8kAkg3HqxJHrotbMgiE9BK54vxx73k7fgN9ohJjdEVWfWSNvxVWe55
y2d9DmFWFq3UjQ1UCGctojrzM8Sf5bq6pBSjV66vIWMVRY95QOP5b8hsbOKL
bjpIL5dOG3PUSlEvpqK4Rq+iEMuw8XJUXA62jF49V3gn3LG0wusmqrhGiJvc
gy2hNjqLwAHhutOqHV7UNuuCYgEUk0vZSCT7ztvKqKqh0VaGK4OprI1gIk+7
740.bn4rhvc6SXR7Ub59CJNcqSo9GDzfYAI6oTaBtLNVovPKJN3N0JwRF3um
lJYcUtlI7++PgC2yru2WiXiuNLFVk.IeC5Pcop0m54uZM+Rm6TeODljCsY4Q
JEARNZK1e1PpZEfHWIv.z0zP1HuYGIMs36.tpxVpZPmqN9NH4tFem.IH8Wsv
AYQUOHJp+U9kGZh9sBf2IUPhvejTvY.Cg7W.UcDvEQJcYhDC25vUc3SZir7m
aAnXg9jFbUVVxxPt2Q0UAndJEksi6cTwp7rD4M7RbQ7IdWp3.WsMKqfo8FJC
WVz7lgcey4GRpblpMhfwksi5rf+6LFq2X1dOf.pTIm6PdwwryUrOIt74Sqjg
9WQ6YooCqHIWK4tTG0szr4L1O5Pt4bHbuQdIl8ZMeW8UxYa39s0MUcxxO4Uc
nSsQj.mDAeJdTOmtZb4WonyfV8dR4g3jtIa85BVm2UMKhFBLWPiwCuJYcbBq
PyqbUb9pD1VYOsI4tZ7gSxZn4M1Fs25x5YGp8ys8KPQiaX5z7Z2jG2gESP0x
YbpTw1bQWP2lPoeU3v8JeCS+Kqh.pAxFRYBvEuvGBvXP0BiJPnYQsMc.7Tqt
LhekpXvoL.ZFrATLqXBJXI7owIvU06qMZ8qGc6ly0rOgq3ZaVRD2vSmgae3p
3x26lX2N7.uFGUt8DVfJ9JSqs3oZjL6o5xwTrMrJs87PFJyRK5pZ3BLjsQLs
4dpZedYg2OB+IgumcnrKygJhjyDWZwP0nHlzYB77Q8JZpRphlVFycWX.5ZKf
+35d+EV9xrhNf7qwoQ7oZ7IJsTz35EaJO94NJbpMf1Zc8Ty.Xj91ZtAcxzmR
4aL.pWsPykIsU1ni3MNVutDnl2d2A2zWYQZ39Z0M.tT8BM7BZT.1tJaMQ10d
5Y53OjaqgI7P1ElmN2XVkoKQxeJzn26LtTW81qNYoSdteNpw46r20YVYMmOH
60i1wrbnVk4gq9tLPKPSxVxYQr0NuyJb9lRFVP6pu5tvx732338B9Sv+1aBP
zAhc42cb55LmMGhc9EQohY4BW+bdM7E1SNJjkPEKmx7KONhFlGMzBsRkQdQw
6ba9G.f4dohTs80zaAA7f9lZXdYZqwYdRalJVYIEDUk5gdHfbIutjOnQzuhr
TQ0L28jyQW9b.8PBQiMT0JRnKP0cdfZZHTrYcoZzhfIEbdecm77g9gpJyYJ5
GYD1W1sWZDgdxQp5Sr50cDs+vxrWexQZzyQPPJFkH1jRjBMXgJMT5HcpF.5B
qOkNjtKEpXvi2oZkUQgmP4kAp0lL0UgLl5RcEfu2cJTEfPjJZutVEx+tlJow
T2Ss4UySszHP9W01Hy7uKxXyDkABTKHDrGtpSol68PQ.8y8PwO2RPawNCsI6
b0Nb6MjcF8I67mrylu90FG6bkMoaH6L3S14O2gaa4fbbVoiqiO+e54q8uXPD
PhJmMtvIztgOo71rS6DokukXu0TvO8qAmJDOR4CKP02EzelWCNFsDbtkKwl8
w6yb18R3K4a3g5K+6BY39.WswlCGa5upog.NJ2mGbFsNCXBMR.Y3lhsr0gon
Msem9R+aysYINI42FeLWpUohpKWTsSJ.Ms1CB6O6Ki0+qr3ULm+YVwp738xt
T6uSrdV+6s4tZAQgCp1HPFdKsH3yUvpgZBqVyZ.OZ81i5OuqfUD9i7BXsO0k
N0kLj60SzyE6xxJ2Zy8+VEFSoNAffSsAebmRabJmOr7SKOfn3nuvxK5VCvGd
Lb+9VmtaOGuK7WUbC9MMrKednNEn4T4LQA0TOOo4rhMXm3RNJ4PtBo8VS+nK
dMYbYpzCwGqVgfpVMkjzOAxtXeUVZkj4mZZ.0lB8iOt+BiU7TPZK7rfiNIa0
26z6KhRtxRiS2y4AzzEBO7XDacHWUyy5wucu9IoQt6E0xuHBOINJKULI5PHD
mtd3D4AUoysMvHuizv8ZdXU0g54hhktyghkg4sJ7Ir9hUEH97RzycYfstr5x
6iSSOAKVlsu+KlKZMk9u7xL9E2Mz6VdkhmOjpt5yhtq+YQkv5degIIUx3ce8
uElFuKrjIpcupKoZtnR6zVUYu6.upq7hlqDwYwWcrYKbayLzX+URpanxQwaD
6YNcNWY3lhtmow5UqScXYkH7ykLtNpphYe7F3BGwEkEaydsn5FOocmN4mur1
hzsUX147C5zwIKBNo5R4lmQQ6aPqCGCq7r+7.KamTfZeDGnZqz1ddzahB5Uc
YudgzZgFTaZw1XrZqMNkYhZy8u8CwGVGW5Ttk4HXS35H+aKbNVAu1uEcNtac
7suRcCf1Bei.Cfucu23aEnaF+mtdDS4Sa0NJHo9XevquQfaMOPawU4KRTEnV
l5rBBniuMiMbv9i9aZLPUnyZ40yV3hiInvy1Lx6E2CuMrZCfo+QB2UxM8iTg
V.oBU6nfm6FoAQLz2N68GQb4IsUd+NieMXS0OrLDpY3xNsXdekV8toBb.bIO
Pr7rAE2s.tL.sPaq5OAbI9dyW1ocEt.5pWDBUtQUVEHh6kPHs6agyBqtKAn2
Hq6ILYKgTju0IiS.pLsiHCXhs+.g6KmqmCwtWqE3iQXI6mI89MKAPw00i2Jx
NjupFHprZ4zEV4gJTFm1Du4etwyBw8YDUaryApgyAuYbNPLbNflw4.1v4.cF
mCHCmC3YbN.FAd.ncNTcx5b0XZuoNpnj6KuKaRxVFlbRHu5Bt15c+pve5eS1
wqCuakMgjDpJ1.Ea1ZCc3Jn3Miq56vUkUKMgqn4hY+HLpu8AC3H1GLTKrVTe
6AFCrKPLvhlGN6Urgtv4OcHO0QzlmaRkqZBmi+tqos.cKCW88M4htws+Rbbb
0Zr7jEYy39ALUs8ZpuDeppkSZWz11o+DpowCGdAWplrEwQrdanCrU2A3Ibju
n6kk4bQ9KwUwGELONnZCIPClWkb6Qg3gerP73EN+mrRG05KWrA7q1Io9nf7Q
T+dQ9.Wuwh8wderv9PA1Wr10bTMj7ujpbE9CB1GhZusnATQIfgi.eSL2nmcw
rHNlsLLu7ernLauip08e71iAApsBp5FluJnKgERSQgA2KLHXgyuS3VgXIqrK
dUd19sYorOJrlTURv8qPq3wxYhtWn0bVw6obqbxL99Zb4Vm586Hm5KTbGXT8
4N4RkKiGhTia0tICxWeiEPHWqhzo0kIBWfODWO1hucxtguXY7l8532Bb6poy
ncJEsMbg87SsGPMmU+qn3yhnC2EVlkOge5lHp0f+UCv3K+S2j7w5l7FE7eZ.
xUXgyBLNnSz48EYt6wPyOCSa5H4av.AvVXfD+DtdwQxqSFAdjaAS03Hf4eno
d5GZ2qangFLzPhMvu.CFIjMXY7buYvjILm1.jnA2JPhZBLQn1XjH2JcHTSzV
Qwygns4CMv1h1TSDsOY90Z6u85FaSDAo.KPaId2JtHB0DXx2FiD1DYPOaLRf
Qf8rqjgQCMDMGF8LQwCI3dMxndLzil8QFDLKDZSF4dfY3UMxXSrZhrgDKl1Q
CdOxQA1ZjtHLYkQxDsPX3bvxXzPifygDJFMBkCW2HAuUVtvfa1HIdInKLRhe
EEqKW50MRvKEtVfsfoKZ22ZXuKpEgNKr8hgFeIINzsBHk+NuYe8JnalAAj+H
b995FI5sRgE5lEvF5loZDAtYXOiTiXgAB5eq.InIrdxeJ4u9gxHe1sw.YjWO
1HgiPixgpU3Htln7bm+gdVRpJznr.RmkzeXh5KuYAnMgP6OKD5qHx1qKJOfQ
kH.ZA4H.8VkEVfIp6rhhU.99Q3vWQNIbuAC8rHhBP2LlHyz6SsxPcq74..tY
nOSLffvyB6o68i8bDxjmUQ4SVa7hw3j0D+IqG9yWK78uN3OcMvK6o59V66e4
u9k+WhPHB6.
-----------end_max5_patcher-----------