< Back to IRCAM Forum

MusicXML exported from OpenMusic is invalid

OpenMusic
#1

Hi, I’m Daniel Spreadbury, part of the team working on Dorico, Steinberg’s new scoring application. I was contacted recently by a user who wants to import MusicXML files exported from OpenMusic into Dorico, and I agreed to take a look at the files for him. In doing so I discovered that, unfortunately, Dorico is unable to open MusicXML files exported from OpenMusic because they are invalid (in terms of XSD schema validation) and contain a number of significant errors in the encoding strategy. I am reporting these issues here in the hopes that the OpenMusic developers will be able to address them at some point.

I’m sure this is not an exhaustive list of the problems with OpenMusic’s MusicXML export: I would strongly recommend adding support for XSD schema validation, or at least DTD validation, so that you can be sure that your exported MusicXML files are at least valid in terms of the schema/DTD. This is by no means a sufficient step to ensure that the MusicXML files actually make good syntactic sense, but it’s an important first step.

1. Invalid values
The value is expressed in the MusicXML file as a fraction, i.e. 1/4. This isn’t allowed: you can specify a non-integral value, but only as a decimal value rather than as a rational number, so 0.25 would be valid, while 1/4 is not. However, if possible, divisions, and duration (which are effectively multiples of divisions), should be an integral value, so instead of using a divisions value of 1/4 such that a whole note is 1, use a divisions value of 1 such that a whole note is 4.

2. Invalid values
This particular file includes:

sharp-three

in lots of places. The element uses an enumeration of valid values, and the correct value is sharp-3, not sharp-three (i.e. the digit, not the word). The OpenMusic developers should ensure they are encoding only the accidental types that are specified in the enumeration:

http://usermanuals.musicxml.com/MusicXML/MusicXML.htm#ST-MusicXML-accidental-value.htm

3. Invalid values
The element in the element must be upper case, i.e. it must be:

G 2

rather than g as it is in these files.

4. Redundant values
Also, it’s not really a bug in quite the same way, but it’s redundant to include identical elements at the start of every bar: you only need to include the element when it changes, so it’s normally sufficient to specify it once in the first bar of each part. It’s very noisy to include these in every bar, and I would suggest that you recommend they omit them if they’re not strictly necessary.

#2

Hello Daniel,
Thanks for these useful comments.
As you noticed the MusicXML support in OM is not of the most rigorous ones :slight_smile:
It has been mostly built empirically, trying to make for OM scores to be exported correctly both in Finale, Sibelius and NoteAbility.
Details are getting adjusted and corrected all the time…
The changes you suggest here do not seem too difficult to implement: I’ll try to get them done in the next OM release.
Looking forward to see OM scores in Dorico !
Jean

#3

Dear Mr. Spreadbury,

As the primary author of this code, i would just like to be more specific about these issues and commentaries given above.
“It has been mostly built empirically…” Sorry Jean it was written following the first specifications given by Recordare, BEFORE the OPA of CODA on the standard.

  • First of all, regarding the divisions issue, as indicated in the MusicXml specification, :

    xs:documentationThe divisions type is used to express values in terms of the musical divisions defined by the divisions element. It is preferred that these be integer values both for MIDI interoperability and to avoid roundoff errors.</xs:documentation>

It is clearly stated that it is PREFERRED to be an integer and not FORBIDDEN a usage of fractions.
This is why most of commercial parsers don’t cope well with the fractional part of this type unfortunately they assume it is EASIER to use their own standard duration. By the way Sibelius (AVID corp.) and Finale (CODA corp.) don’t have the same standard. [ I don’t know about Steinberg???)

  • Now concerning other issues, these must be modified by other contributors (accidentals, etc…)

  • Concerning the clefs issues, unfortunately this is due to non-case sensitive proprietary compiler Lispworks. This used and works on Mac and Unix format. I assume that your user exported a DOS format, This is trickier …

To end this, i will just mention, that MusicXMl not being a REAL open standard anymore I personally don’t trust and use it and therefore dropped the support. Now i use the lilypond excellent stable free format. It is working for ALL plateforms without any problems.

Best regards

K

#4

Karim, thanks for your feedback. However, I’m sorry to say that you are mistaken in your belief that can be expressed directly as a fraction: it uses the type positive-divisions which is defined in the schema as: “The positive-divisions type restricts divisions values to positive numbers.” (See here for the documentation on the MusicXML site.)

MusicXML is more of an open standard today than at any time in its history: its development is now under the stewardship of the W3C’s new Music Notation Community Group, which guarantees that the current version of the standard, and any future versions produced under the auspices of the W3C CG, will be made available under a free license to ensure that it can be freely implemented by any developer, whether their product is open or closed source, and whether or not they are a commercial enterprise. I encourage you to check out the Community Group for more information, here.

LilyPond’s input format is of course excellent and is truly open source, but it is unfortunately completely unportable. I would strongly encourage you to continue supporting MusicXML, and to fix the problems that exist in OM’s implementation, to ensure that OM can be as useful as possible to as wide a range of musicians as possible.

Thanks for your consideration,

Daniel

#5

Dear Daniel,

Thank you for your feedback.
I have checked the code, and you’re right about . It was not that clear to me at first.
Now i think i am getting the issue here. is related to on a measure level (am i correct ?)
So, as i don’t like to leave unfinished business, i am working on it.
Just want your advice here.
Since i am working on Linux, and dont own no Finale, Sibelius or any other commercial typesetter, do you think it is wise to test my fixes against musescore import?
i.e is it 100% compliant with musicxml standards ?
You see we had a great deal of discrepancy results between Finale and Sibelius users. But now i think this will be fixed and will work with any MusicXml compliant application. (I hope :-))

I already fixed the issue. It was related indeed to LispWorks which is as i mentioned before very case-sensitive.
I also agree about the redundancy timesignature factor and will fix this. (In Musescore, timesignature is displayed each time, which is really annoying…)

Thank you again

Best
karim

#6

Just to say, I use both Sibelius and Finale, and would be happy to test anything you produce. This is a topic very important to me!

Nigel

#7

I’m really pleased that you’re able to look into this, Karim, and I think it will make OpenMusic more useful for more musicians.

Now i think i am getting the issue here. is related to on a measure level (am i correct ?)

You are correct. All of the values have to be expressed in terms of the prevailing value. One strategy for setting this is to look at the rhythmic values of all of the notes to be exported, and set an appropriate divisions value based on that. With notes that are scaled in tuplets, it can be hard to choose an integral divisions value that can be cleanly scaled up and down to tuplets of arbitrary complexity, but in general most applications seem equipped to handle this relatively well. (Dorico, for what it’s worth, does express rhythmic durations using rational values to avoid these kinds of precision issues even with complex tuplets, but of course MusicXML doesn’t work that way, so we still have to import and export using integral divisions values.)

MuseScore is a very good test-bed for MusicXML export. I would also suggest that you try Soundslice, which has a good and free MusicXML viewer: simply upload your MusicXML file here and it will render it for you. Another good free tool is Noteflight, which allows you to upload MusicXML files into its online editor.

Unfortunately there is no such thing as 100% standards-compliant MusicXML at the moment, because there is no fully testable specification that dictates what that means. That is one of the things that the work in the W3C CG is intending to address in the long term.

A good strategy for MusicXML export is to run the resulting MusicXML through a validator; ideally you would integrate libxml, xerces or a similar validator directly into OpenMusic so that you could be sure that your exported MusicXML is syntactically valid, but if you can’t do that, I believe it’s possible to run MusicXML through online validators, provided you can give them appropriate access to the DTD/XSD files.

Once you are sure that your MusicXML file is syntactically valid, checking how it renders in MuseScore, Soundslice, and Noteflight (and indeed LilyPond, via its musicxml2ly script, and Finale and Sibelius if you can) is a good way to check that other applications will interpret it as you intend.

#8

Thank You Daniel,

this is really precious informations and advice. I really appreciate.
Will let you all know about the advance of the code. Unfortunately i think in order to make it work in a REAL generic way, the code must be re-written partially from scratch.
By the way i found some annoying stuff about duration and most particullary regarding tuplets. In this example :
http://usermanuals.musicxml.com/MusicXML/MusicXML.htm#EL-MusicXML-tuplet-normal.htm?Highlight=tuplet

I f you notice in the tuplet 3/2 durations are unequal :
we have
228
and
227

The reason i have assumed that it was better to use fractions. But i know floating points are not the eden of computer calculation,
Anuhow, i am preparing a new strategy… Hope this works.,…

BEst to you all
K

#9

Hi Karim,

Yes, that problem you describe with the value in tuplets where at least one of the values is unequal is exactly the kind of situation that I am referring to when I talk about tuplet scaling. I think this design decision was taken in MusicXML years ago because both Finale and Sibelius use integral units for duration, so MusicXML opted for the same method rather than going for rational numbers at the time.

Do let me know if I can provide you with any further support or advice. I would also be happy to test out any updated MusicXML files exported from OpenMusic in Dorico.

All the best,

Daniel

#10

Hi All,

Working on this, and encountering a lot of issues.
Apparently embedded tuplets are not really handled… even if it is documented here :
http://usermanuals.musicxml.com/MusicXML/Content/EL-MusicXML-tuplet-actual.htm

Tried with Finale, (it worked) but not im musescore, neither in soundslice…

Any Ideas ?
K

#11

Hi Karim,

Thank you very much for your work on improving the MusicXML export from OpenMusic. I know several Finale users who will really appreciate this.

MusicXML has a great many features and I know of no program that implements all of them. The more complex the notation, the less likely it is to be supported everywhere. If you do find any problem with nested tuplets or other features in Finale, please let me know so we can look at improving those in the future.

MusicXML represents durations as rational numbers, where the duration element is the numerator, and the divisions element is the denominator. You don’t need to keep the same divisions value throughout a part, though currently most software does. Perhaps OpenMusic would do better by changing the divisions as needed for places where there are complex collections of tuplets. For a notation application like Finale, the exact values of the duration element are not so important as long as the note type and tuplet information are specified correctly. But for other applications, the exact duration information can matter a great deal.

Best regards,

Michael Good
VP of MusicXML Technologies
MakeMusic, Inc.

#12

Dear Michael,

Thanx for your feedback,
Yes indeed i have figured out the the divisions/duration issue and have fixed this in the new code. However as you put it, there’s np way to have compliant mxml code for all programs. We have tried to make it work for both Finale and Sibelius because we have users of these typesetters.
I will base this on Finale of course. Too bad mxml is not that generic, or let us say, the parsers aren’t.

Will send the clean code to Jean Bresson in order to compile it with a new OM version.

Best to you all
K

#13

Hi there, I don’t know if this may be related to the problem that you already solved, but just in case… I exported in MusicXML a VOICE object from OM 6.10.1, with the rhythm I attach “origin OM.png”, and then I imported the xml into Musescore. The result in MuseScore is “destiny musescore.png”.

As you can see, in Musescore it appears ugly. The rhythm is correct, but visually weird (also musescore said that it wasn’t a valid MusicXML file, although it was able to open it). I don’t know if this is a problem of Musescore of because of the export issue that you mentioned/fixed.

Also I wanted to THANK you for all of your work. The OM tool is something really great.

Cheers,
Alberto.

destiny-musescore.png

#14

Dear Alberto,

Thanx for your feedback. Yes this is partially resolved, It’s due to MuseScore parser. It doesn’t connect automatically steming according to beat. Finale does this automatically. I found a way to do this for musescore. However take notice, musescore’s musicxml parser doesn’t unfortunately handle nested tuplet.
[even if you write nested tuplet in musescore, export them in musicxm; import back again, this will not work.
This issue was alreaady reported some time ago to musescore team.

Best
K

#15

Dear All,

I’m very happy to see that there is the effort to make Dorico and OM work together via music-xml.
I would love to see microtones (as small doable via xml) work as well.
Please don’t forget this issue as I have serious problems with finale and xml (quarter tones work only with a huge workaround and eight tones not at all) - so the Dorico OM combination is my last hope.

Thanks and all the best

Fran

#16

I absolutely agree with Fran. Lots of us are waiting for the Dorico & OM integration! :slight_smile:

#17

Hi everybody,

I have made much progress in order to be the most compliant with musicxml following Michael’s and Daniel’s precious advices.
Working on Linux and having neither, Finale, Dorico or Sibelius, (i tested on my old mac for Finale and online on soundslice and musescore), i would be glad if i posted some xml files if anybody can test them with their own environments and report any problems.

Issues:

  1. I have noticed that there’s no way to get nested tuplets to work with soundlsice and musescore. Finale’s Parser works fine. Don’t know with Sibelius…
  2. grouped beams. Since some parser handle them automaticaly, (and this is great meaning no need to put every beam and beamlet direction in xml), i don’t know for others. These work fine with : musescore, finale. In soundslice beamlets are always by default to the right direction giving awkwartd typesetting.
  3. Didn’t test yet the 1/4 tones but will do (is it important to me also…!)

So i will be posting soon 1 or two files to test.
All this in order to update the om code.

BEst to you all and many thx
K

#18

Hello again,

Here are two mxml generated files from OM with the new export code.

  1. stems.xml : in order to check if it works. (on Finale and musescore they do)

  2. nested.xml : nested tuplets. Works only in Finale.

Thanx for trying them (if someone can on Sibelius, this will be great)…

Next step will be :

fixing more issues such as 1/4 tones.

Best
K

nested.xml (4.34 KB)

#19

Hi Karim,

I’ve just tried to open two files (“stems.xml”, “nested.xml”) with Sibelius7.5 and Finale2014,
and I obtained the same result for “stems.xml” in Sibelius and Finale,
but “nested.xml” doesn’t work with Sibelius.

Best regards,

Keita

PS. I’m very happy to see this topic and really waiting for the combination Dorico and OM as you all!

nested.xml-Sibelius.png

#20

Dear Keita,

Thanx for your feedback. So everything is as expected. Stems and durations work for both Sibelius and Finale. Nested tuplets don’t for Sibelius (same thing as musescore), but Finale’s parser know how to deal with this because the musicxml code is compliant.
By the way do you have Dorico ? is it possible to test these files with it ?

Best
K