MusicXML Coding/Research

[Nicholas]

The built-in songs are pretty easy to get to: if you're already on a Mac you can just right-click the app and choose "Show Package Contents".

Otherwise, even on Windows, the DMG file format that Mac's use for apps is just a renamed zip file! So if you grab the Mac version of Synthesia and extract it as a zip, you can always find all the included songs at this path inside: "Synthesia/Synthesia.app/Contents/Resources/music.zip".

[revilo2]

Hello Nicholas,

i've downloaded the last dmg version of synthesia but impossible to open it as a zip file...

[Nicholas]

Er... I got my file types mixed up, sorry! An Android APK is just a zip file. In any event, 7-Zip can extract all of them (and two dozen other formats). You can even tell it to extract files from the Windows Synthesia.exe file(!) and it'll pull mostly-useful things out of it.

(7-Zip is always the first app I install on a new machine, right after the OS.)

From the Android APK: /assets/songs/music.zip
From the Mac DMG: /Synthesia.app/Contents/Resources/music.zip
From the Windows EXE: /.rsrc/SONGS/MUSIC

[revilo2]

Thanks Nicholas



I've done the translation of your files.

i've start to compare the musicxml files to the midi ones.

Visually i see no differences between them when openig in musescore even if the integrated comparision tool indicates changes....

What do you think about it ?

Do you you want to compare them with Gilbert De Benedetti's PDF ? (they won't have exactly the same display...)

[Nicholas]

Do you you want to compare them with Gilbert De Benedetti's PDF ? (they won't have exactly the same display...)

Hmm, ideally we should be able to nudge the MusicXML versions to being nearly identical (if not completely identical) without moving/changing any of the notes. The discrepancies should only be details like beaming, missing articulations, and things like legato/slur lines.

We should be able to omit the finger hints in the PDFs, because those have already been encoded via Synthesia's own mechanism. And the way those work with MIDI files should translate over to MusicXML without any change.

[revilo2]

Do you retrieve fingering and left/right hands split in the musicxml files i've sent to you ?

If it is, i don't understand why (in the synthesia 11 process) you want the integrated song to be simililar to Gilbert's PDF

The midi integrated files aren't

And the musicxml files are visually identical to the midi ones

in addition, Gilbert's pdf don't always match to its own files (theres are some key signature changes, for example)....

Maybe i'm wrong but it seems to me that the most important thing (in synthesia 11 process) is the possibilty of loading musicxml integrated files as the midi ones (if they are visually identical in Musescore, they won't be differences in synthesia, No ?)

[JimNYC]

I believe that ultimately the MusicXML file would render as sheet music in Synthesia like the original and include proper notation, slurs, etc. (as already mentioned). That is part of what MusicXML support will bring -- better sheet music than current midi guessing.

[Nicholas]

Yeah, the shortest version of that is "Synthesia's sheet music is often incorrect, so we don't want it to look exactly the same as Synthesia's current sheet music".

In which of Gilbert's songs did you see a different key signature? Despite the notes being correct, that's still a pretty big error to have missed after all these years.

[revilo2]

The very first one i saw is "A la claire fontaine"

I agree that synthesia's sheet music must be improved, what i didn't understand is that the internal files have to be be modified to display them correctly....
So you have to modify alll Internal files (musicxml files) to make them match with the pdf , it's a huge work...

A other option will be to "create" new musicxml files (pdf scan)...
But in this case, do keep finering and seperate hands ?

(Sorry for my english, i'm french)

[jimhenry]

So you have to modify all Internal files (musicxml files) to make them match with the pdf , it's a huge work...

Yes and yes.

Technically the internal files for the built-in songs don't have to be modified. MusicXML files will be an alternative to MIDI files in Synthesia. So the internal files could remain as MIDI files indefinitely. It is that MusicXML files contain additional information to control the appearance of sheet music. Replacing the internal MIDI files with MusicXML files allows the addition of that additional information so that the appearance of the sheet music display is much better. However, that requires that the additional information be added and probably a lot of it will have to be done by hand. That is huge work, as you recognize.

Gilbert DeBenedetti must have some type of file that generates the PDF files he posts on his site. If he was willing to share those files with Nicholas, or generate MusicXML files for Nicholas, that would be by far the easiest way to acquire MusicXML versions. Otherwise, it could be a community project to create MusicXML files for the built-in song. Huge work spread amongst many hands.

[jimhenry]

This program converts PDF to MusicXML:

• https://www.myriad-online.com/en/produc ... sicpro.htm

I used it to convert the first page of the Keyboard Classics version of Fur Elise. The image below shows the Synthesia sheet music, the deBenedetti PDF at the lower left, and the PDF created by MuseScore from the converted MusicXML in the lower right. It is not perfect but it looks like gets you most of the way there.

BeethFurElise_compare.png (115.91 KiB) Viewed 19988 times

[revilo2]

Otherwise, it could be a community project to create MusicXML files for the built-in song. Huge work spread amongst many hands.

I agree, maybe i could create some musicxml files (let's say the 20 first one of the internal files (alphabetical order)

[Nicholas]

It is not perfect but it looks like gets you most of the way there.

"Most" is an understatement! Did you add the finger hints to the MusicXML version manually? (Otherwise I'm not sure where they would have come from.)

In any event, that's most-likely the right sort of workflow to use during this process. Thanks for the comparison.

I agree, maybe i could create some musicxml files (let's say the 20 first one of the internal files (alphabetical order)

Again, this feels a little bit weirder these days now that the project is in a much healthier place--Synthesia LLC has the money to pay for this sort of effort--but in the absence of anything better organized, I'd say go for it. (Going alphabetically by the included music.zip might be a little tricky because all the difficulty levels are jumbled together, but otherwise it's as good a method as any.)

[jimhenry]

Did you add the finger hints to the MusicXML version manually? (Otherwise I'm not sure where they would have come from.)

No, I did not add the finger hints. They were picked up from the deBenedetti PDF file.

PDFtoMusicPro has a batch mode available in the registered version. It is likely it would make short work of converting all the deBenedetti PDFs. Nicholas, perhaps you could create the converted library with PDFtoMusic Pro and then make proofreading and clean up a community project when Synthesia is displaying sheet music from MusicXML files. (It is hard to know if the "shortcomings" seen in the converted MusicXML are the fault of PDFtoMusic or MuseScore.

[Nicholas]

Oh! I'd missed the first PDF conversion step in your process, sorry. I thought that was a MIDI loaded directly into MuseScore and they just did that nice of a job straight out of the box (which seemed implausible, but I was willing to give them the credit for it). Music OCR makes a lot more sense.

I'll take a look at a few of the choices and poke at the results inside the MusicXML file to find the balance between accurate translation but with the least amount of weird junk left over. I'm happy to see that at least the couple links I clicked through seemed to offer trial versions for exactly those sorts of comparisons.

Your plan after that sounds like a good one.

[jimhenry]

PDFtoMusic is not OCR. They need a PDF that was generated directly from a music editing program, which the deBenedetti PDFs are. PDFtoMusic translates the PDF glyphs to MusicXML.

I think they also have an OCR path but I don't know how good that is. I have used their notation program, Harmony Assistant, for a long time. It includes OCR but I never had very good luck with it. But it has been many, many years since I tried it.

Myriad is small company run by two French brothers who have been doing music software for a VERY long time. They are extremely knowledgeable about music notation and they have a community of like-minded music notation nerds. I would expect that any app of theirs will produce notation of high quality.

[JimNYC]

I was looking at Piano Marvel, which supports uploading your own songs. Interestingly, they require uploading both a midi and MusicXML file for their upload process: http://pianomarvel.com/uploads/document ... ection-2-a

I think anyone who was going to use MusicXML files could certainly also obtain a corresponding Midi file if that helped in the initial implementation of the MusicXML development process. Or maybe it just complicates things. I don't know, but just thought I mention it.

[JimNYC]

Also noticed this page on Piano Marvel's site:

http://pianomarvel.com/uploads/documentation#section-7

The "Guide To What Is And Is Not Supported In Uploads" section is interesting to see what issues they've encountered and what aspects of MusicXML they are and are not currently handling.

[revilo2]

Hello Nicholas,

Did you receive my files ?

What do you think about it ?

[Nicholas]

PDFtoMusic is not OCR.

You're correct; I knew what process was involved, but I used the wrong word. Or at least the wrong first letter. It's not optical. Maybe DCR?

Interestingly, they require uploading both a midi and MusicXML file for their upload process...

Hmm, that makes me worry about just how "compatible" the MIDI-Compatible Part of MusicXML actually is.

Although, requiring both makes me wonder how they synchronize the files exactly. Outside of being exported side-by-side through the same app, one after the other, it seems like it'd be really easy to have small differences between the two that would be challenging to reconcile.

The "Guide To What Is And Is Not Supported In Uploads" section is interesting...

That is a helpful list, if it only helps to inform which things might be trickier than others. (I have already spent some anxiety over how repeats should be handled!)

To be sure, Synthesia will have a similar list when all is said and done. If the tiny fraction of Behind Bars that is actually about general+piano notation is any indication, then the dozen chapters about harp, percussion, and all the other specialty notations are going to remain completely unsupported.

Did you receive my files ?

I do see the email. And there do appear to be files attached. Thanks! Although, until I've got something up and running in Synthesia to load them (or until I even just learn a little more about MusicXML files), it'll be hard to give any feedback about whether they seem to be correct or not.

(Sorry for the wait either way, I should have responded sooner. I have been super far behind on support for months now. Despite getting a few of my ducks in a row and nearly/almost resuming a semi-normal development schedule, I continue to drop the ball on support.)