Automatic Fingering Prediction 0.1ish

[Frost]

I have hopes that one day next week, or maybe the next one after that, or maybe...? would be the big day for AFP 0.2.

Anyway, I wanted to ask Nicholas, or any other knowledgeable person here. Do you have any recommendations for a .NET midi playback framework? I mean, for sending midi events to the OS for playing it. I don't want to handle timing, so not "note on" "note off", but something very simple along the lines of "play note X for duration t", that's it. It's not important, but during transcription of notes into AFP, people (me ) make mistakes, so it would help if you can check the song by ear. Although assuming midi loading would be error-free the need for such measures would be less, but if it's something I can do it in one evening, I'd like to add it.

[Nicholas]

I don't. Sorry. There are a few people in the mididev Google Group that have been trying to put together friendly .NET libraries, but I've never used any of them so I don't know if they offer the high-level functionality you're looking for.

[Seanpek]

What happened here? The thread completely stalled!

[vicentefer31]

I think I have very good news about this feature. I have found a free software called "PianoRollComposer" that let you add fingers to a midi. This is the webwhere you can find it, and this is the direct downolad.
To add and see the fingers in options you have to check "Show Fingers" in Options and after that you select the note you want with the left button of the mouse and write the finger you want to use.
Like you can see in the picture I made the fingers are saved in the midi.
Is it not amazing?

[Frost]

Wow, it looks good. Thanks, I'll have to check how it stores the fingering data. If I can parse it, the users can provide sample fingering by using that program. I can't check now since I'm in work, can you tell whether the interface for inputting fingering is easy (for a complete song)?

[vicentefer31]

Wow, it looks good.

Yes, it's very, very good. I think it's amazing the author does a software so good like this like hobby and also the source code is in the Public Domain.

can you tell whether the interface for inputting fingering is easy (for a complete song)?

It's very easy to add fingering but you have to do note by note but I wonder if it will be possible to add a feature to do several fingering in just one click, something like the videoI made a a year ago.
We also should know what Nicholas think about this way to add fingerings to the midi, because maybe he has a better idea.

[TonE]

Wow, could you share all those exercises with us, at least the Hanon ones?

Sorry for the almost one year delay, here: viewtopic.php?f=7&t=1794

[TonE]

Dear Frost,

how far did you come with AFP 0.2? I am asking and wondering if you were also planning straight text or csv format export/saving of the fingering analysis results? I was thinking of following processing chain:

midi file input --> AFP --> .txt/.csv output

Then using another tool/trick/technique to add this fingering information from AFP output,

1. into the midi files themselves
2. into some other form of meta file related to the midi file

which are supported by Synthesia. So my question is how difficult would it be for you to make AFP 0.2 export that kind of .txt/.csv output? If AFP would support the command-line we could add fingering information to hundreds of songs in one shot!

[Lanselta]

Hey Team!

I am new to synthesia, and i must say, i LOVE it. I am also new to piano, so this tool has be excellent. It takes away another barrier to learing piano by making the music easy to read. After three days ive managed to play the proper version of Moonlight Sonata with the midi taken from Mutopia. Alright its slow, and i still hit like 300 wrong keys when i play it, but its a major breakthrough. Trying to play it off the score is a bit confusing. But this program made it so much easier.

The reason why im posting here is i believe one of the only things missing from Synthesia is the finger information. This would ensure my technique is as it should be for the song. I can see from this thread that you've been on this for a while, and i dont mind having to transpose the finger information myself. To me its time well worth spent in learning a song. Just make it possible to see the finger number on the falling notes or keys. That would in my eyes complete this program. BTW, it was well worth the $19!

Keep up the good work!

[TonE]

Just make it possible to see the finger number on the falling notes or keys. That would in my eyes complete this program.

My favourite three missing features in Synthesia to feel complete, meaning like 1.0 are:

1. fingering information, first manual fingering, later also automatic fingering
2. direct jump to a marker via the command line which would open up all kind of looped practicing, using any kind of algorithm.
3. real-time error output from Synthesia as midi events which would open up external real-time error monitoring, analysis, plotting, detailed comparison to older performances rather than comparing only the scores.

[Nicholas]

direct jump to a marker via the command line...

You're using AutoHotKey, right? How about key presses to get there? I was thinking the other day about throwing 1-9 in there as shortcut keys to jump between the first nine bookmarks.

[TonE]

I was thinking the other day about throwing 1-9 in there as shortcut keys to jump between the first nine bookmarks.

Yes, why not, also good, why stopping at 9, you could also take 0 as the tenth bookmark.

[Nicholas]

I didn't want to say "1-0" because then I would have had to add extra explanation... like "1-9 and 0". I was apparently feeling especially lazy when I wrote it.

My intent is that 10 bookmark hot-keys would be available. (Unless you have a custom software keyboard... at which point it would override them.)

All of this is stop-gap stuff until the user assignable input.

[TonE]

direct jump to a marker via the command line...

You're using AutoHotKey, right? How about key presses to get there? I was thinking the other day about throwing 1-9 in there as shortcut keys to jump between the first nine bookmarks.

Nichoooolaaaaaaaaaaaaaaaaaaaaaaaas, stop with the development on this point.
I forgot another point: I was hoping you might use in future the keys 1..0, plus in the row below q..z (on a German keyboard), q..y (on US keyboard), for switching on/off midi channels 1..16 of the current song. I am using this feature in MidiSwing, again I made this workaround using AutoHotkey, and I really love this feature, especially, if we can see somewhere also the names of all currently playing instruments. Let me add another screenshot:

MidiSwing instrument view

MidiSwingInstrumentView.jpg (73.7 KiB) Viewed 45482 times

Seeing all playing instrument names in a similar way in the falling notes display would be really great, at least during the "toggling phase" for a few seconds at some place where it does not overlap with the falling notes, then fading out slowly again or using another method. If anyone is interested in my small AutoHotkey workaround for MidiSwing, I can post the code also here. Basically it just left clicks on certain coordinates on MidiSwing according to key presses 1..0 and q..z. This is interesting because using any generic midi file you found in internet you can fastly find out which parts you would like to keep in your song playback, some instruments have sometimes some kind of "distractive sound", those I am removing usually, to keep a clean nice sound where only the main musical elements remain, ready to rock in Synthesia! Meaning you can find the right settings with MidiSwing, then use this selected combination from MidiSwing for Synthesia on its command line! But other users might like to have such a feature directly in Synthesia, it feels really good in my opinion. You listen only to what you want, not to everything, and best part of it, without any additional preprocessing of the song files.

But ok, we could use also control-1..0 and control-q..z for those.

So we have to find out what is more important on 1..0 and q..z:

A) jumping among markers
B) toggling of midi channels of the current song

The less important part we/you could put on control-1..0 and control-q..z.

[Seanpek]

I don't really get what the previous few post are talking about, but today while I was playing some Chopin pieces, I got an idea, since playing from actual databases are not really working ( Think about it, since if you use Czerny and Hanon, without looking at the other notes, this would be disaster). I suggest that we bulid up databases that have to be correct. I.e. If you have 12 notes played within two rests, we can strictly limit the fingering to the set that would be most favourable, the idea is to list out all possible 12 notes series, playable by the piano and chose the most optimal. For example, the chain that you play in Chopin Fantasia Impromtu, 48 37 48 47 48 53 56 55 53 55 53 52 55 56 60 ( G4# A4 G4# G4 G4# C5# E5 D5# C5# D5# C5# C5 D5# E5 G5# ) played by the right hand is strictly
2 3 2 1 2 3 5 3 2 3 2 1 2 3 5, though the current AFP states, 5 1 3 2 3 4 5 4 3 4 3 2 3 4 5 , which is not possible to play ( Considering how fast it is suppose to be played). The pattern is roughly related to the mathematical concept relatively prime in pairs. The idea is that you want to achieve the fastest possible way to play it. So for the string of notes A B C D, A B must be playable fast, B C must be playable fast, C D must be playable fast, A B C must be playable fast and B C D must be playable fast. This should be the most optimal fingering.

[TonE]

The fingering information should be written into a separate .txt file.

I just had another idea how we can implement manual note fingerings for Synthesia using simple text files. As any programmer and many users should know simple text files have many advantages, you can edit them in any text editor of your choice easily, you can parse them using any tools without big complexities. Let me write now the new .txt format I have in my mind:

1. Each line of the text file represents one bar in the midi file. So line 5 of the .txt represents bar 5 of the .mid.
2. All notes in each bar are ordered using the same method as also used in Keykit, which is from left to right in time domain, then from bottom to top in pitch domain. So whenever we have "chords" or multiple simultaneous notes with exact starting times (= onset) then in the ordering the lowest note will come first, then going higher. Then to the next onset.
3. For fingering information I thought we can use single character numbers, for the right hand 1..5 as known, then for the left hand 6..0, 0 meaning 10. So thumb of the left hand would be a 6. If we jump over a note without adding any fingering information for it we could use a "dot" = .
4. We decide for a delimiter to be used between the fingering numbers, for example a TAB = \t
5. Thats it!

This would make it very simple and quick to map any score fingering information into a .txt representation. Somehow I really love this method, it sounds very simple to me.

Then Synthesia could do following: If I have a song "moonlight.mid" in some directory, I would simply create moonlight.txt in the same directory and Synthesia would link such two files, if both exist in the same directory.

Any improvement ideas?

[Nicholas]

If all of your symbols are single character, why do you need a delimiter between symbols? Seems like a line between measures is probably plenty. You probably don't need Tabs in there.

[TonE]

If all of your symbols are single character, why do you need a delimiter between symbols?

You are right, if we would keep the system in 100 years still the same we would not need any delimiters. Delimiters might allow us to add later additional stuff, like maybe "lyrics" in the form of "ff", "mf".

Seems like a line between measures is probably plenty.

I do not understand what you mean with this?

But for now, yeah let us kick out even those TABS, concentrating only on the fingering information, nothing else. This is most important anyway for piano practicing. Maybe we can allow any kind of spacing, as the user wishes for visual context, but the parsing algorithm would simply remove all of those again, meaning removing any SPACES and TABS.

[Nicholas]

I like the new lines. If these are being entered by a person, it's a nice error correction mechanism. If we stripped out all the white-space on parse, a single missing (or extra) character would mess everything up for the rest of the song and be pretty tough mistake to find.

If a new line cuts off the note annotations until the next measure, a single missing/extra character will only cause a problem in that measure and be relatively easy to detect.

Also, the tendency I see in sheet music is that only the first couple notes of a passage even has fingering information if the rest can be inferred from those initial positions. You could just leave blank lines to skip measures... or just put in the first couple notes and then go to the next line to avoid having to put the requisite number of dots.

[TonE]

Now only Frost's AFP should output in this new .txt format.