Automatic Fingering Prediction 0.1ish

Synthesia is a living project. You can help by sharing your ideas.
Search the forum before posting your idea. :D

No explicit, hateful, or hurtful language. Nothing illegal.
Posts: 6

Post by FireBreather » 07-09-12 1:29 pm

Finally bought the "Learner pack" reading through these posts,

im abit confused,
is there anyway off activation "Finger hint Prediction" i know you can manually place the finger hints, but i dont know where to place my fingers
So is there anyway to activate this finger prediction or is it still just a running idea

P.S. im on a mac if that helps

Posts: 321

Post by Pianotehead » 07-09-12 3:03 pm

There's no automatic fingering prediction yet in Synthesia, if it's been put in I missed it. If you're wondering about finger use for a particular song, check out how others are playing it, for example if there are some covers or tutorials on YouTube. Just make sure it follows the arrangement you have, at least approximately.

Posts: 12439

Post by Nicholas » 07-09-12 6:58 pm

Yeah, this topic is covering an early experiment by Frost... who appears to have (sadly) abandoned the project. I would love to see this picked back up and incorporated into Synthesia some day.

Still, it is definitely a challenging problem. Frost was using pretty recent research done by universities to develop what he'd done so far.

Posts: 321

Post by Pianotehead » 07-09-12 8:50 pm

MIDI Illustrator has fingering prediction. I tried it few months back, and since I can't change into an octopus (like in Prometheus) I decided to stop using it. That's how far out of reach the fingering suggestions were.

Posts: 11

Post by davidddd » 09-27-12 2:20 pm

And if you want to take us RIGHT INTO THE 21st century.... :ugeek:

A graphical library of hands that are reaching that appear on the keyboard actually showing you where that fingering would place your hands.

We can dream. :roll:

Posts: 12439

Post by Nicholas » 09-27-12 4:29 pm

davidddd wrote:A graphical library of hands that are reaching that appear on the keyboard actually showing you where that fingering would place your hands.
You have no idea how long I've wanted that. I think it was before I even sat down to write the first line of code for Synthesia.

The little hands that slide in when you select a hand in the simple song view (in the latest dev preview) are actually 3D rendered. I licensed a super high-quality pair of arms/hands that were all pre-wired for animation. Don't you second-guess me! I'll do it! :lol:

Seriously though... how cool would that be?

There is another piano instruction thing out there that tries to do it, but with 2D hands that stretch in strange ways. The effect is more creepy than anything. Unless I were able to show proper posture and technique clearly such that it wasn't distracting, it would probably be more eye-candy than anything. If I do it, I want it to be useful.

Posts: 11

Post by davidddd » 09-27-12 9:59 pm

Well, there are some articulated hands and body parts that are manipulatable within Solidworks - that's what I use as a mech e, but I think those models are pretty pricey. On the other hand, Poser may be able to get you what you need. Dance departments use Poser to capture choreography, and you could use either to generate the series - reaching a total of 12-15 keys, leaves a very finite number of possibilities. The user could set hand width, etc...

But yeah, it may be pie in the sky.


Posts: 6

Post by andwan0 » 06-11-14 12:20 pm

This is similar to "Midi Illustrator 2" intelligent finger numbering system too.

Just curious, how are the finger numbering stored for Synthesia? Are they stored in separate files? If so, possible to export & import the finger numbering from Midi Illustrator 2 program?

Posts: 12439

Post by Nicholas » 06-12-14 8:11 am

The finger hints use our own format, described here. You'll find that in fingers.xml in the Synthesia data directory, or if you use the data export feature in the metadata editor, you can keep the hints alongside your MIDI files inside .synthesia files.

Posts: 1

Post by carlosalbertopavon33 » 10-03-15 9:58 am


Posts: 110

Post by revilo2 » 11-29-16 5:07 am


By searching for an auto-fingerings software for piano sheets, I found on Google :

It seems very cool. there is a source code. unfortunately, i was unable to use it (I am not very talented in computing... Maybe Someone could... (Nicholas, Frost, ... )

Anyway, i took the liberty to contact Dorien Herremans, the author of this research with the aim of becoming known him Synthesia's forum and especially the Automatic Fingering Prediction section...

Nicholas, Maybe, would like to contact him....

Posts: 12439

Post by Nicholas » 12-02-16 3:22 am

Ooo, new research. I like seeing what people come up with. (I was happy to find that I'm familiar with almost all of the papers in his paper's bibliography. I'll have to take a closer look at the other new ones, too.)

The 110 second search time is a little scary. Although, I've got a handful of strategies in mind that reduce the size of the problem (or increase the speed of the algorithm) by a couple orders of magnitude, so when we do get a chance to finally tackle this, the target is still sub 1-second finger hint calculation time.

Posts: 110

Post by revilo2 » 04-07-17 1:58 am

News from Dorien Herremans

Hi Olivier,

Sorry for my very late reply. The software indeed takes a musicXML score as input. But as it is, the code needs some cleaning up. I'm afraid at the moment I don't have time for it. Feel free to promote it on forums though, maybe a coder will make the parsing more robust (our software is opensource) :)


Posts: 7

Post by duncanisgoed » 06-28-17 2:23 pm

I attached a PDF of the paper.
Herremans Generating Fingerings for Polyphonic 2015.pdf
(431.83 KiB) Downloaded 205 times

Posts: 110

Post by revilo2 » 08-16-17 3:16 am

This is really interesting !!

Has anyone tested this C code ? I don't understand anything in programming and i wonder if there is an executable version of this code (with xml files as i/o)... ... s/overview

Posts: 12439

Post by Nicholas » 08-16-17 4:44 am

It looks like there's an .exe in both the Release and Debug folders, here. Haven't ever tried it, but if you download the whole repo, you might be able to tinker with it.

Posts: 110

Post by revilo2 » 08-16-17 9:54 am

It would be an amazing function to include in Synthesia, isn't it ?

Posts: 12439

Post by Nicholas » 08-17-17 10:26 pm

Yes! I am very excited about it. It's kind of the holy grail for a practice tool aimed at beginners: without the experience to know how to pick their own finger hints, our current (100% up-to-you) finger hint feature leaves a lot to be desired.

Integrating/developing this kind of technology is a long-term research project I'd love to tackle some day.

Posts: 110

Post by revilo2 » 08-18-17 8:28 am

Maybe you should contact Dorien Herremans, in order to use his code...

Apparently it works and it is free of use...

Posts: 12439

Post by Nicholas » 08-18-17 2:46 pm

Eventually, when we reach this item on the task list I may do something like that. I've collected a folder with around 20 different researchers' ideas on the problem over the years. About half of them include source code or pseudocode.

Beyond that, I've had a few of my own ideas on how to combine the best parts of a few of their algorithms and even how to speed them up. My goal is that Synthesia will be able to calculate finger hints in real-time -- under 1 second for an entire song. When I tried Dorien's Release executable a couple days ago, it took over 3 minutes to calculate the fingers for a 1 minute song. So that will end up needing to be 180x faster. :D

Post Reply