Automatic Fingering Prediction 0.1ish

Synthesia is a living project. You can help by sharing your ideas.
Search the forum before posting your idea. :D

No explicit, hateful, or hurtful language. Nothing illegal.
FireBreather
Posts: 6

Post by FireBreather »

Finally bought the "Learner pack" reading through these posts,

im abit confused,
is there anyway off activation "Finger hint Prediction" i know you can manually place the finger hints, but i dont know where to place my fingers
So is there anyway to activate this finger prediction or is it still just a running idea

P.S. im on a mac if that helps
Pianotehead
Posts: 325

Post by Pianotehead »

There's no automatic fingering prediction yet in Synthesia, if it's been put in I missed it. If you're wondering about finger use for a particular song, check out how others are playing it, for example if there are some covers or tutorials on YouTube. Just make sure it follows the arrangement you have, at least approximately.
Nicholas
Posts: 13132

Post by Nicholas »

Yeah, this topic is covering an early experiment by Frost... who appears to have (sadly) abandoned the project. I would love to see this picked back up and incorporated into Synthesia some day.

Still, it is definitely a challenging problem. Frost was using pretty recent research done by universities to develop what he'd done so far.
Pianotehead
Posts: 325

Post by Pianotehead »

MIDI Illustrator has fingering prediction. I tried it few months back, and since I can't change into an octopus (like in Prometheus) I decided to stop using it. That's how far out of reach the fingering suggestions were.
davidddd
Posts: 11

Post by davidddd »

And if you want to take us RIGHT INTO THE 21st century.... :ugeek:

A graphical library of hands that are reaching that appear on the keyboard actually showing you where that fingering would place your hands.

We can dream. :roll:
Nicholas
Posts: 13132

Post by Nicholas »

davidddd wrote:A graphical library of hands that are reaching that appear on the keyboard actually showing you where that fingering would place your hands.
You have no idea how long I've wanted that. I think it was before I even sat down to write the first line of code for Synthesia.

The little hands that slide in when you select a hand in the simple song view (in the latest dev preview) are actually 3D rendered. I licensed a super high-quality pair of arms/hands that were all pre-wired for animation. Don't you second-guess me! I'll do it! :lol:

Seriously though... how cool would that be?

There is another piano instruction thing out there that tries to do it, but with 2D hands that stretch in strange ways. The effect is more creepy than anything. Unless I were able to show proper posture and technique clearly such that it wasn't distracting, it would probably be more eye-candy than anything. If I do it, I want it to be useful.
davidddd
Posts: 11

Post by davidddd »

Well, there are some articulated hands and body parts that are manipulatable within Solidworks - that's what I use as a mech e, but I think those models are pretty pricey. On the other hand, Poser may be able to get you what you need. Dance departments use Poser to capture choreography, and you could use either to generate the series - reaching a total of 12-15 keys, leaves a very finite number of possibilities. The user could set hand width, etc...

http://www.youtube.com/watch?v=4_9pevkel0g

But yeah, it may be pie in the sky.

D
andwan0
Posts: 6

Post by andwan0 »

This is similar to "Midi Illustrator 2" intelligent finger numbering system too.

Just curious, how are the finger numbering stored for Synthesia? Are they stored in separate files? If so, possible to export & import the finger numbering from Midi Illustrator 2 program?
Nicholas
Posts: 13132

Post by Nicholas »

The finger hints use our own format, described here. You'll find that in fingers.xml in the Synthesia data directory, or if you use the data export feature in the metadata editor, you can keep the hints alongside your MIDI files inside .synthesia files.
carlosalbertopavon33
Posts: 1

Post by carlosalbertopavon33 »

HELLO JUST AS run the program with the parameters JPG IMAGE WORKS BUT NOT AUTOMATIC PREDICTIVE SYSTEM TO DO AFTER PUTTING ALL THE VALUES ?, JUST CLOSE THE WINDOW?
revilo2
Posts: 135

Post by revilo2 »

Hello,

By searching for an auto-fingerings software for piano sheets, I found on Google :

http://dorienherremans.com/automatic-piano-fingerings

It seems very cool. there is a source code. unfortunately, i was unable to use it (I am not very talented in computing... Maybe Someone could... (Nicholas, Frost, ... )

Anyway, i took the liberty to contact Dorien Herremans, the author of this research with the aim of becoming known him Synthesia's forum and especially the Automatic Fingering Prediction section...

Nicholas, Maybe, would like to contact him....
Nicholas
Posts: 13132

Post by Nicholas »

Ooo, new research. I like seeing what people come up with. (I was happy to find that I'm familiar with almost all of the papers in his paper's bibliography. I'll have to take a closer look at the other new ones, too.)

The 110 second search time is a little scary. Although, I've got a handful of strategies in mind that reduce the size of the problem (or increase the speed of the algorithm) by a couple orders of magnitude, so when we do get a chance to finally tackle this, the target is still sub 1-second finger hint calculation time.
revilo2
Posts: 135

Post by revilo2 »

News from Dorien Herremans
http://dorienherremans.com/automatic-piano-fingerings


Hi Olivier,

Sorry for my very late reply. The software indeed takes a musicXML score as input. But as it is, the code needs some cleaning up. I'm afraid at the moment I don't have time for it. Feel free to promote it on forums though, maybe a coder will make the parsing more robust (our software is opensource) :)

Dorien
duncanisgoed
Posts: 7

Post by duncanisgoed »

I attached a PDF of the paper.
Attachments
Herremans Generating Fingerings for Polyphonic 2015.pdf
(431.83 KiB) Downloaded 281 times
revilo2
Posts: 135

Post by revilo2 »

This is really interesting !!

Has anyone tested this C code ? I don't understand anything in programming and i wonder if there is an executable version of this code (with xml files as i/o)...

https://bitbucket.org/mballiauw/piano-f ... s/overview
Nicholas
Posts: 13132

Post by Nicholas »

It looks like there's an .exe in both the Release and Debug folders, here. Haven't ever tried it, but if you download the whole repo, you might be able to tinker with it.
revilo2
Posts: 135

Post by revilo2 »

It would be an amazing function to include in Synthesia, isn't it ?
Nicholas
Posts: 13132

Post by Nicholas »

Yes! I am very excited about it. It's kind of the holy grail for a practice tool aimed at beginners: without the experience to know how to pick their own finger hints, our current (100% up-to-you) finger hint feature leaves a lot to be desired.

Integrating/developing this kind of technology is a long-term research project I'd love to tackle some day.
revilo2
Posts: 135

Post by revilo2 »

Maybe you should contact Dorien Herremans, in order to use his code...

Apparently it works and it is free of use...
Nicholas
Posts: 13132

Post by Nicholas »

Eventually, when we reach this item on the task list I may do something like that. I've collected a folder with around 20 different researchers' ideas on the problem over the years. About half of them include source code or pseudocode.

Beyond that, I've had a few of my own ideas on how to combine the best parts of a few of their algorithms and even how to speed them up. My goal is that Synthesia will be able to calculate finger hints in real-time -- under 1 second for an entire song. When I tried Dorien's Release executable a couple days ago, it took over 3 minutes to calculate the fingers for a 1 minute song. So that will end up needing to be 180x faster. :D
Post Reply