Live video feed of hands

Synthesia is a living project. You can help by sharing your ideas.
Search the forum before posting your idea. :D

No explicit, hateful, or hurtful language. Nothing illegal.
Nicholas
Posts: 11911

Post by Nicholas » 11-12-11 6:01 pm

jimhenry wrote:Maybe this would be a manageable way to provide hints with a live keyboard feed.
This is actually pretty close to what I had in mind. Though, using a few simple image transforms you can correct for camera lens distortion and draw the keys straight-on, orthographically the way they are in the interface today (even if the camera isn't positioned exactly above).

Also, with some combination of chroma-key and image subtraction (from a still frame of the keyboard by itself), I expect I'll be able to have the hands drawn on top of note indicators/coloring, and still do it fast enough for real-time use. Really my only enemy is going to be input latency from the camera itself. Intel's OpenCV library can do incredible amounts of work in just a few milliseconds.

User avatar
jimhenry
Posts: 1752
Location: Southern California

Post by jimhenry » 11-13-11 2:02 am

It would be interesting to find out if seeing your hands from top down rather than in front of you is disorienting or still helpful.
Jim Henry
Author of the Miditzer, a free virtual theatre pipe organ
http://www.VirtualOrgan.com/

ProgerMan
Posts: 9

Post by ProgerMan » 01-04-12 3:57 pm

I think this would be an excellent idea. I am really struggling to use Synthesia at the moment because I can't judge the distance between the keys yet. Being able to see a video view (do they do widescreen cams?) top-down would be a superb feature. :)

Lemo
Posts: 313

Post by Lemo » 01-16-12 8:41 pm

lemo wrote:Maybe it's more like 0.3sec than 0.5, but that's still a lot. You may not notice it when you slowly wave your hand to the camera, but it's different when you clap your hand for example (or play piano).
Very interesting variations with this spam bot, I feel honored
(You may now auto-destruct this message as well :p)
Stuff & experiments for Synthesia: Gramp v0.2SkinboxFireSynthVideoWebradio

Nicholas
Posts: 11911

Post by Nicholas » 01-16-12 9:18 pm

That was a pretty good one. I wonder if my little anti-spam thing catches stuff in signatures too. We'll find out. ;)

Post Reply