Pitch Interval Visualization for Game Play and Free Play

Synthesia is a living project. You can help by sharing your ideas.
Search the forum before posting your idea. :D

No explicit, hateful, or hurtful language. Nothing illegal.
Post Reply
jtmcginn3388
Posts: 2

Post by jtmcginn3388 »

Dear Synthesia Development team,

Thank for making a great product. What I love about your app- from someone who never really got good at sight reading- is that it cuts out the process of converting sheet music into a visual representation of the keyboard. For me at least I need a mental image at least of the piano in order to play and understand it. I have been struggling with finding some way to learn intervals as I learn music through Synthesia. How would that look? To start, I was thinking for in free play or game mode use the upper left corner note heads up display (HUD) for the interval made between the first note played and the next and then have it keep tracking the new interval played after hitting the next note and so on. I was hoping the display could be more like a lighting of the background behind the keyboard with a visual color coded representation of the interval just played. We could use the primary ROYGBIV colors (with accompanying darker shades for the minors). And just to be clear the colors are not staying consistant with scale number but the intervals. Like 1 whole-step should be Red, if hit C to D or F to G. So that would be relatively easy to do I imagine. But then I have a more ambitious challenge. I was thinking how can we both make the user visually aware of the interval they are playing for more complex music. I was thinking you could use "strings" or linear aura graphics of the primary colors that tie notes together on the screen. For C to D, as the notes fall you can see that c's note blokc thats falling towards the piano is connected by a red linear aura or string graphic that binds to D. This gives the user now 2 visual cues to the note to hit. It also reveals the interval patterns of the music within the song. I am hoping that with time you can see these intervals and use them for melody and chord formation quickly on the fly. I was also hoping that the interval strings and notes could also be recorded in the free play buffer you guys have in free play mode. And a way to record an improvised song.

I humbly ask your assistance. Again I love this software and I feel like this would be a great addition for many people.
Nicholas
Posts: 13135

Post by Nicholas »

I actually like a number of these ideas. I think it would be a little tricky to disambiguate whether a new key press was intended to be added to the currently-played chord or whether it's been long enough to simply register as the "next interval", but with some experimentation that feels like a solvable problem.

There are still one or two technical hurdles left before we can arbitrarily control the colors of notes (right now the six color choices are baked right into the app's textures), but once we had that capability, this would be fairly straightforward to add.
User avatar
jimhenry
Posts: 1899

Post by jimhenry »

These ideas reminded me of something I was taught long, long ago when I was just starting to learn piano, namely thinking in terms of the interval to the next note to be played. If I remember correctly we were told to think in terms of up/down and number of half-steps. So if you had the notes C E A G, you would play C and then up-5, up-6, down-2. So my take on this would be to apply the idea simply to the top notes on the assumption that they are the melody and as soon as a note is played display the interval to the next melody note.

Somehow I think I am wrong about the half-steps as that seems difficult to digest quickly. Maybe someone who has encountered this technique more recently can clarify.
Jim Henry
Author of the Miditzer, a free virtual theatre pipe organ
http://www.Miditzer.org/
jtmcginn3388
Posts: 2

Post by jtmcginn3388 »

Thank you for your response. Based on what you have said I would like to expand upon what I See as the eventual goal. I want to quickly recognize intervals while playing with a color code but then also pair it ib my mind with the number of half steps between the notes. So another way to pair them (in addition to pairing them with strings or linear auras on the descending notes) would be to perhaps have the keys you need to press to play the gamemode have the circular guide that some songs have but then apply a color matching the interval played to the half-step keys in between the notes of the interval (again on the keyboard at the bottom).
So I understand that you will have to work on the timing issue. I am hoping that if you have to make cut offs for what constitues a chord or just a basic note-to-note melody that you have some adjustments in the settings to control that time zone of interval recognition (expand or contract the recovery time so to speak before a new note can be registered as part of the last note). Now I also see how this would get messy with particularly complex peices of music. Are we going to have lines going all the way across the screen? Have keys between a C3 and C2 all light up some wierd color you had to use cause you ran out? (I was also thinking symbols might help if we get low on colors or its found that the colors aren’t different enough. Or better yet add symbols to convey the scale number in addition to the interval).
I was thinking this system would work best this way...
Inagine a beginner-intermediate user sitting to play a song simple song like marry had a little lamb. They progress through it while seeing the intervals (and scale number) on both the falling notes and the guide keyboard. They learn about intervals but also learn a new way to quickly scan the screen for playing instructions. Now imagine they move onto learning chords. Same thing. Now imagine having them try to put it together for the first time with a song that perhaps has chords in the right hand and single-note-to-single-notes melody (or vice versa). I am thinking it might be easier to see the whole keyboard and where the music is going with a path paved with intervals. Like the rest of the program it would take time to learn but once you do it would be extremely powerful. Like how I think most ppl see the keyboard who use synthesia to play. They see the guide bars for the first smaller half of the octave and the other larger half and see a notes place in that grid without the intervening notes and know quickly what to hit. I think adding more data would give ppl the ability to start anticipating where a song will go based on the patterns they are seeing in the intervals (and scale numbers)

Again thank you for your time.
Post Reply