Fix Melody Practice mode

Synthesia is a living project. You can help by sharing your ideas.
Search the forum before posting your idea. :D

No explicit, hateful, or hurtful language. Nothing illegal.

Postby ignat980 » 08-04-17 7:13 pm

I've been actively using this program since 2014. I love this program, it helped me progress a lot in my piano practice. There is one problem I would like a fix for.

The melody practice mode is amazing, but I think it has one major flaw. Midi files with trills, glissandos, slightly offset chords (not at the same time), and live-recorded midis with small offsets are all the bane of my existence. This shouldn't be the case.

The problem I have is simple. Let's say I have a chord (many examples in the attached midi) that you have to arpeggiate quickly up, and I hit the next note of the chord at the same time as the first note, what happens? The correct note gets highlighted green and the supposed "incorrect" note is grey. Ok. The song progresses for a small moment, and then stops because I have to hit the next note, right? But wait, I thought I hit the other note too! It is highlighted gray right there! This is where the problem is.

This is what I want: grey/incorrect notes to be counted as correct when it is their turn to be pressed, aka when they turn green (if the hand selected is green). A grey note only happens when it is pressed. What if I want to intentionally hit the whole chord at once? What if my glissando accidently hits the next white key before the black key? I want to song to continue, I mean I hit the darn thing correctly.

One other thing which is related: Incorrect notes are counted when they are hit, not when they are let go. Oh how many times I intentionally hit notes (which are correct) just for synthesia to say "nuh-uh. You're too early, that was wrong, you're wrong, try again." Especially in trills! I think it should be changed so that incorrect notes are counted when you let go of a note and it is still gray.

Final thought if this is implemented, if you have to repeat the same note over quickly many times, obviously the song should stop until you lift your finger to hit the note again. It's not continue if note is pressed, but continue if the note was hit! Down and up :)

Undertale - Its Raining Somewhere Else.mid
Many arpeggiated chords in this song
(5.07 KiB) Downloaded 21 times
Posts: 6

Postby Nicholas » 08-08-17 12:42 am

ignat980 wrote:Midi files with trills, glissandos, slightly offset chords (not at the same time), and live-recorded midis with small offsets...

That's a fairly complete list! You are absolutely correct that Synthesia causes more friction than it does good in those cases. I've experienced that frustration myself.

There's even one more case that isn't quite the same but I lump into the same group because it's the worst offender of all: playing the first note following a loop repeat. That first note you aren't allowed to play early at all! The song has to come to exactly that position (and stop in melody practice) before Synthesia will register it correctly. :?

Like you mentioned in your final paragraph, this is something that can get a little tricky. Which notes should be allowed "early" and which later, etc.? In the case of trills, is it really so important that you switch between notes exactly the indicated number of times? Or should Synthesia be happy if you're trilling throughout that time period at some speed that is a reasonable approximation to the one indicated? (And if that case requires special behavior, how does Synthesia even detect a trill for that matter?! With MusicXML this becomes a little easier because it's explicitly notated as one.)

I worry that in order to tackle all of these situations with any kind of behavior approaching satisfactory, we'll need a kind of score following algorithm.
Posts: 10969

Postby ignat980 » 08-08-17 2:25 pm

Thanks for the reply.

Again, I don't think Synthesia should try to be smart about which notes should be allowed "early". I say all notes are allowed early. You could hit a note 10 measures early and it counts. The only thing: it counts only when it is hit eventually! You have to hold down that note until it hits.

Right now when you press a note, the keyboard shows the color as gray if the note is wrong and green (if say practicing the right hand) if it is correct. My proposition is that this needs to change that gray means "not yet hit" and green "has been hit". Say you have an arpeggio, you can play it as written, or just hit them all at once, and wait for the keys to turn green. Trills could be pressed both at the same time. I think it is up to the user on how they want to practice a piece, eventually they will get faster and will attempt to mimic the piece better. Then, the song in melody practice mode will stop only when a note is not pressed or when you have to hit a note on one that his been hit already (many consecutive notes).

This means incorrect notes are, actually incorrect. It's when you hit a key and it was gray when you let go.

Like, I think this is the solution. Very huge nice improvement to melody practice. You could even tie a "stale-ness" score to every note you hit (how long until the notes were hit), and then use it to subtract from the general performance score (to prevent people getting perfect scores by just lying down on the piano or whatever).

The score following algorithm you mentioned looks cool, but again, unnecessary. Could be another feature in a much later Synthesia version. What I'm asking is a simple fix to how incorrect/correct notes are interpreted.
Posts: 6

Postby Nicholas » 08-23-17 10:54 pm

ignat980 wrote:Say you have an arpeggio, you can play it as written, or just hit them all at once, and wait for the keys to turn green.

Hmm, this seems like it would encourage bad behavior. Granted, I agree that Synthesia is too strict today, but being able to play any note whenever feels like going to the opposite end of the spectrum. I suppose a "stale-ness" factor like you described would help but there is a fairly large difference between preventing the song from continuing (today's punishment for sloppiness) or making the number you attain afterward a little smaller (the proposed punishment for sloppiness) instead.

The former requires every user to care about it. The other only requires score-focused players to care about it. This feels a little like throwing the baby out with the bathwater, just to eliminate the frustration of playing a little too early.

This might not be a half-bad workaround in the meantime:

  1. Hold your Shift key while launching Synthesia to open the configuration window.
  2. Find the "Gameplay.NoteWindowUs" setting.
  3. Change the leading "4" on the rather large number to, say, "20" so you end up with 2 million microseconds (which is to say 2.0 seconds). That will make Synthesia scan forward 2 seconds instead of 0.4 looking for notes that can be played.
That won't solve all the frustration points, but it might help some of them.
Posts: 10969

Postby ignat980 » 08-25-17 1:11 pm

The extended note checking is pretty neat, but it doesn't address my issue: Even if I set it to 2 trillion microseconds, the song doesn't register me hitting the next correct notes if the song is stopped waiting for me to hit the current correct note in melody practice mode. Let's take a simple fast arpeggiated chord (you can see many examples in the MIDI file I posted in the first post on this thread), when the song stops waiting for me to hit the first note, if I (very quickly) hit the second note before the first one, the song continues because I hit the first note, then immediately stops waiting for me to hit the second note. And then I get mildly more aggravated with the program and repeat this for the other two notes in the chord. Then I restart the song trying to get it "perfect" according to Synthesia. And again every time I come across another chord like that. Which are very prominent in jazz music.

Interestingly, even with the default value of 0.4, I can hit that whole chord at once just right before the song stops for the first note and it plays fine (because all of the notes are within that 0.4 window).

All I'm asking is to add checking for correct notes being hit while the song is stopped (looking ahead the default 0.4 seconds). Like, all of my issues would disappear. A trill will get decimated like butter being cut with a hot knife because I'm actually hitting the notes. Glissandos? Oh yea, my fat finger hits 4 notes instead of 3, but now the song doesn't stop because it will register that I hit the note just a bit before.

When I'm talking about arpeggios, I'm not talking about 4 quarter notes. I'm talking sixteenths. Again, if the song is stopped waiting for me to hit the next note, it doesn't look at any other notes ahead by 0.4 seconds to check if the note I hit is correct. Do you get what I'm saying? I feel like this would be a simple fix.

Posts: 6

Postby jimhenry » 08-25-17 1:30 pm

Hmmm, if I understand ignat980 correctly, playing an arpeggiated chord with all the correct notes at time X will be accepted in Melody mode if X lies within the early acceptance window for all the notes. But if I don't play the chord before Melody mode stops to wait for the missing note, then only the missed note is accepted while the falling notes are stopped, even though the remaining chord notes are within the early acceptance window. If that is how Synthesia behaves, then I agree with ignat980 that this is incorrect behavior.

When Melody mode stops the progress of time Synthesia's response to anything that happens at the stopped time should be the same as if time was not stopped except for possibly having the additional effect of starting time.

P.S. I find these types of quirks in Melody mode so annoying that I hardly use it. And I usually make a special simplified version of the song if I do feel I must use Melody mode for some reason.
Jim Henry
Author of the Miditzer, a free virtual theatre pipe organ
User avatar
Posts: 1682
Location: Southern California

Postby Nicholas » 09-02-17 11:54 pm

Hmm, I've been letting this simmer in the back of my head for a week now and for the life of me I can't think of (or remember) a good reason Synthesia behaves differently after it has stopped in melody practice. I want to say it was a legacy thing that evolved over time. Its handling of "concurrent notes" (so you're made to play all the notes in a chord at once before it will continue) came along in pieces over several releases (many years ago).

The next option down in the configuration window ("Gameplay.NoteWindowUsAfterFirst") specifies the time window for all the notes that should be considered part of the same "beat" that need to be played in melody practice once things have been stopped. (That one is NOT a good candidate to increase indiscriminately because it will start lumping notes from adjacent beats in with what needs to be played "now").

... and while I was refreshing myself on what that option did, I still couldn't think of a reason the behavior should be any different whether the song is moving or stopped.

So the bad & good news is that while Synthesia 11's road-map is far too full to do an investigation and round of changes on this facet, Synthesia 12 will be concerned primarily with doing exactly that. It will be completely revamping the scoring/progress metrics and re-evaluating the pedagogical basis for lots of things. I just added this to the top of the list for the 12 release cycle.
Posts: 10969

Postby ignat980 » 09-03-17 12:06 am

Omg you don't know how happy this makes me feel :D :D :D . I am very glad that this is now on the roadmap.

Also, I'm a developer, if there's anything I can do to help speed this up just say so. I know Objective-C and Swift (the Apple languages) pretty well (and pretty much every scripting language), I know version control tools like git/github; and this project is one of my biggest passions (I tell people about Synthesia as often as I can). I would say automated tests/debugging are my strongest suit.

So, like hit me up if you're accepting any help.
Posts: 6

Postby IvanL » 10-03-17 2:23 am

Another dev here, throwing my axe in.
C/C++, C#, Java. MSc in comp sci. Have professional gamedev experience.

I doubt Nicholas is gonna open up his baby to strangers ;) , but just in case, I'd help out as well.
Posts: 39

Postby Nicholas » 10-03-17 12:12 pm

It's not completely unheard of.

I told the story a few years ago about a Synthesia user that went from "a stranger" to "has access to the private GitHub repo". There have been even smaller instances of contracting out little bits and bobs to others that didn't require access to the whole source tree: I'd send over just the couple C++ headers they would need to complete their task. That is how the chord detection code in Free Play came to exist.

The thing that I try to avoid are full-time positions. Being responsible for making a living from your own business is scary. Also making a living for your wife and (now 1.5 year old) child is downright terrifying. So, the fewer people I can add to the list of "if I make a big enough mistake, these people go without food or shelter", the more relaxed I'm able to sleep at night! On the contrary, part-time roles let me give people what essentially becomes "bonus" money on top of whatever they already do. That is much more pleasant. :D

So, then the challenge becomes finding bite-sized features that people can work on part-time. The chord detection stuff is a nearly perfect example of a system that doesn't interact with anything else. And while Timothy's first big task of porting Synthesia to Android did touch all of the low-level systems (input, graphics, MIDI, filesystem, etc.) it freed him from having to worry about any of my wacky app code. :lol:
Posts: 10969

Return to Feature Requests

Who is online

Users browsing this forum: No registered users