Synthesia 10

Archived development update discussion from past versions
Archived development updates.
Nicholas
Posts: 13135

Post by Nicholas »

Just a little status update since I've been quiet lately: all sorts of work has been steady for weeks now. The next "quality of life" preview doesn't just mean your quality of life. It also includes the Synthesia codebase's quality! I talk about technical debt from time to time, but things had really reached the breaking point in the last few core systems that haven't ever been revamped since writing them originally around 2006.

Most of the front-end UI stuff is very nice to work with these days, but the oldest things -- MIDI loading/reading/flexibility and the keyboard bookkeeping/drawing code -- is nearly unchanged since the first release, outside of haphazardly jamming new bits into the cracks. The keyboard/falling note code saw one (basically mandatory) update for the iPad release. But that only cured about a third of the madness.

Some of the upcoming features looming on the horizon like track splitting, changing hands during play, and MusicXML would be foolish to attempt without going through and making those parts of Synthesia sensible. It's kind of a fun trip down memory lane... except I want to scold my 2006 self every time I rediscover one of these bad decisions. I will try and learn from this experience. ;)

There have even been a few freebie optimizations that have come along for the ride, if only because each new implementation is a little smarter than it was before. Fixing these things with the perspective of where Synthesia is going to be in 3-5 years from today vs. trying to write the original version as quickly as possible (just a couple years after I'd finished college and didn't really know what I was doing :lol: ) makes a huge difference!

In any event, this first part -- paying off the "debt" -- is going to make this preview take a little longer, but like all the other major system revamps I've done over the years, we get more than one benefit:
  • New work is greatly accelerated.
  • Adding big new features isn't nearly as scary, so I'm more likely to go in that direction.
This is good news for everyone. :D Anyway, there won't be anything to see for a little while longer yet, but I've been really happy with the state of the code as this progress continues.
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

Losing my measure based jumps
That's me in the corner
Trying to keep up with you
Everybody hurts now, da da da da daaaaaa :)

Which song is it?

Great decision, and updates, thanks for everything!
User avatar
jimhenry
Posts: 1899

Post by jimhenry »

I thought that I heard you laughing
I thought that I heard you sing
I think I thought I saw you try

"Losing My Religion" R.E.M.
Jim Henry
Author of the Miditzer, a free virtual theatre pipe organ
http://www.Miditzer.org/
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

Very good, correct, I admit, was not that difficult.
Nicholas
Posts: 13135

Post by Nicholas »

TonE wrote:Losing my measure based jumps...
Actually, measure-based FF/RW is already done and it turned out rad.

I should have done that years and years ago when you first requested it, TonE. :lol:
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

Nicholas wrote:... and it turned out rad.
What does this mean, sorry for my lack of English?
Nicholas
Posts: 13135

Post by Nicholas »

It's a (silly) old expression from the 1980's that means "really cool" or "awesome".

Sorry for the confusion. ;)
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

So you implemented it in an unreleased version? Why was it 'awesome' for YOU? What did you like mostly, in which context, my context you know already?
Nicholas
Posts: 13135

Post by Nicholas »

TonE wrote:So you implemented it in an unreleased version?
Yep.

So far in my testing it just feels "right". Before the RW/FF never took me exactly where I wanted to go. This one seems to. So much so that it's now the default behavior. (3 second RW/FF is gone forever.) Especially in melody practice, jumping to the start of the current measure (or the start of the next) -- assuming there is a note at the beginning of the measure -- means you are immediately at a note and are ready to start playing again. That is fantastic for faster practice iteration time.

It feels so strong that now I'm worried iPad users will be missing out. By default RW and FF aren't mapped to anything on the iPad. So, I'm considering changing the behavior of the Back/Forward buttons at the top of the screen to mean "Back/forward one measure". Though I would lament the loss of "Back/forward one bookmark" being mapped to those. I wonder if I could add something like "Hold it for 250ms and you'll get the bookmark behavior. Just tap for the single-measure behavior." That might solve both problems. Though some educational material would have to point that out someplace. There is definitely room for improvement in the tutorial. That might be the right place for that.
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

I see, thanks for the explanations.
Enetheru
Posts: 7

Post by Enetheru »

you have a couple of options, tap, press, double tap, press+swipe direction
Nicholas
Posts: 13135

Post by Nicholas »

Alright, I've burned through most of the technical debt slated for Synthesia 10. (Things are feeling nice with the keyboard drawing code about 30% smaller than it used to be and more flexible than ever. Even better, the MIDI loading code lost about 40% of its lines while fixing several bugs and speeding a few other things up!)

After that it was very easy to add a short "cooldown" to the various states a keyboard key might undergo, including: being held down, showing sparks, showing an upcoming note marker, or sending out the signal for lighted keyboards. (All four of those are used in subtly different ways.) In any event, the new "cooldown" means successive notes now show a tiny 1-2 frame break in the key being activated in any of those ways. It's a visual indication that the key should actually be released and pressed again. Until now a string of 64 sixty-fourth notes had the same visual appearance on the keyboard as 1 whole note. Now you'll see 64 individual key presses.

The beautiful merging of the simple and advanced song views -- while somehow dropping most of the complexity of both in the process -- is coming up after the 2 or 3 "Quality of Life" development previews. Though I just decided to rein in a snowball effect that had started happening:

The ultimate destination for the song setup screen is to only talk about hands insofar as you describe which tracks "belong" to which hands. Ideally you aren't specifying which hand you are actually interested in playing until you're actually playing! At that point you'd be able to change hands on the fly. Real practice involves much more dynamic switching of hands-apart/hands-together than Synthesia allows today.

But! If hand-selection moves to the play screen, that has some major implications for song scoring and song progress (hence this topic). That would mean delving into all of this new scoring/progress work (not to mention pushing hand-selection to the play screen involves a bunch of work on its own).

So, for the sake of making regular progress, dynamic hand switching and the new scoring stuff will have to wait. It was never officially on the schedule in the first place and only crept in while I realized just how clean the upcoming song settings screen could look. For now I'll just wedge hand-selection on that screen someplace and leave scoring alone for the 10 release. It's a shame... "10" is a pretty significant number, and all of those changes would have really earned them. Still, track splitting is a big endeavor and with all these long-standing quality of life improvements, I think it'll still be worthy of the new title.
MarkPas
Posts: 10

Post by MarkPas »

Just wanted to throw my "2-cents" into the fray... I really enjoy the software and because of that I want to use it more! My primary use is for learning and the thing I want the most is a way to choose music to learn and use Synthesia to help me break it down and practice. That being said, whether hand-splitting or MusicXML is what I need, I can only hope that some method of importing a composition into Synthesia for practicing that keeps the notes on the intended clef so that the left and right hand parts are assigned correctly will happen in the near future.
User avatar
jimhenry
Posts: 1899

Post by jimhenry »

Based on my impression of the number of posts regarding hand assignments in one form or another, I am pretty sure that is more important than MusicXML. If the current sheet music display is enhanced to use the hand assignment information in assigning staves, then I think the demand for MusicXML will be reduced considerably. I am not convinced that most of those waiting for MusicXML realize that they will need something much more than MIDI files to derive any benefit from the MusicXML feature. MusicXML is a gateway to a bigger future for Synthesia but there is no instant gratification from it.
Jim Henry
Author of the Miditzer, a free virtual theatre pipe organ
http://www.Miditzer.org/
Nicholas
Posts: 13135

Post by Nicholas »

jimhenry wrote:I am not convinced that most of those waiting for MusicXML realize that they will need something much more than MIDI files to derive any benefit from the MusicXML feature. MusicXML is a gateway to a bigger future for Synthesia but there is no instant gratification from it.
Jim is correct here: it's going to take a lot of good MusicXML content to make MusicXML support really shine and that content doesn't exist today for the most part. As far as standards go, it feels like MusicXML has reached critical mass -- all the major commercial sheet editors can export to it now and some use it as the native format -- but we're still in the early part of the adoption curve. The final output most people still look for is a PDF or printed page. Once the benefits of keeping and sharing the intermediate MusicXML file are more widely known, I suspect we'll start to see more of them crop up all over the place.

That said, there will be some "free" improvement in Synthesia's MIDI-to-sheet generator that comes along for the ride. Once Synthesia is capable of expressing all the things MusicXML can talk about, the MIDI-to-sheet generator will have access to all those new toys. So, in the same Synthesia release that MusicXML gets strong, the plan is to knock out a bunch of long overdue improvements to the MIDI-interpreting side of things, too.

Better staff selection is one of those long-overdue improvements.

And Jim's idea of using the hand-assignment information to improve staff selection even further is a very good one!
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

I wanted to write yesterday, 'Jim is right', but did not. :D Now reading Nicholas sees it similarly, is ..., aahh, as expected. :P

Concentrating on
a) auto-hand separation for staff and
b) pdf display, e.g. as transparent background where SumatraPDF is smiling at us. It does not have to be even transparent, now where we have windowed Synthesia we can use one half of the screen for Synthesia the other for SumatraPDF. Only Synthesia and SumatraPDF have to talk to each other, auto-switching pages in SumatraPDF correctly, in sync to Synthesia. My suggestion you know already how this would work. Copying current measure number mXX into clipboard. AutoHotkey listening to those changes and sending appropriate commands to SumatraPDF for page turning or page jumping. This solution is not limited to SumatraPDF only, LilyPond and friends in the same way.

For hand separation algorithm, something like this might work, to some degree, not perfectly in each situation: For each measure or quarter beat do: Check greatest available pitch gap, above some threshold *maybe*, like 6 semitones, then use this gap as separation point for hands. Having a 'visual algorithm' would be great, where you could test this or another algorithm against test files where correct hand separation is available.
revilo2
Posts: 135

Post by revilo2 »

While waiting for MusicXML or PDFDispaly, Why don't you simply display fingering on the actual sheet music bar (In Synthesia 10, please...)

I've been waiting for this for years (i don't think i'm alone) and, as for me, it will be VERY HELPFULL to progress.
MarkPas
Posts: 10

Post by MarkPas »

It was great to read all the feedback...thanks! To bring my original thought back into focus and since I'm a database programmer not an app programmer so particular methods are outside my expertise. I was merely trying to express my strong desire to use Synthesia to practice/learn other musical pieces by using some method of importing a song into Synthesia and then breaking it down for practicing by individual hand parts. Whether by hand assignments or MusicXML or supercalifragilisticexpialidocious methods... that's for Nicholas to decide, prioritize and bring to fruition. In any case, I enjoy the program for what it is today and am a better player because of it.
User avatar
jimhenry
Posts: 1899

Post by jimhenry »

Nicholas wrote:And Jim's idea of using the hand-assignment information to improve staff selection even further is a very good one!
It is a good idea but it is not mine. Someone else pointed out that the fingering information should be used to aid staff assignment.
Jim Henry
Author of the Miditzer, a free virtual theatre pipe organ
http://www.Miditzer.org/
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

Yes, in those cases where fingering is available for sure. What doing in the rest cases? Simply adding fingering? At least in a way like: per measure or per beat, Click, everything above click pitch is right hand. So you can click through the song. Click, click, click, click, click. Until, finished.
Locked