Synthesia 11 Release Estimate

Try new versions before anyone else!
Always the latest dev version: WindowsmacOSiPadAndroid (or .apk)
Please report comments and bugs!

Your data hasn't disappeared: PC/Mac development previews store their data in a different place. Details here.
User avatar
jimhenry
Posts: 1907

Post by jimhenry »

I wonder if MuseScore can read all, or at least most, of the MusicXML possibilities. If so, then supporting only MusicXML that MuseScore writes in Synthesia, and relying on a read/write cycle in MuseScore for anything else might be a good starting point for Synthesia support of MusicXML.
Jim Henry
Author of the Miditzer, a free virtual theatre pipe organ
http://www.Miditzer.org/
Nicholas
Posts: 13170

Post by Nicholas »

I'm not too worried about it either way. Again, this has been a reasonably pleasant experience so far (even if there are lots of possibilities to cover) that hasn't exceeded any of the existing MIDI-based infrastructure (yet).

The status update was more about the slow velocity due to real-world stuff than it was about any struggles with the work itself. I was mostly just chatting about the progress so I'd have something else to say besides bad news. :)

When it's all said and done, I know Synthesia is only going to use a subset of the spec when generating sheet music (easily omitting <harp-pedals> among dozens of other examples), but the goal is for the actual notes loaded from the file that appear in the falling note area to always be correct regardless of what's in the file. I want Synthesia's MusicXML compatibility (at least in that sense) to be as good as it is with MIDI files: I like being able to claim "you can load anything" without any caveats. Of course there's always the "garbage-in, garbage-out" problem and there will certainly be some files you probably won't want to load, but I'd prefer covering a few different app's peculiarities to telling people they have to re-save it someplace before Synthesia can handle it.
User avatar
jimhenry
Posts: 1907

Post by jimhenry »

Nicholas wrote: 08-13-21 1:25 pm (easily omitting <harp-pedals> among dozens of other examples)
I don't think I have ever seen a post from someone using Synthesia to practice harp music. But, from what little I know about the harp, it is actually not so different from a piano. So maybe harp pedal notation should be toward the top of the list of omitted items that might be added later.

Supporting drum notation will probably be more work than harp notation. But if there was support for mapping falling notes for the basic keyboard drum sounds, probably less than ten, to the appropriate sheet music drum notation, the ability to use Synthesia to learn keyboard drumming would be an interesting extension of Synthesia's capabilities into an area that probably is of a lot of interest but underserved with learning resources. One of the bigger problems might be that drum notation relies heavily on repeat notation.
Jim Henry
Author of the Miditzer, a free virtual theatre pipe organ
http://www.Miditzer.org/
Hobbes
Posts: 8

Post by Hobbes »

All this work on MusicXML looks awesome! I understand that this is a very satisfying challenge for you.

However, in my opinion, pitch-based color for falling notes should have a higher priority in your list because:
- all guitar hero games have this feature
- it's even more complicated to find your way with the 88 keys of a piano than the 6 strings of a guitar
- this feature is very close to the initial purpose of Synthesia and it would help a lot piano beginners
- it seems easier to implement than MusicXML as there are fewer parameters

Thank you for your work and all my wishes for your real-world stuff.
Nicholas
Posts: 13170

Post by Nicholas »

Pitch-based color is coming before Synthesia 11 is complete. :D

Right now it's on line 631 of the list (or 208 lines from the end if you want to use the chart in the top post as a countdown).
Hobbes
Posts: 8

Post by Hobbes »

Then, reverse the list! :lol:
oc67
Posts: 5

Post by oc67 »

Hi,

Can you add an opinion poll on all the points to do or complete in your program?
This would allow requests to be prioritized by popularity
(i don't speak about bugs)

What do you think of this proposal?

thank
Nicholas
Posts: 13170

Post by Nicholas »

A good portion of the list is under-the-hood improvements (not necessarily bugs) that I'm not sure anyone would ever vote for, but that I'd still like to add. It's hard to justify any one of them as necessary, but taken as a whole I suspect it's going to be whatever the opposite of "death by a thousand cuts" is. (A thousand band-aids?)

Software today is suffering from... well, a lot of things. As a point of professional pride, I'm doing my best to avoid being part of the problem.

So, yes, that does mean a necessarily slower pace, but the alternative is to try to continue tacking more and more complexity on until the app collapses in on itself. :grimace:

Bringing it back around to your question of voting: much of the list isn't up for debate and the rest of it is either directly applicable to MusicXML or things I've already promised to users. So the items themselves are fixed. I suppose voting might shift the order a little... but many of the tasks have their own built-in ordering dependency. (I can't draw ties or slurs from MusicXML until Synthesia knows how to draw closed, anti-aliased Bezier curves. They're two separate tasks on the list but no amount of voting for ties+slurs is going to bump that item ahead of its dependency.) Again, this is true of so much of the list that I'd worry there's very little to shift around.

There are exceptions, of course. Recently jimhenry made a good argument that the basic MusicXML parsing step could be pushed forward rather dramatically. So, I did just that for the 10.8 release.

In the end, even if it's not visible, there is still a kind of "voting" going on by my carefully considering all of the feedback and suggestions I hear from users. The things that can be, do get shifted around in the list pretty regularly.
User avatar
jimhenry
Posts: 1907

Post by jimhenry »

Nicholas wrote: 02-10-22 2:10 am Software today is suffering from... well, a lot of things. As a point of professional pride, I'm doing my best to avoid being part of the problem.
This seems like where the description of "life by a thousand band aids" should be applied. :lol:
Jim Henry
Author of the Miditzer, a free virtual theatre pipe organ
http://www.Miditzer.org/
Nicholas
Posts: 13170

Post by Nicholas »

The tragic part is that I'm also responsible for the complexity that's already there. I started this project a couple years out of college and now, sixteen years later, I can tell immediately upon opening any given source code file that, "oh, this part was written by Intern Nick" (as opposed to "haggard, old, battle-hardened Nick"). :lol:

There are maybe 3 or 4 distinct coding styles that have come and gone over the years and whenever I encounter the earliest one when I'm going in to make a change or add some feature, I know I'm in for trouble.

Still, it's getting there. Slowly.

This is a low-level, nitty-gritty, "bare metal" programming talk (hilariously dunking on the use of C++ as too high level a language, delivered at the premiere C++ conference to C++ programmers). So I wouldn't recommend it unless that's something you're interested in (although he's got a pretty quippy answer during the Q&A starting at 1:17:40 that may be entertaining to a general audience.) It was recorded in 2014, I think I first saw it in 2016, and it took a couple more years to really sink in.

I saw this joke a few years ago, around the same time. There's a little bit of inside baseball going on there, but the general idea is that as you develop as a low-level systems programmer, you discover the allure of all the fancy toys built into your language, but eventually learn that they're more trouble than they're worth. When I discovered that video, I was knee deep in the middle panel. Now I'm almost ready to make the transition to the last one.

The horrifying discovery that started leading me down that path came when I found a clever way to ask Microsoft's C++ runtime to tell me how often I was asking the system for more memory. (Technical version: they give you a hook that will run your callback whenever "malloc" or "new" is called so not only can you track the number but you can also see exactly where in the code the request was made.) I wondered how bad it could be...

Pretty bad.

On the play screen, with just about every feature disabled, as of Synthesia 10.8, with the song paused(!), the app asks for new memory more than 1,500 times per frame. Much worse: if you turn on any label mode (even just "Octaves"), that number jumps to more than 4,100 allocations per frame! So, assuming the usual 60Hz, Synthesia is banging on the memory allocator around a quarter million times per second. (This scales linearly with the number of notes on the screen at once. These numbers are from a relatively "intense" song. For a simple song you get ~700 / ~2400 with no labels vs. labels enabled.)

Whenever I actually let myself stop and think about it, I become disgusted that I let it spiral out of control like that.

It is only because we have these fantastically powerful super computers that an absurd number like that can fly under the radar in the first place. At least I'm in good(?) company. Each key press in the address bar in Chrome used to cause 25,000 allocations. :? This same kind of carelessness is why browser tabs idle around a couple hundred MB of RAM these days. Abstractions sloppily built on top of abstractions, ad infinitum.

Assuming "Watch and Listen Only" with a completely hands-off user (no MIDI, mouse, keyboard input, etc.), Synthesia knows precisely 100% of everything that is going to happen during the course of the entire song, completely in advance... so that number should be zero allocations per second (after the initial loading).

But to get there would take a dramatically different programming style. Something closer to pure C. I'm not crazy enough to rewrite everything yet, but Synthesia 10.9 is going to have a new label system (with near-zero allocations) and I have vowed that each release after that is going to at least halve the number of per-frame allocations on the play screen.

These changes won't really do anything visible to the user unless you're on a really low-spec machine, but fixing all of this will help me sleep better at night. (If I were forced to pick silver linings: your tablet battery will probably last longer after these changes and it might help pave the way for something like a Raspberry Pi version of Synthesia someday. Those aren't the reason I'm doing it, but they're benefits that will come along for the ride.)
revilo2
Posts: 137

Post by revilo2 »

Hello

These items are part of your internal list before Synthesia 11 ? (because the chart doesn’t seem to be changing)
Nicholas
Posts: 13170

Post by Nicholas »

I explained some of the recent lull in development over here. (Sorry!)
Nicholas
Posts: 13170

Post by Nicholas »

Progress Update!
Nicholas wrote: 02-10-22 11:18 pmOn the play screen, with just about every feature disabled, as of Synthesia 10.8, with the song paused(!), the app asks for new memory more than 1,500 times per frame. Much worse: if you turn on any label mode (even just "Octaves"), that number jumps to more than 4,100 allocations per frame! So, assuming the usual 60Hz, Synthesia is banging on the memory allocator around a quarter million times per second.
Well, this was a lovely result just now. I got through the first stage of reworking the label system and there have been some improvements:

• Per-frame allocations with labels disabled: from 1,500 before, down to 195 now.
• Per-frame allocations with labels enabled: from 4,100 before, down to 216 now.

Which is to say that enabling labels only adds ~20 per-frame allocations (instead of 2,600) and it no longer scales with the number of notes on the screen. :D

That's plenty of under the hood optimization for 10.9, so I'll move on to things you guys will actually be able to see. :lol:
Drali.g

Post by Drali.g »

Hello
According to average slop of autogenerated progression chart , Synthesia 11 will be finished somewhere between 2025 to 2026 :shock: !
really have to wait until then to get it? :grimace:
Nicholas
Posts: 13170

Post by Nicholas »

It's very possible. It's also possible that it might happen a year sooner or even several years later than that.

That said, from the top-post: "Granted, it's a rather free-form document where an easy task might have a half-dozen lines of detail but some other much harder task may just take one line. There's lots of white-space to help keep things a little better organized, too. So, I will admit it's not the best metric. But it is a metric!"

So it's also possible that the next hundred lines on the list might only take a week. It may appear that the last couple months have included little or slow progress on the graph, but I've been rather happy with the steady work that has been going on. We had just happened to hit a section of the list where single lines were whole, reasonably large features. (And during two of them, things came up where it made more sense to bump something up from a few hundred lines down.)

Sections of the graph may appear linear, but then you find notes like this (currently) on line 308:
The Synthesia 11 Task List wrote:==> From here, down is a disheveled mess with 60%+ unnecessary stuff that should be filed under "Deferred" or deleted outright
Like I wrote there, once I get to that point in the list, I expect to rapidly disqualify something like a hundred lines in just a few days as I sift through some of the stranger old notes and lower priority requests I've taken down from before the 11.0 feature list freeze that prompted this graph in the first place. You may be happier with the slope of the graph at that point than you are now.

Drali.g wrote: 06-27-22 6:54 pmreally have to wait until then to get it? :grimace:
One more thing to consider: yes, Synthesia 11 may still be years away, but you won't have to wait that long for 95% of it. You can already load MusicXML files (which was originally planned as an 11.0 feature). And between the nine or so planned releases between now and 11.0, you'll be getting a steady stream of new features leading up to it. The graph reaching zero is just an arbitrary finish line I've drawn in the sand at this point.

There are many (most?) users that aren't particularly interested in improved sheet music. Balancing their interests is also a bit of a tightrope walk. So I try to include something for those users in each release, too. And for my sanity, I try to tackle at least one of the embarrassments in each release as well.

While all of that hopefully makes things more varied and interesting for all Synthesia users, it does mean you're watching the progress of a rather long-term roadmap at this point.
Drali.g

Post by Drali.g »

Getting 95% of Synthesia 11 features through updates is really promising!
You already know about sheet music clef problems. For all serious piano learners who practice with sheet music the music notation is essential. I can't say the correct music clef is how much of these 95% but it is definitely very important for us.
So that would be very nice if we can get that on earlier updates sooner.
We only care about G-clef and F-clef, the G-clef is on the right hand most of the time, for the left hand it is either G-clef or F-clef.
Clef changes through the song can stay for the later updates but these earlier correction make us more than happy.
Benjamin_J_Crawford
Posts: 4

Post by Benjamin_J_Crawford »

Hello,

Im just dropping in to say thank you so much for your software. I can not begin to describe what an amazing gift it is to be able to relearn the piano masters at a much more mature age....this has brought me immense peace and satisfaction....Brahms just feels so much more relevent when one is over 40 :-)

I have spent a lot of money and time trying out all the midi/piano roll apps and this is def up there as the best. If you need any insights on other platforms, i likeley have tried it.

Im also an experienced marketer for big business, so if you also have any business/commercial needs, please let me know.

Just dropping in.

Hi from New Zealand
Nicholas
Posts: 13170

Post by Nicholas »

Hi Benjamin. Welcome and thanks for the positive feedback!

I was just headed this way to make an update post so it's a nice coincidence that you dropped in here today, too. :)

Progress Update!

After many years of saying there are still many years to go on Synthesia 11, all of a sudden it is looking like there is a good chance that the entirety of beautifully-engraved MusicXML-based sheet music might only take a few more months of integration work instead of more years of from-scratch development.

I recently stumbled upon the LenMus project, created by Cecilio Salmerón over even more years than Synthesia. He has (incredibly graciously) open-sourced the sheet-rendering part of his project as a library called Lomse. It's in C++, the license is compatible with Synthesia, and the dependencies are (rather coincidentally) all things that Synthesia already has integrated into its codebase. It uses the same musical font that Synthesia has been using for several years and I've even seen half a dozen references to Behind Bars in the code, so it appears we've even been using the same reference book to make decisions.

Lomse's output for a given MusicXML file is already beautiful and from a couple experiments I've conducted and the conversations I've had with Cecilio so far, getting the library to fit as a drop-in replacement for Synthesia's own sheet music renderer sounds like a rather tractable problem.

I was just sitting down to begin the ~60 hours or so I had been planning to get variable/alternate clefs into the sheet music when I discovered all of this. Then, maybe 90 minutes later, I was staring at my clef test file being rendered perfectly through a quick, proof-of-concept harness I'd cobbled together. But it wasn't just clefs. It can do all of the things that are missing like beams, slurs, triplets, and everything else! :shock:

It's a large library that is used to handling everything (including playback and scrolling the music) internally, so it will take some effort to get things integrated such that Synthesia is only using the pieces it needs and everything lines up correctly... but man, if it works, this might save something like 2000 to 3000 hours of effort.

I think I'm going to hold off on the clef feature for now in 10.9 and ship it as-is... and there is a chance that 10.9 will be followed (as it should have been, logically) by Synthesia 11 (instead of one of those app versioning abominations like "10.10" or something) with the bulk of everything done at once!

I can't wait to watch this graph in the top post begin to crater as the integration proceeds! :lol: More than that, I can't wait to have the schedule freed up to work on all the other cool stuff people have been asking for in the meantime.
revilo2
Posts: 137

Post by revilo2 »

GREAT !!!!!
what about fingering in sheet music ?
Nicholas
Posts: 13170

Post by Nicholas »

I haven't tested it yet, but I know the library has extensive support for arbitrarily printing text (lyrics, tempo directions, TAB notation, tuplet beat counts, etc.) in the music. So, even if it's something that doesn't exist yet, it will be easier to add it there than to add it to Synthesia. :anxious:

The code is right on GitHub and the author seems to welcome outside contributions, so it's something I could help maintain, add features to, and fix bugs. (That's probably the most attractive part of the idea. If it were a closed library, I would be stuck with whichever bugs were in there and might have to tell users "sorry, I can't do anything about it". For such an important part of Synthesia, I would never put myself or its users in that position. But because there's ready access to fix things, it should work nicely.)
Post Reply