Synthesia 11

Try new versions before anyone else!
Always the latest dev version: [ win ] [ mac ] [ android beta opt-in ]
Please report comments and bugs!

Your data hasn't disappeared: development previews store their data in a different place. Details here.
Posts: 11911

Post by Nicholas » 12-21-18 7:04 am

Not yet. Synthesia 10.5 was just last month and there have been seven bug-fix updates to the Android version since then! (It's been a busy holiday season so far!)

The good news is that the very next Synthesia 11 dev preview is the one that will allow full-screen sheet music, so there shouldn't be too much longer of a wait.

Posts: 11

Post by spstarr » 12-24-18 8:30 pm

Looking forward to trying this out Nicholas, very much so, great work :D

Posts: 62

Post by Korados » 12-28-18 11:28 am

Nicholas wrote: If you're still getting clipping, you might even try reducing the volume on the Settings --> Built-in MIDI Synthesizer screen down below 100%. That may solve it.

So, a bit of nomenclature: both Synthesia and VirtualMIDISynth use BASSMIDI under the hood. In this case, "I guess Bassmidi doesn't preload..." is contradictory because "... like Coolsoft's Midi Synth" implies it does. ;)

What you mean to say is that Synthesia doesn't preload the fonts. And you would be correct. Until the next silent 10.5 update where it will preload fonts by default (but let you turn it off if you don't want to multiply Synthesia's RAM use by 10-200x when loading giant SoundFonts). :D
Oh, I wasn't aware of that. I only use the General User Soudfont, and it's only 35 MB big, so I guess preloading it wouldn't use so much RAM. I'm looking forward to the next update!
I wish you a happy new year!

Posts: 96

Post by revilo2 » 04-04-19 10:20 am

In the final version of Synthesia 11, the display of the score will be the same as a musescore's score ?

What about barlines ?

Posts: 11911

Post by Nicholas » 04-04-19 12:56 pm

The goal is to be very close to MuseScore's display. It won't be 100% identical, but I'll take 95% instead of what is probably closer to 20% today.

What did you mean about bar lines? How are they different?

Posts: 96

Post by revilo2 » 04-05-19 2:08 am

I'll will try to explain this in english (not easy for me) :

I was talking about the barlines of the score : "Start repeat" and "End repeat"

Musescore display the whole score and manage the barlines as a musician will do :

When the cursor is at the "End repeat" position (for the first time), it return the the 'Start repeat" position.

But Synthesia doesn't display the whole score.... :)

Posts: 11911

Post by Nicholas » 04-05-19 9:26 am

Repeated sections are on the list for Synthesia 11 with a MusicXML file that contains them.

For MIDI files it would be trickier because we'd need some algorithm for comparing measures and detecting repeats manually (because that information isn't stored in a MIDI file), so I'm holding off on those for now.

Posts: 11

Post by spstarr » 04-06-19 12:04 am

For the Sheet music mode, will we be able to show the letters of the notes on the staff? This looks great so far!

Posts: 11911

Post by Nicholas » 04-06-19 12:20 am

The musical font we're using includes note heads with labels, so it's a possibility. (You'd have to scale the music up pretty far to make any use out of them though.)

So far I've only got a note on my internal Synthesia 11 roadmap that they exist. It's more of a "stretch goal" at this point with how long Synthesia 11 development has already been ongoing and how much is still left. If it doesn't make it into 11.0, I'm hoping to clean up any of the missing features during a few small 11.x updates that will follow shortly afterward.

Posts: 96

Post by revilo2 » 04-09-19 7:32 am

and fingering ?

Posts: 11911

Post by Nicholas » 04-09-19 12:50 pm

That one will definitely be included. Those are just little numbers above/below the staff, and Synthesia is going to have to know how to draw a variety of text styles there already for other things, too.

Posts: 11911

Post by Nicholas » 05-30-19 5:10 pm

Progress Update!

I haven't checked-in in a while. Here's what's been going on these past two months:


Ever since 8.2 back in 2012 when Synthesia adopted most of iOS's menu conventions, I've been using the little check mark at the right side of lists both correctly and incorrectly. When you're choosing one of several options, the checkmark is the right answer:
goodCheckmarks.png (4.89 KiB) Viewed 1427 times
But when a check mark just means "on" or "off", that's confusing:
badCheckmarks.png (8.96 KiB) Viewed 1427 times
It's not clear at all that you can toggle those independently. Instead, confusingly, it looks like you're choosing between the two.

Having a proper "switch" control was long overdue. They make the above distinction immediately clear and remove 80% of the work of changing options on the Settings screen. Virtually all of the "drill down to a sub-menu to choose between two options" can be collapsed outright into a cute little switch:
switches.png (14.31 KiB) Viewed 1427 times
One caveat: the sub-menus had a little "(Default)" to indicate a default setting. With switches, that information is lost. On the Advanced settings screen, this might cause some trouble if someone just goes through and turns them all on (while poking around or whatever) and inadvertently causes something to misbehave. So, to make sure everything is covered, there is a new "Reset to Defaults" button at the bottom of both the Advanced and Gameplay tabs that will restore those things to their default, "safe" values.

Rendering Engine Work

The switches were actually an excuse to test some new technology that is being added to the back end by our new graphics developer contractor. (For anyone that does graphics and is curious: static vertex buffers that persist between frames instead of "dynamic everything all the time!") This is the technology that will eventually solve the very last of the major inefficiencies in Synthesia. Once the falling notes are all in static buffers that don't need to be sent to the graphics card each frame, the app should realize some big improvements in battery usage and smoothing out the last of the frame rate hiccups. The minimum hardware target should drop pretty substantially, too. Something with the performance of a Raspberry Pi might not be out of the question (porting to a new platform notwithstanding).

Synthesia Support Assistant

I don't like taking time away from working on the app, but the support workload has been steadily increasing for 12 years now and it was time for something better than the little troubleshooting drop-down that used to be on the support page. The new support assistant can do a few cool things right now (including completely automatically issuing an unlock refund), but the best part is that it's super flexible and I can trivially add new answers to the little Q&A tree; much easier than the old system. The assistant is also going to learn how to do a few more neat tricks like updating email addresses for older unlocks and a few other time-saving things to hopefully reduce the email workload even further.

This last one actually took the better part of last month, but it was a lot of fun. I got to play with WebSockets for the first time and the tool has a lot of room to grow. The ask/respond format there can handle just about anything in the future and a chat window is a very familiar interface these days. A single click/tap from a row of choices is a much friendlier UX interaction (especially on a touchscreen) than messing around with drop-down boxes.

The changes there actually helped make the whole support section of the site make a lot more sense. Instead of separate FAQ, support/troubleshooting, and guide list pages where everything felt scattered to the wind... everything is in one place now: the guides are listed next to the support assistant, which in turn can answer all the old FAQ questions. For things that were only masquerading as frequently-asked but were actually high-level project information, I made a new (also long-overdue) About page.

Ancient Website Migration

Since 2014 (around the time we added the Music Store), the website has actually been split between old stuff (ASP.NET WebForms) and new stuff (ASP.NET MVC). For the last couple years, something like 95% of everything had been migrated over. In a recent attempt to finish that migration, I thought it would be safe to finally switch off the ancient version of the "unlock via short code" process that hasn't been used since Synthesia 10.2, nearly five years ago... and got to learn firsthand how many people are apparently still using those older versions by the flood of email asking why unlocking doesn't work anymore.

In any event, I just spent a couple days migrating the old to the new in a more full-featured way so Synthesia 10.2 and earlier can be unlocked via short code once again and the WebForms site can finally be shut down. One less thing to support, keep track of, and maintain. Whew.

MIDI Input Latency Investigation

Since day one, MIDI input has been handled on the UI thread. This seemed reasonable at the time when it was a hobby project by a kid just a few years out of college. It's not reasonable anymore. Including the BASS-based synth already helped things dramatically: a "default" user that just downloaded the app for Windows saw latency drop from 240ms down to 80ms. That's a world of difference, but I can squeeze a little harder.

I put a trace in the code to measure how long it took Synthesia from seeing MIDI input from the OS until it was processed through the main (graphics) thread and sent back out to the OS as an output message. At 60 frames per second (16.6ms per frame), you'd expect the average to be about 8ms... and it was. Curiously, there were two peaks in the chart:
inputLatency.png (14.56 KiB) Viewed 1426 times
That's a tiny image, but the peaks are at 5ms and around 15ms. The more intense the test, the more that ends up landing in the 15ms range. Worse, a few sample spiked out into the 30ms and 40ms range! I think my least favorite part is how wide the base of that graph is: sometimes your notes are played in 2ms. Sometimes they take 19ms. That's a lot of jitter that is jarring at best. This is one of those things where even if you didn't notice it, subconsciously you were having a poorer experience.

These are all things that can be solved by driving the MIDI listening part of Synthesia in a more event-driven way where it's handled (and sent back out) immediately after it arrives. Once fixed, the graph should just be a single tiny spike below 1ms. That will be a lot of under the hood work (that is not planned for 10.6), but with any luck, eventually we should be able to drop the "default" latency that a user sees by another ~15ms. Every little bit counts. (Best of all: if you happen to use a real digital piano instead of a software synth, Synthesia's own 15ms or so is the majority of your wait, so fixing this should make a huge difference in those cases.)

All of that is a bit of a grab bag. Here's a rough list of things that will be done before the next 10.6 preview:
  • Three new "actions" for the support assistant (again, I don't like giving up app development time, but this actually increases velocity in the long run!)
  • Sprinkle the new UI switch controls in about 8 more places.
  • ~11 more improvements/fixes suggested here or via email. Most of them are very small. A few of them are medium (and should be pretty exciting).

Post Reply