Progress Update!
I haven't checked-in in a while. Here's what's been going on during April and May 2019:
Switches
Ever since 8.2 back in 2012 when Synthesia adopted most of iOS's menu conventions, I've been using the little check mark at the right side of lists both correctly and incorrectly. When you're choosing one of several options, the checkmark is the right answer:
- goodCheckmarks.png (4.89 KiB) Viewed 102877 times
But when a check mark just means "on" or "off", that's confusing:
- badCheckmarks.png (8.96 KiB) Viewed 102877 times
It's not clear at all that you can toggle those independently. Instead, confusingly, it looks like you're choosing between the two.
Having a proper "switch" control was long overdue. They make the above distinction immediately clear and remove 80% of the work of changing options on the Settings screen. Virtually all of the "drill down to a sub-menu to choose between two options" can be collapsed outright into a cute little switch:
- switches.png (14.31 KiB) Viewed 102877 times
One caveat: the sub-menus had a little "(Default)" to indicate a default setting. With switches, that information is lost. On the Advanced settings screen, this might cause some trouble if someone just goes through and turns them all on (while poking around or whatever) and inadvertently causes something to misbehave. So, to make sure everything is covered, there is a new "Reset to Defaults" button at the bottom of both the Advanced and Gameplay tabs that will restore those things to their default, "safe" values.
Rendering Engine Work
The switches were actually an excuse to test some new technology that is being added to the back end by our new graphics developer contractor. (For anyone that does graphics and is curious: static vertex buffers that persist between frames instead of "dynamic everything all the time!") This is the technology that will eventually solve the very last of the major inefficiencies in Synthesia. Once the falling notes are all in static buffers that don't need to be sent to the graphics card each frame, the app should realize some big improvements in battery usage and smoothing out the last of the frame rate hiccups. The minimum hardware target should drop pretty substantially, too. Something with the performance of a Raspberry Pi might not be out of the question (porting to a new platform notwithstanding).
Synthesia Support Assistant
I don't like taking time away from working on the app, but the support workload has been steadily increasing for 12 years now and it was time for something better than the little troubleshooting drop-down that used to be on the support page. The new
support assistant can do a few cool things right now (including completely automatically issuing an unlock refund), but the best part is that it's super flexible and I can trivially add new answers to the little Q&A tree; much easier than the old system. The assistant is also going to learn how to do a few more neat tricks like updating email addresses for older unlocks and a few other time-saving things to hopefully reduce the email workload even further.
This last one actually took the better part of last month, but it was a lot of fun. I got to play with
WebSockets for the first time and the tool has a lot of room to grow. The ask/respond format there can handle just about anything in the future and a chat window is a very familiar interface these days. A single click/tap from a row of choices is a much friendlier UX interaction (especially on a touchscreen) than messing around with drop-down boxes.
The changes there actually helped make the whole support section of the site make a lot more sense. Instead of separate FAQ, support/troubleshooting, and guide list pages where everything felt scattered to the wind... everything is in one place now: the guides are listed next to the support assistant, which in turn can answer all the old FAQ questions. For things that were only masquerading as frequently-asked but were actually high-level project information, I made a new (also long-overdue)
About page.
Ancient Website Migration
Since 2014 (around the time we added the Music Store), the website has actually been split between old stuff (ASP.NET WebForms) and new stuff (ASP.NET MVC). For the last couple years, something like 95% of everything had been migrated over. In a recent attempt to finish that migration, I thought it would be safe to finally switch off the ancient version of the "unlock via short code" process that hasn't been used since Synthesia 10.2, nearly five years ago... and got to learn firsthand how many people are apparently still using those older versions by the flood of email asking why unlocking doesn't work anymore.
In any event, I just spent a couple days migrating the old to the new in a more full-featured way so Synthesia 10.2 and earlier can be unlocked via short code once again
and the WebForms site can finally be shut down. One less thing to support, keep track of, and maintain. Whew.
MIDI Input Latency Investigation
Since day one, MIDI input has been handled on the UI thread. This seemed reasonable at the time when it was a hobby project by a kid just a few years out of college. It's not reasonable anymore. Including the BASS-based synth already
helped things dramatically: a "default" user that just downloaded the app for Windows saw latency drop from 240ms down to 80ms. That's a world of difference, but I can squeeze a little harder.
I put a trace in the code to measure how long it took Synthesia from
seeing MIDI input from the OS until it was processed through the main (graphics) thread and
sent back out to the OS as an output message. At 60 frames per second (16.6ms per frame), you'd expect the average to be about 8ms... and it was. Curiously, there were two peaks in the chart:
- inputLatency.png (14.56 KiB) Viewed 102876 times
That's a tiny image, but the peaks are at 5ms and around 15ms. The more intense the test, the more that ends up landing in the 15ms range. Worse, a few samples spiked out into the 30ms and 40ms range! I think my least favorite part is how wide the base of that graph is: sometimes your notes are played in 2ms. Sometimes they take 19ms. That's a lot of jitter that is jarring at best. This is one of those things where even if you didn't notice it, subconsciously you were having a poorer experience.
These are all things that can be solved by driving the MIDI listening part of Synthesia in a more event-driven way where it's handled (and sent back out) immediately after it arrives. Once fixed, the graph should just be a single tiny spike below 1ms. That will be a lot of under the hood work (that is
not planned for 10.6), but with any luck, eventually we should be able to drop the "default" latency that a user sees by another ~15ms. Every little bit counts. (Best of all: if you happen to use a real digital piano instead of a software synth, Synthesia's own 15ms or so is the majority of your wait, so fixing this should make a huge difference in those cases.)
All of that is a bit of a grab bag. Here's a rough list of things that will be done before the next 10.6 preview:
- Three new "actions" for the support assistant (again, I don't like giving up app development time, but this actually increases velocity in the long run!)
- Sprinkle the new UI switch controls in about 8 more places.
- ~11 more improvements/fixes suggested here or via email. Most of them are very small. A few of them are medium (and should be pretty exciting).