Synthesia 11

Post by **Nicholas** » 03-13-16 9:45 pm

I can hardly remember a time before all I knew was making Android bug fix releases over and over, as each new hardware release and OS update break things again and again. It has been long enough! It is time to get this show on the road.

MusicXML! Better sheet music! These are the things Synthesia 11 will be made of!

This development effort segments much more nicely into quite a few development previews than the last few big releases have. So you guys will end up getting to see the new stuff on a pretty regular cadence. Here is the dev preview break-down you should expect:

Platform Update - Complete, released in 10.3! (Mar-2016)

It has been a long time. We used to do these yearly when it was a good opportunity to update all our tooling and libraries to the newest versions. This one is a biggie! It was finally time to cut Windows XP and Mac OS 10.6 compatibility so that we could finally get our hands on full C++11/14 (and libc++) support across all our platforms. It was a steep price, but modern C++ is practically a different language that brings enough developer convenience, safety, simplicity, and performance that it was (well!) worth the cost of backwards compatibility. At this point the code has been dramatically modernized: probably more change (going on for months now) than between any two consecutive builds. Everything will look exactly the same, but under the hood there has been a tremendous boost to all of the areas I just mentioned.

You may notice I've been speaking in the past-tense. This preview is just wrapping up and should be ready (and posted) in a few days.

Bug Fixes / Quality Pass - Complete, released in 10.3! (Sept-2016)

I've tried to keep the task list absolutely clear of anything that isn't sheet music related this time around, but the usual assortment of bugs and rough edges have cropped up. I've been tackling a few of these during the platform update but there are still 29 small items on the list to fix before pressing onward.

Off-screen Sheet Music Compositing - Complete, released in 10.4! (Nov-2017)

The sheer number of new symbols that Synthesia will be using while drawing its sheet music means it's probably best to migrate from a rasterized texture containing a limited set of glyphs to the way everyone else does it: a music font. There are a handful of nice (freely available) notation fonts that we'll get to evaluate. This also means new, anti-aliased drawing routines for beam/slur/tie primitives.

The upcoming complexity of the notation will finally tip the scales over to making it worthwhile to render it "offline" (in memory) once, and just draw it to screen as a single, big texture. (Compare against today's method where each stem, note head, dot, accidental, etc. are all sent to the graphics hardware separately each frame.)

Pros: Drawing speed no longer depends on the complexity of the song. Sheet music becomes essentially "free" performance-wise. (This is good for tablets!) And, because the sheet is already being drawn as an image, we might even be able to tinker with saving "pages" out to disk. Those experiments would only output a raster format like PNG, rather than a vector format like PDF, but it will still be interesting.

Cons: Smooth sheet music scaling will probably have to change to a fixed number of size steps. Instead of dragging the separator bar, there will be + and - buttons.

Multiple Lines of Notation - Complete, released in 10.6! (Mar-2019)

This is one of Synthesia's most-requested features.

Being able to turn off the falling notes and use full-screen sheet music means being able to do everything you can today with the falling notes (bookmarks, loops, etc.) right on the sheet music. This will be... interesting. Probably the biggest visual change to the play screen in Synthesia's history.

Portrait Mode

Multiple lines immediately brings to mind pages of traditional sheet music. Right now (for several UI reasons), Synthesia is restricted to landscape orientation on tablets. This preview will bring first-class UI support for portrait orientation.

Bare-bones MusicXML Parsing

Step #1 is to be able to load the files like we load MIDI today. Starting with typical MIDI-like note information. After this preview, the falling notes should behave exactly correctly for MusicXML files.

Fully-featured MusicXML-based Notation

Step #2 is using all the extra musical intent stored in MusicXML files to improve the sheet music dramatically. After this preview, the notation should appear essentially exactly how the arranger intended it.

This includes all the currently missing bits: 32nd notes, triplets, stem directions, ties, better beams, proper staff selection, and much more. (Much of this will be brought over to MIDI sheet music generation for free.)

Wrap-up, Clean-up, and a Couple Goodies

While you guys are providing feedback and bug reports for everything else, I have one or two fun bonuses for the sheet music. This will also be the time to start integrating/testing all the MusicXML versions of the built-in songs.

I've been talking about MusicXML support for a few years now. It's exciting to finally break ground and get this started!

kiwi · Post by **kiwi** » 03-14-16 11:36 pm

Usually i don't use the sheet feature but for sure many users 'll be happy!
It's really a major step and a ticket on the pro software like Sibélius,finale and so on...
I mean Synthesia can run very well side by side with them

Question: With the xml does it 'll be possible to have tabs on Synthesia?
You 'll only need 0 to 24 for the fonts.

I know it's not the goal of Synthesia but if it is not so much works it can be good to read tabs in fullscreen.

Post by **Nicholas** » 03-15-16 12:21 am

We'll see. This is already going to be a pretty huge effort, so I want to try to keep the scope as limited as possible. No promises this time! (Although after Synthesia 11, it might be an easy addition.)

kiwi · Post by **kiwi** » 03-15-16 9:27 am

Nicholas wrote:We'll see. This is already going to be a pretty huge effort, so I want to try to keep the scope as limited as possible. No promises this time! (Although after Synthesia 11, it might be an easy addition.)

Good!
Anyway honestly i just want the colours on Synthésia that's all cause the soft is already perfect for me

silvrica · Post by **silvrica** » 03-15-16 5:21 pm

I have a question about the new Musicxml score in Synthesia:

How is going to be played by Synthesia 11 the start and end repeat bars of a score? As the midi do not follow the score format, are we going to be able to see in the score the blue guide returning to the start repeat bar as many times as indicated in the score? I hope I am clear.

Post by **Nicholas** » 03-15-16 11:12 pm

I suspect falling bars are going to continue working the way they usually do. (A duplicate set of falling notes will stream down as per usual.)

We'll see about the sheet music. I'd like to have them show correctly with repeat markings. It does get a bit tricky with the page-flip animation to make sure the next notes are correctly visible. Especially if the repeat is near the beginning of the line. I'll have to think about how that might work.

meirppp · Post by **meirppp** » 03-16-16 11:16 am

Would we be able to change the falling notes colors as well?

Thanks
Meir

Post by **Nicholas** » 03-16-16 3:03 pm

No. Only sheet music features this time around!

(Arbitrary falling note colors will still take a fair amount of engine and UI work. More than I can sacrifice for a release that is already so large. Sorry!)

draco2023a · Post by **draco2023a** » 03-18-16 10:17 pm

These are some really great changes coming up and we know it's a lot of work. We appreciate the effort.

That being said...do you have a guess on what kind of timeline we are looking at before we see the notation and XML improvements?

Post by **Nicholas** » 03-18-16 10:56 pm

If I'm normally bad at estimating these kind of things, there is an even bigger complication this time! My wife and I are actually expecting our first child sometime in the next couple weeks... so work schedules are going to get a bit hectic.

There are a few other people that help with Synthesia these days, but I still do the majority of the client work myself. So we'll see! I am sorry to report I don't have any estimate at all.

kiwi · Post by **kiwi** » 03-19-16 2:53 am

Maybe a future Synthesia user 'll born who knows :p

How did you call it? Amadeus?

MarkPas · Post by **MarkPas** » 03-21-16 7:05 am

Pretty techie discussion. To cut to the chase... will Synthesia 11 allow note to hand assignments in sheet music view and is there a tentative release date? Synthesia has the potential to be a premiere tool for learning music from sheet music.
Keep up the awesome work!

draco2023a · Post by **draco2023a** » 03-21-16 7:06 pm

Oh CONGRATS Nicholas!!!!! Take your time....It already is a great product...We can wait!

MarkPas · Post by **MarkPas** » 03-22-16 7:46 am

So I did a little more experimentation. Using Musescore 2.0 I created a score, and assigned different instruments to the right and left hand parts. When I brought the midi into Synthesia it was good to see that the hand parts were separated correctly... progress. Unfortunately, some notes appeared on the wrong staff. For example, a middle C from the left hand part shows in note view was moved to the treble clef for practicing the left hand only. I guess this would work okay if I was going to memorize the piece. But since I just wanted to practice/learn the piece it's awkward to learn the hands separately with one notation and then try to play it with another notation. I'm unsure if I'm simply trying to use Synthesia in a manner it wasn't intended for or if there's something I'm missing.

WalkerBoh · Post by **WalkerBoh** » 03-22-16 8:23 am

This is a ROADMAP post. The sheet music improvements have not yet been implemented. Please be patient and check for more recent development updates.

jimhenry · Post by **jimhenry** » 03-22-16 10:35 am

Yes, eventually (I am guessing toward the end of 2016) you will be able to practice in Synthesia with sheet music that looks almost exactly like what MuseScore produces for sheets by exporting a MusicXML file.

matevz81 · Post by **matevz81** » 05-10-16 2:41 am

Any update on progress?

Post by **Nicholas** » 05-10-16 9:44 am

matevz81 wrote:Any update on progress?

Yes, quite a bit.

Post by **Nicholas** » 05-14-16 3:51 pm

Here you go. Progress!

revilo2 · Post by **revilo2** » 06-09-16 2:02 am

Hello,

Is the figering on sheet music sheduled on step 3 ?

Thanks