Modding Wishlist

[Nicholas]

So what I'm suggesting is that Synthesia could be a core platform for experimenters who want to tinker with keyboard designs!

This is a very cool direction, though it's such a big idea that I'm not sure I'd include it in Synthesia directly. An experimentation platform almost sounds like a different app altogether built using the same underlying technology.

Like kiwi alluded to, these are definitely ideas that I'm open to -- and would even like to tackle one day -- but finishing the work on our core competency ("a fun and engaging piano practice tool") has to take priority. Unfortunately, the remaining work is not a short list.

My hunch is that playing short sequences in random order would result in much accelerated learning...

This type of experiment does fall inside the core competency, which means it's the kind of thing I could see adding much sooner.

... consider synching several Synthesia tablets to the same "conductor"...

This is another thing that would be absolutely amazing to witness in a room full of users, but I fear it would be a third app on top of Synthesia and the instrument experimentation platform.

[kiwi]

My hunch is that playing short sequences in random order would result in much accelerated learning...

This type of experiment does fall inside the core competency, which means it's the kind of thing I could see adding much sooner.

Random is not the good direction cycle of 4th or 5th or jump to one degree in a scale (it's call substitution) and why noàt a random button but
It's a little like my feature request:
viewtopic.php?f=5&t=6727

Hability to wrote a grid (have the chords displayed on an interactive keyboard).
Or choose a tonality and some random cadenzia (I V VI II V per eg) and the goal is too play the chords both hand and séparate.

If i am not very clear let me know.

This is really important for to improvise on piano

[godspace]

Hello, here some ideas of piano midi gameplay :
https://youtu.be/fnSVZDKXKrs
https://youtu.be/uDi6fSakJiM
Thanks!

[Nicholas]

Those are a lot of fun. Thanks for sharing those videos.

I'm actually pretty excited about a smaller side project coming up (after Synthesia 12 is finished) where we'll get to explore some ideas like these.

[KaleidonKep99]

Can you add support for external textures for both piano keyboard and notes? That would be awesome!

Also, I made a preview of what I am actually talking about (I sent you an e-mail about it, no answer tho):


As I said, this would be an awesome feature!

[Chompy]

More colours in notes/piano (arbitrary colour support), would be amazing for me too!

I wonder if you can add much more colour support as soon as possible in Synthesia 11! 100% awesome!

Also about Synthesia 11, is there a testing preview out there for me to explore the new version (Mac/Windows) OS?

This is intended for Synthesia 11, but I've been seeking to find out a dev preview for Synthesia 11 to test using Wine, since it's not for Snow Leopard Macs anymore. (Synthesia 11)

By "Synthesia 11" I mean "Synthesia 10.3"!

Best regards - Zac

[Nicholas]

Can you add support for external textures...

It's probably going to be a resource-editing thing for the foreseeable future. It wouldn't be too hard to add, but there are lots and lots of things that are higher priority right now. Sorry!

Also, I made a preview of what I am actually talking about...

Those keyboard keys aren't half bad. (I never ran the version you sent; I try to avoid running modified executables from sketchy sources, hehe. )

I wonder if you can add much more colour support as soon as possible in Synthesia 11! 100% awesome!

You're not going to make me link to the same post saying "Not in 11" for a third time now, are you?

Also about Synthesia 11, is there a testing preview out there for me to explore the new version...

Not yet. The video creator was just wrapping up when I discovered a new curve ball thrown by Apple. That switch-over is taking a little longer. After that we'll start pushing out previews of Synthesia 11. Keep an eye on the Development Updates section of the forum.

[Dendrophobe]

Is there any way to get a dump of statistics for a song? The thing I like most about Synthesia is the ability to see how well I'm doing in terms of score, but it'd be awesome if I could get some more fine grained data. Stuff like longest combo, how close I am to holding all the notes correctly, etc, would be a great start. Building on that, it'd be neat if I could build a chart of where errors typically occur in a piece to figure out where I should be practicing next (histograms of each practice session, with errors on the y axis and measure numbers on the x would be pretty helpful). Just being able to get a CSV of performance data for each hand would be useful.

Anyway, I'm thrilled with the software, so keep up the great work!

[Nicholas]

The best you can get for statistics is the table on the song setup screen (on the "Points Earned" tab). Double-click anywhere in that chart to copy the whole thing to your clipboard in a format you can paste right into something like Excel. (I think it's technically TSV.)

Regarding a chart where errors usually occur, that is something we're hoping to add to Synthesia 12: a replay feature where your mistakes are overlaid on the song's notes. Also, showing measures in the timeline that have mistakes.

[KyleF217]

Hi, this might already exist but I sure haven't found it, I think it would be great to have our own folders within Synthesia. They already have the "Easy" "Medium" etc folders, so it should be possible to make our own folders right?

[Nicholas]

You can organize things into folders (the same way Synthesia does for the Easy, Medium, etc. songs) using the Metadata Editor.

[DrTechDaddy]

Re Alternate Keyboards Layouts per my post of 09-08-14:
Are we any closer to being able to create the "experimental platform" to explore alternative keyboard layouts?

I do see movement on key coloring.

A simple color scheme I'm fond of (which can be done on real keyboards with markers or masking tape) is to color the F and B keys gray--this divides the keyboard into two complementary pentatonic keyboards--one white, one black--with the gray keys adding the diatonic scale notes to both.
Given that Synthesia is played from piano-roll vs. traditional notation, it's no harder to "read" the key of F#/Gb than the key of C,
so this mod would make it possible to learn to play songs in both the "key of white" and the "key of black",
and easily transpose a song from one to the other to fit a voice range. Songs traditionally scored in F or G could be easily played in F#/Gb instead.

A first step toward an alternate keyboard would be an alternate coloring scheme for the keys, e.g. alternating black-white vs the 7-5 pattern. Then, just cover the bottom part of the (formerly) white keys, and you have a keyboard of 12 skinny keys in alternate colors!

The next step would be to change she "shape" of the keys.

Another idea, to go with key color change, is change the mapping between the computer keyboard and the piano keyboard, so I could implement a 6-6 pattern on the computer keyboard if not on the actual piano visual:
Example: the chromatic scale from C could be: a W s E d R f T g Y h U j I k O l P ;
where I have capitalized the computer keys from the upper row to highlight the row distinction.
this adds the computer keys "r" and "i" and moves notes F G A B to the "upper" keys.
One gets slightly more range by starting with note "C" on the upper row with letter key "W" and leaving notes F G A B down on letters F G H J.

Activating the upper and lower rows of keys would provide additional fingering options.

I see it's possible to change the background image. How hard would it be to overlay the keyboard,
or shrink it's vertical dimension so I just see a row of "twinkling lights" as the keys are pressed?
I would imagine the problem area would be adjusting the size and number of keys.

[jsbftw]

Can you make it output the right note (pitch value) even if the played note is incorrect?

Stay with me here.

Playing music is about expression via timing and loudness.

What's the point of learning to play the right notes?

Sure I'll never generalize to a non-cheat-equipped instrument, but I'm ok with that.

I guess this involves sending messages back to the midi controller, which isn't otherwise done in Synthesia.

If no hope from Synthesia please tell me so I can get started doing it myself.

thanks!

[Nicholas]

Actually, we've discussed a "Rhythm-only" type of practice mode where the emphasis is on getting the timing right, regardless of pitch. It sounds like that might match up with what you're asking about here. (Granted, it's more of a wish-list idea at the moment. It's not scheduled for any near-term updates.)

[jsbftw]

Yeah i see what you mean... the prescriptive attitude bums me out but thats not your fault. Pretty please make it happen.

Since rhythm can't be fixed automatically in the simple way pitch can, the melody practice mode ignores rhythmic errors, i.e., waits for you to play.

But in the proposed rhythm-practice mode, errors can actually be fixed, resulting in good-sounding music played through, which is satisfying, so more time at the keyboard, etc.

[jsbftw]

How about allowing acoustic instruments, using mic input?

I'm aware its not computationally trivial, but for keyboard music this is very do-able.

[Nicholas]

For monophonic (single note at a time) input, it's pretty easy these days. Generalized polyphonic pitch detection is still being actively researched and the state of the art is still pretty mediocre.

That said, Synthesia doesn't need the fully general case, at least not for practicing against a known song. (It would need to be more general for Free Play mode.) When you're in the context of a known song, the algorithm can really just compare the input sound against the pitches that are expected at the current point in the song. Then, instead of output being a list of pitches being played, it can be something simpler like a confidence percentage of "how likely is it that the sound coming in from the mic matches the current set of expected pitches?"

It'd be a neat project, and when I get a chance (read: sometime after Synthesia 11) I'd like to try my hand at it.

[jsbftw]

The general case is hard, but a keyboard instrument is super constrained and so makes for a much easier problem.
Ive seen people do near-perfect automated scoring of polyphonic music by leveraging that... for example by using analysis-by-synthesis methods that exploit knowledge of the sound source.
Also scoring is a harder problem than you need to solve (template comparison). Still want a bayesian framework though.
Anyway just saying it might be an easier problem given the particulars of Synthesia.

[Nicholas]

I was even thinking Synthesia might be able to afford a luxury that most of the more general solutions can't: a calibration step. It would force users to only use a single instrument on their digital pianos, but if we could capture a frequency spectrum for each note (on the Settings screen, during initial setup), in theory it shouldn't take much more than running the input (during play) through the SVD and reading off pitches as linear combinations of the singular values. (Calibration data also solves the problem of poorly tuned pianos, etc. by not assuming an absolute pitch.)

I haven't tried it yet, so I don't know how well it would work in practice. My gut instinct is that the real-world version of the problem will end up more challenging than that, but I can't imagine that having some up-front calibration data could hurt.

Using the same a priori "upcoming notes in the song" knowledge combined with the calibration frequency histograms, you could generate a sort of expected "when they play all of the upcoming notes, the spectrogram should look like this" and just do an auto-correlation until the app finds the thing it's expecting. We'll see. Again, it's an interesting topic and I'm excited to make an attempt.