Key to color mapping?

Synthesia is a living project. You can help by sharing your ideas.
Search the forum before posting your idea. :D

No explicit, hateful, or hurtful language. Nothing illegal.
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

Can Synthesia get this colors per key or pitchclass, optionally which you might be able to toggle whenever you want, channel based coloring vs. key based coloring:
Image
http://extras.humdrum.org/man/mkeyscape/
Nicholas
Posts: 13132

Post by Nicholas »

There is some legal trickiness involved, but this is a direction I want to try once we've got arbitrary note colors. Synthesia 8.4 was the first step in that direction (runtime texture composition), but there is still a little more work involved.
kiwi
Synthesia Donor
Posts: 1180

Post by kiwi »

I like this palette :)
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

Nicholas, with current Synthesia version, is it possible to set freely the midi channel note colors? If yes, a preprocessing can bring us this feature, which would be fantastic. What we need only is defining for each 16 midi channels ourselves the colors.
Nicholas
Posts: 13132

Post by Nicholas »

Not yet. There quite a few features waiting on that piece of technology at this point. All of the old, legacy reasons for baking the colors right into the key textures are gone now.

The rest of this is technical graphics programming talk, so feel free to ignore it. ;)


Per-vertex color has been cheap for a couple decades. It already was when I first started the project.

But it's not just a matter of adding a white note graphic and using vertex colors to alter it. While that would preserve the shadows just fine, the highlights would get completely washed away by the color multiply. Instead, the right answer is to color the note graphic and then slap a 2nd texture highlight on top of it. This is how the buttons are drawn in the interface: with three layers: base, shadow, and highlight.

But! The UI has a fixed number of buttons, so multiple layers is fine. There will always be a fixed amount of drawing going on.

In a song, there could be literally thousands of notes on the screen at once. At that point, every bit of drawing you can save, the better. Whether you're running on old hardware, a limited tablet, or you're just playing a super intense song, there are lots of reasons to keep the falling note drawing as lightweight as possible.

The two alternatives I'm exploring:
  1. Allow dynamic colors but composite static textures once at the start of a song and draw using them the way we do today.
  2. Use pixel shaders to perform the dynamic texture composition on the fly.
Both methods have their pros and cons. #1 takes more memory and can get dicey if you've got 100 tracks and ask for each one to be a different color. #2 takes a little more GPU muscle that I'm not sure I'm willing to give up on the really low end platforms like some of these Android tablets.

... I suppose the funny part is I could just sacrifice the subtle white highlight and all of this would be moot. We could have colored notes immediately without any extra work. :lol:
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

Not colors per track, but ONLY per midi channel. So maximum number of colors will be always 16, never 100. I know you use per track coloring, but from midi software assingning point of view, changing midi channels is the only powerful way to use, and NOT being limited while editing, no matter which other software we switch to. From these experiences, I do all my 'decomposition experiments' simply by splitting or decomposing into the 16 midi channels. Then, for example, even using a simplistic software like Van Basco Karaoke Player, you can still, quickly and easily mute any of those decomposed layers. No need to switch into a special powerful software, even such simplest softwares can do this 'filtering work'. So using those 16 available midi channels in various forms is not a bad idea. Here it would be for 'color decomposition', using a special midi channel assigning preprocessor.
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

Here the color rgb values from above mkeyscape, help text, for reference:

Code: Select all

 !! mkeyscape default key colors based on 12
 !! tones per octave.  For example, C-sharp
 !! is represented by the same color as D-sharp.
 **index	**pixel	**comment
 !! Minor Keys
 0	0 255 0	C major
 1	18 237 73	C-sharp major
 2	63 95 255	D major
 3	218 9 73	E-flat major
 4	255 0 0	E major
 5	255 255 0	F major
 6	182 255 0	F-sharp major
 7	63 191 255	G major
 8	109 50 255	A-flat major
 9	127 31 255	A major
 10	255 132 0	B-flat major
 11	255 127 0	B major
 !! Minor Keys
 12	0 191 0	C minor
 13	14 178 55	C-sharp minor
 14	47 71 191	D minor
 15	164 7 55	E-flat minor
 16	191 0 0	E minor
 17	191 191 0	F minor
 18	137 191 0	F-sharp minor
 19	47 143 191	G minor
 20	47 92 191	G-sharp minor
 21	95 23 191	A minor
 22	191 99 0	B-flat minor
 23	191 95 0	B minor
 !! Other Colors
 24	0 0 0	silence
 25	255 255 255	background
 *-	*-	*-
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

It would be great if we could assign those 16 rgb colors for the 16 midi channels, via command line, too. For me it would be even enough, if only the below keys would show those colors, not the falling notes themselves. When watching those keys moving, we could see quickly the harmonies or note mixtures easily by color only, no need to think, our brain would adjust to the meaning or sounding of each color over time. Every specific chord would have its color combination and its sound attached to it.
Nicholas
Posts: 13132

Post by Nicholas »

Hey, how do you guys feel about completely flat notes? If Synthesia did its drawing with that style, no technology changes would be required and we could have arbitrary note colors immediately. (As an added bonus, flat seems to be en vogue right now since Windows 8 and iOS 7 have both moved to flat elements.)

A long time ago, I was super adamant about having the highlights and bevels. But now, for whatever reason, this mock-up looks really clean to me.
flatNotes.png
flatNotes.png (65.79 KiB) Viewed 18870 times
kiwi
Synthesia Donor
Posts: 1180

Post by kiwi »

Well Nicholas you already now my answer about this: I love Flat has i requested it years ago ^^
I just wondering why these flat notes 'll make the colors more easy to add only because of the layer?

By the way i have mixed the castel palette with the first one of this topic and i find it better less confusing and the diablo note is in blue ^^
Attachments
Palette perso.png
Palette perso.png (6.18 KiB) Viewed 18861 times
Nicholas
Posts: 13132

Post by Nicholas »

kiwi wrote:I just wondering why these flat notes 'll make the colors more easy to add only because of the layer?
Yes. The very short version of all the technical jargon a few posts above is:

The current notes have a white highlight drawn on top of the color. To show this correctly, you have to draw the note more than once. You can do it way ahead of time (in the graphics files), just ahead of time (creating graphics dynamically at song start), or each time you draw a note (2x the drawing during play or using pixel shaders).

#1 is how it's done today. You don't get any color flexibility but notes are fast to render.
#2 is the required technology work I keep talking about. You get color flexibility and it's still fast to render.
#3 is pretty easy, but double-drawing would kill performance on older hardware and tablets.

By switching to flat notes... the highlight goes away. A note is always a single layer and there are no extra steps required to avoid drawing notes more than once. You just draw the note using whatever color tint you want and you're done.

(As an added bonus, the size of the installer would drop by 140 KB or so without all the old note graphics.) :lol:
kiwi
Synthesia Donor
Posts: 1180

Post by kiwi »

Thanks Nicholas i understand better :)
What 140 kb! Well why did you wait for to add the flat style ^^ (and the colors) the incoming build is promising!
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

Yes, flat is perfect. Simplification is always good, and here it would be on the right place. Quickness is important. Synthesia would come a little closer to Atari ST, Notator timing. No unnecessary cpu distractions. We can set then colors per midi channel? If yes, via command line, too? That would be perfect.
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

Kiwi's color modification I do not like, it has to be as at the top, for me. ;) So, it would be perfect if Synthesia allows this freedom.
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

Nicholas, is it possible making those 'note number fonts' freely selectable? I would select some Atari fonts, if we had this freedom.
Nicholas
Posts: 13132

Post by Nicholas »

For the time being, there are some limitations on the font subsystem in Synthesia, too. So Atari fonts will have to wait. :D
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

No problem, I can have those fonts inside emacs, that's enough for now. :)
kiwi
Synthesia Donor
Posts: 1180

Post by kiwi »

TonE wrote:Kiwi's color modification I do not like, it has to be as at the top, for me. ;) So, it would be perfect if Synthesia allows this freedom.
What has to be at the top?

You mean this fonts?(i really don't like them but i understand you like them by nostalgie.
Image

By the way Nicholas i really like how are the new vertical lines now ;)
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

Well, yes, I meant those fonts. While reading various texts I am switching into those fonts inside emacs, instant retro feeling. I increase the font size, too, for 3 paragraphs per screen, vertically, for easier far distant reading. Or letting it read to me, automatically, via TTS. World of automation. You can increase your reading amounts, in front of the computer, without destroying your eyes. Instead, thinking, taking notes with paper and pen. I like this method.
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

kiwi wrote:What has to be at the top?
The colors, as posted at the top, not as you posted later, for me.
Post Reply