LinnStrument Light Guide Support via Web App

Fannon · Post by **Fannon** » 04-12-23 2:12 pm

Recently got a LinnStrument and wanted to try if it works well with Synthesia. There's no official support for it - which I can understand as it's a very niche instrument.

So I took some inspiration from the Komplete Kontrol script here: viewtopic.php?t=9220

The resulting project can be found here: https://github.com/Fannon/linnstrument- ... de-support

This is how it currently looks like:

I've started as a CLI tool but then decided to make it a web app, so users only need to have a modern browser to use it. It also allowed me to add LinnStrument / Grid Controller specific visualizations - as the layout is different than the Piano shown in Synthesia.

Now, I also wanted to have a bit more feedback / statistic on how I played the notes against the guide notes, so I've added some functionality to compare this. Here I ran into a problem, not sure if anyone here knows this: It seems that if I play the note very shortly before the Light Guide would appear, the Light Guide signal is not sent at all. I understand that this might not be necessary for giving visual feedback, but for my app it's really unfortunate because then I don't have the Light Guide signal to compare my played note to.

Post by **Nicholas** » 04-13-23 12:15 pm

This is very cool. I don't know much about the LinnStrument, but I'm always happy to see these sorts of integrations being tackled by the community!

Fannon wrote: ↑04-12-23 2:12 pmIt seems that if I play the note very shortly before the Light Guide would appear, the Light Guide signal is not sent at all.

This is one of those things that might be tricky to work around. There are two things happening in tandem here: Synthesia sends light messages whenever it encounters a note that is meant to be played by the user but hasn't been played yet. Separately, there is a small time window before and after the onset of a note when Synthesia will begin looking for the user to play a note. Taken together, you find the behavior you're running into: playing the note slightly ahead of time will mark the user-assigned note as having been played. So when Synthesia finally reaches that time point when the note should be played, it doesn't send out a light message because it already has been.

I suppose the alternative would be to always send the light messages, but this starts to behave a little strangely in certain circumstances. Imagine playing a song in Melody Practice mode while it is stopped at a chord. If you're using Settings --> Gameplay --> Continue When --> Each Note Has Been Pressed, you will be able to press each note separately, and the key lights will disappear one after the next until you've hit them all (and the song continues). If Synthesia always showed light messages for every "current" user note--whether they've been pressed or not--you wouldn't be able to tell which had been struck yet.

I'm not sure how to reconcile the two behaviors...

Fannon · Post by **Fannon** » 04-13-23 1:08 pm

Nicholas wrote: ↑04-13-23 12:15 pm This is very cool. I don't know much about the LinnStrument, but I'm always happy to see these sorts of integrations being tackled by the community!

Thanks! Synthesia is a nice target for this

Nicholas wrote: ↑04-13-23 12:15 pm I suppose the alternative would be to always send the light messages, but this starts to behave a little strangely in certain circumstances. Imagine playing a song in Melody Practice mode while it is stopped at a chord. If you're using Settings --> Gameplay --> Continue When --> Each Note Has Been Pressed, you will be able to press each note separately, and the key lights will disappear one after the next until you've hit them all (and the song continues). If Synthesia always showed light messages for every "current" user note--whether they've been pressed or not--you wouldn't be able to tell which had been struck yet.

I'm not sure how to reconcile the two behaviors...

Thanks for confirming, this was also my "feeling", but it's really not easy to replicate the situations

Would it be feasible to have a special Light Guide mode where Synthesia sends out light messages just as they're currently (to be) played in Synthesia? In that case, you would also send the light message for the notes the user is playing or just has played. For some use cases, that wouldn't be really an issue as your device / your receiving tool already knows which note you have played and can reconcile it. E.g. by preferring the user play indication over the guide indication. In my tool, I always show both at the same time - with some transparency.

Just an idea, though

Fannon · Post by **Fannon** » 04-13-23 1:10 pm

One quick question: I played around with "ONE Smart Piano" Light Guide mode. Could it be that there the light notes are always sent out the way I described in the previous post? I've added support for that as well (uses Aftertouch instead of note on/off and has an offset of -21?).

If one of the properitary modes already works the way I need it, I could just support that one.

Timmy Put · Post by **Timmy Put** » 04-15-23 9:49 pm

Seeing Fannon’s work is what made me find the iPad
App. It would be amazing to have a Linnstrument Grid view as an addition to score. I’ve been sending midi to another iOS app called Beatpads to achieve this view, so nice. Of course if it was all in one it would be totally way rad.

As a side wild idea a 3d Linnstrument view with rising notes with size correspondeding to note length as if the piano cascade was falling upwards.

Thanks for great work and inspiration

Post by **Nicholas** » 04-17-23 1:32 pm

Fannon wrote: ↑04-13-23 1:08 pmWould it be feasible to have a special Light Guide mode where Synthesia sends out light messages just as they're currently (to be) played in Synthesia? In that case, you would also send the light message for the notes the user is playing or just has played.

I could see a much richer information feed that combined all of the above and would send messages out for each kind of event. Things like "user correctly played note X", "note Y begins now", "note Z ended and was not played in time". It would definitely be chattier than anything in there today, but it would be able to handle all the use cases. Combine that with some beginning-of-song information like the list of colors picked for each part (or finger) and you'd really have everything you needed.

Fannon wrote: ↑04-13-23 1:10 pmCould it be that there the light notes are always sent out the way I described in the previous post?

I believe it's the same for all of the light modes, unfortunately. The different modes will send out different kinds of MIDI messages, but what triggers them in the first place is uniform across all the modes.

Fannon · Post by **Fannon** » 04-17-23 2:07 pm

Nicholas wrote: ↑04-17-23 1:32 pm I could see a much richer information feed that combined all of the above and would send messages out for each kind of event. Things like "user correctly played note X", "note Y begins now", "note Z ended and was not played in time". It would definitely be chattier than anything in there today, but it would be able to handle all the use cases. Combine that with some beginning-of-song information like the list of colors picked for each part (or finger) and you'd really have everything you needed.

Oh wow, that would be much better than what I originally asked for

I could see that opening up lots of new ideas and tools around Synthesia.

Nicholas wrote: ↑04-17-23 1:32 pm I believe it's the same for all of the light modes, unfortunately. The different modes will send out different kinds of MIDI messages, but what triggers them in the first place is uniform across all the modes.

Ok, thanks for the insight. Then this is confirmed and I know what I get