How do we get effects like from this image into Synthesia???

[Nikorasu95]

This is freaking beautiful and I want to know how the guy in this video went about making Synthesia have particle effects.I know he is using a led strip on his keyboard but how the freak did he do the particle effects? I made a comment but the guy never responded.
Check out his YouTube channel https://www.youtube.com/channel/UCs7O9sOUQiBGBxaaAguIwig/videos I've never seen something so amazing involving piano and software!

[Nicholas]

It's almost certainly an effect created in some video editing software like Adobe After Effects (most-likely with a rather specific, possibly custom plugin for the task).

That said, it's definitely a task that could be done on a graphics card in real-time. And you're right that it is rather striking. It would be fun to try and enhance Synthesia's own spark effect to get somewhere closer to this.

[Nikorasu95]

In the YouTube description of the guys video he said "In this video I have combined my live piano playing with the visualization of the music material by programming my own "Synthesia"-like software"
I asked him about open sourcing or selling his software so users can download but I noticed he doesn't respond to anybody which is really really rude! It would be nice to have particle effects like this in Synthesia along with more color choices for falling notes. Can you make this happen please? This is beyond awesome!!! Also, i'm sure more people besides me would also be happy.

[wacher]

My idea:
- The lights and shadows seem very real on the keyboard. I think that part is an RGB LED-stripe or he use a keyboards with key light or something.
- The uppers side is an recorded midi, transcoded to video with VideoCreator plus fire&smoke effect added.
The first one is tricky second one is not really hard.

[yudihas]

rousseaumusique also does something like this.

He told a brief about his set up here: https://www.reddit.com/r/videos/comment ... _etude_by/

I quote a bit of his explanation:

The final product is a mash up of both. The upper half of the video was done in After Effects, and the LEDs themselves connect via Arduino to the piano's MIDI output. They were set up on a piece of wood which slots into the digital piano slightly behind the keys, close enough to look natural but far enough to not get in the way (yet at the same time inconveniently covers some of the buttons).

It would be really great if we can have this feature in Synthesia without messing around with After Effects. Really, the dust particle effect in the video is awesome.
Anyway, Nicholas.. is it possible for this kind of particle effect (or maybe something more enticing than that ) be built in Synthesia for future updates? For myself I don't mind pay an extra cost for the update as long as it's not overpriced.

[Nicholas]

It would definitely take a lot of engine work to get there. For years and years I've wanted to change some of the core underlying technology in Synthesia (the "big optimization" I discuss here). My background is in computer graphics, so those tasks are the most exciting for me to think about, but I haven't ever been able to justify taking the time away from features requests from users just to make things "faster" when Synthesia already runs pretty smoothly across all our platforms.

But!

I finally decided a couple weeks ago that if I couldn't justify the time, perhaps I could justify the cost instead. I just contracted the services of a graphics programmer (of considerably more talent than myself) to tackle some of those things while I continue to follow the same roadmap as before. The most critical bit of work will be a new Metal rendering back-end for macOS and iOS since Apple announced they've deprecated OpenGL on their hardware. But, the next task on the list after that is the "big optimization".

That should help to bump Synthesia's performance up substantially enough to start considering these sorts of effects.

[yudihas]

That's really nice to hear, Nicholas.

I do hope that those sorts of effects could be implemented in the updates in the foreseeable future. Thanks so much your creating Synthesia, listening to us and making Synthesia better and better.
Wish the best luck.

[yudihas]

Hi Nicholas,

Is there any news about it, whether this feature is plausible to be included in the upcoming (next) Synthesia update?

Thanks.

[Nicholas]

Not the next release, no. I can confirm this won't be in (at least) the 10.5, 10.6, or 11 releases. Some of the groundwork is being laid by the new graphics engineer, but this is a little ways out yet.

[yudihas]

Thanks for replying. Have a great time.