Page 1 of 1
Posted: 05-15-23 2:09 am
by badmozart
Hello.

I've been using Synthesia for more than a year and it is great.

However, I really wish it had support for lyrics. I believe it's possible for MIDI files to have lyrics embedded in them as text events. It would be cool if Synthesia would pick a format for text events in midi files such that when I play those files it displays the lyrics along with the notes.

I think the best interface would put the lyrics in white text just above the keyboard at the bottom of the screen (i.e. the same place where you look to see which note to play next).

Where to get midi files with lyrics embedded in them? I think the best approach would be to build a tool that automates this where you pair a midi up with an mp3 file, rip out the voice track with spleeter (or other), audio to text it with another AI tool, match the notes up with spectral analysis to place the individual words in the right place in the file.

Pretty interested in this, might start poking around to build an open source solution if nothing already exists.

Posted: 05-15-23 10:03 am
by jimhenry
Interesting ideas. There are a lot of MIDI karaoke file available. They have a .kar extension. This article about MIDI karaoke formats might be of interest: https://www.mixagesoftware.com/en/midik ... rmats.html

It would be nice if the karaoke syllables fell in time with the notes but that might be tricky given that their width is variable.