Instruments Play Incorrect Sounds

Trouble with Synthesia, your keyboard, or adapter? Think you found a bug?
When describing problems, always mention your OS and game version (shown at the bottom of the title screen).

If your keyboard has USB or MIDI ports, there is a tremendously high chance (>99%) it will work with Synthesia. See what you'll need on the keyboards page.
Post Reply
User avatar
DJ_Fox11
Posts: 4

Post by DJ_Fox11 »

I have encountered an issue in the latest version of Synthesia as well as the developer version where some instruments will play incorrect sounds. This seems to happen with MIDI files that have a large number of instruments. In my case, the Bassoon, Clarinet, and French Horn are currently playing the sounds of Tremelo Strings. This issue isn't only for this one MIDI file; as stated before, it happens with any file that has a large instrument count. This isn't the best for listening and, in my opinion, ruins the songs.
Cheers,
DJ

OS: Windows 11 22H2
Synthesia Version: 10.8.5686
Bavi_H
Posts: 116

Post by Bavi_H »

Could you share a MIDI file that demonstrates the problem?
User avatar
DJ_Fox11
Posts: 4

Post by DJ_Fox11 »

Bavi_H wrote: 01-24-23 12:02 pm Could you share a MIDI file that demonstrates the problem?
Thanks for the reply, one of the MIDI files this issue happens with is here:
MRKB_Hoppers.mid
(81.81 KiB) Downloaded 30 times
Through testing in Musescore, and other MIDI players, Synthesia appears to be the only app to trigger this.
Cheers,
DJ
Bavi_H
Posts: 116

Post by Bavi_H »

Thanks for sharing an example MIDI file.
DJ_Fox11 wrote: 01-24-23 5:42 pmThrough testing in Musescore, and other MIDI players, Synthesia appears to be the only app to trigger this.
Out of curiosity, what other MIDI players worked for you?

I'm not sure if I can suggest a good solution yet. This is one of the limitations of MIDI files. In overly simplified terms, a MIDI signal consists of 16 MIDI channels, so a MIDI file can only support 16 instrument sounds at the same time. If a MIDI file needs to support 17 or more instrument sounds, one or more of the channels must have its time divided into different instrument sounds. (See MIDI Channels and Multitimbral Sound Modules for an analogy using "robotic musicians".)

Synthesia is mainly designed for MIDI files and MIDI output devices, so the 16 MIDI channel limitation comes with that. Perhaps MuseScore's notation format and sound synthesizer isn't as closely tied to MIDI concepts so it is able to generate 17 or more instrument sounds at the same time.

One laborious solution would be to manually fix the MIDI file so that some channels have their time divided into different instrument sounds, or move the notes for similar instrument sounds on different channels into a single compromise instrument sound on one channel. Perhaps we might find a tool or process to automate fixing MIDI files in this way. (For example, if an automated process could determine when musical parts are not used at the same time, it could insert Program Change messages at different times on the same channel to change the instrument sound.)


Multiple MIDI output devices
  • The MIDI Manufacturers Association created a new MIDI file event ("Device Name") that can indicate a track should correspond to a different MIDI output device. Cakewalk created a non-standard MIDI file event ("Port") that serves the same purpose of indicating a track should correspond to a different MIDI output device. These events can be used to help indicate more than 16 channels of instrument sounds. However, Synthesia doesn't use Device Name or Port events, and these events aren't consistently supported in other MIDI software. If MIDI software does understand these events, you have to enable multiple MIDI output devices to ensure all the channels play correctly.

    Your example MIDI file contains some Port events, but the Program Change events are all at the beginning of the first measure. After Synthesia completes its part splitting process, if a channel is reused, Synthesia will end up sending multiple Program Change messages to the channel at the beginning of the playback and only the last one sent will control what instrument sound you hear all parts that use that channel. (Also be aware that the example MIDI file might not be using the Port events correctly. In the example MIDI file, the Port events come after the Program Change and several Control Change messages for each channel. And in the tracks that contain multiple channels there are multiple Port events. My understanding is the Port event is supposed to appear at time zero and before any channel events and there should only be one Port event in a track.)

    (If Synthesia used Device Name or Port events to actually route notes to different output devices, you'd have to select multiple output devices to hear more than 16 channels. If Synthesia used Device Name or Port events as part of its process to split notes into different parts, it probably wouldn't help. There's supposed to only be one Device Name or Port at the beginning of a track. Since Synthesia already uses tracks to split notes into different parts, looking at the Device Name or Port events wouldn't add any more information to help Synthesia split notes into parts.)
User avatar
DJ_Fox11
Posts: 4

Post by DJ_Fox11 »

Thanks for that explanation, that really helps!

The MIDI players that don’t make this issue occur, (through my testing) are Ableton Live, LMMS, MIDITrail (iOS app), and Musescore, as stared before.

Is it possible for you to give an explanation on how to fix the MIDI file? Or is it too complicated?

Cheers,
DJ
Bavi_H
Posts: 116

Post by Bavi_H »

In this post:

1. Use Sekaiju to examine MIDI files
2. Example MIDI files with many instrument sounds (re-used channel)
2a. Staggered Program Change
2b. Device Name or Port
3. Fixing your MIDI file
4. Other issues in your MIDI file
5. Search for tools


1. Use Sekaiju to examine MIDI files

To investigate the technical details inside a MIDI file, I recommend using the MIDI sequencer Sekaiju. In particular, Sekaiju's Event List window will show you every event in the MIDI file. To open an Event List window that shows everything, use the View menu command "Show new Event list window" or click the toolbar button that looks like a little spreadsheet table. Once you are in an Event List window, you can use the top-right pane of check boxes to change which tracks are shown in the event list.


2. Example MIDI files with many instrument sounds (re-used channel)

Take a look at these example MIDI files I made.

16-instruments.zip
(1.25 KiB) Downloaded 26 times

Each MIDI file has 16 instrument sounds. I skipped over Channel 10, the drum percussion channel. I used the other 15 channels (Channels 1 to 9 and 11 to 16) for 15 different instrument sounds. Then I re-used Channel 1 for a 16th instrument sound. You can examine the example MIDI files in Sekaiju's Event List to understand how they are built, then you can open them in Synthesia and various MIDI file players to see how they handle them.


2a. Staggered Program Change

In a MIDI signal, a "Program Change" message selects a new instrument sound for a channel. In the "staggered Program Change" file, the Program Change messages are all at the beginning of measure 1 except for the last track. Because the last track re-uses Channel 1, I put its Program Change message at measure 10, after "all the notes" for the the first instrument sound that uses Channel 1, and before "all the notes" for the second instrument sound that uses Channel 1. (In my example files, there's only one note per instrument sound, so "all the notes" for each instrument sound is just one note.) When you play the "staggered Program Change" file in Synthesia, you will hear all of the intended instrument sounds.

staggered-program-change.png
staggered-program-change.png (7.24 KiB) Viewed 1528 times


2b. Device Name or Port

In a MIDI file, the "Device Name" or "Port" events tell the MIDI file software to send the messages in a track to a different MIDI output device. In the "Device Name" and "Port" MIDI files I made, I put all the Program Change messages at the beginning of the first measure.

port.png
port.png (8.86 KiB) Viewed 1528 times

If you import the "Port" MIDI file in LMMS, it will recognize the last track, which re-uses Channel 1, as if it were a completely unique channel.

If you open the "Port" MIDI file in Sekaiju it will set the last track to use the 2nd output port (which defaults to nothing). If you happen to have two MIDI devices, you can go to the Setup menu command "MIDI Devices and Instruments" and choose different devices for Out Port01 and Out Port02, then when you play the file you'll hear the last instrument sound on the 2nd device.

Synthesia doesn't use the "Device Name" or "Port" events to determine output devices, so when you play these files in Synthesia, it will send both Program Change messages for Channel 1 at the beginning of the first measure. The last Program Change message that was sent will control the sound you hear for both the first and last note.


3. Fixing your MIDI file

The general idea is to change your "Port" kind of file into a "staggered Program Change" kind of file.

After you open your MIDI file, you need to examine the groups of notes intended for each instrument sound to see if there are any groups that don't overlap. If you find some non-overlapping groups, then you could edit the MIDI file so those groups use the same channel and put an appropriate Program Change in front of each group to set the intended instrument sound.

If all the instrument sounds are always used at the same time, then the other idea I mentioned was to look for instrument sounds in the MIDI file that are almost the same or are part of the same instrument family. You might decide to change them into one "compromise" instrument sound. For example, if a MIDI file has both violin notes and viola notes you might decide to change them both to violin notes and put them on the same channel.


4. Other issues in your MIDI file

Be aware that your MIDI has some annoying things going on in it. [It appears your MIDI file was created by MuseScore. I have only used MuseScore a little bit and I have been frustrated with its MIDI export and import.] If I were fixing the MIDI file, I would also clean up these other issues:

Your example MIDI file incorrectly puts notes into the tempo track. Although lots of MIDI file software including Synthesia doesn't care about this and will play the notes anyway, I believe some MIDI file software won't see notes in the tempo track. (If you open your MIDI file in Sekaiju, Sekaiju will ask you if it can move the notes and other channel messages from the tempo track into a new track. If you click No, Sekaiju opens the file in a non-editable state. This lets you view the MIDI file, but not edit or re-save it, because Sekaiju doesn't want to let you create a non-standard MIDI file.)

Your example MIDI file confusingly puts key signature events on every track and possibly uses the "fingering notation" key signature for transposing instruments. MIDI files usually only contain the "sounded" key signature in just the tempo track which applies to all tracks in the file. (The MIDI file specification doesn't specifically say key signature events should only go in the tempo track, but that is the de facto behavior. When you open your MIDI file in Sekaiju, Sekaiju shows another "fix or view-only" prompt for the key signature events.)

Your example MIDI file has some tracks that contain messages for more than one channel. However the tracks with multiple channels only have notes on one channel, the other channels have setup messages, but then no notes. (A track containing multiple channels is allowed per MIDI specifications, but most MIDI file software is designed around the idea of a track containing a single channel. Sekaiju doesn't complain about it, but it is easy to get confused while editing the file when there are different channels in a track.)


5. Search for tools

I'd like to find MIDI file tools that might help during fixing MIDI files like this. In particular, I'd really enjoy some way to split MIDI file events into separate tracks based on their original track, channel, and program change number -- the same criteria Synthesia uses to determine "parts". That kind of track splitting process would help organize messy MIDI files into separate tracks, then I could more easily use Sekaiju to examine and fix the file the way I like.
User avatar
DJ_Fox11
Posts: 4

Post by DJ_Fox11 »

Thanks for all the information, I’ll see what I can do!
The in-depth explanation really helped me understand what was going on, and how to fix it!

Thanks for all the help!

Cheers,
DJ
Post Reply