MP3

Synthesia is a living project. You can help by sharing your ideas.
Search the forum before posting your idea. :D

No explicit, hateful, or hurtful language. Nothing illegal.
Post Reply
Dunya
Posts: 8

Post by Dunya »

Hello Nicholas
is there any way to make the program works with MP3 format or any other known formats ?
this will help alot and make everybody likes to buy this great software
Thanks
Dunya
Rickeeey
Posts: 647

Post by Rickeeey »

On the feature voting page there is a feature called mp3 background track:

https://www.synthesiagame.com/features.aspx

The description is:

"A MIDI song should be able to be paired with one or more background MP3/OGG/etc tracks. Ideally the MIDI would contain only the "You Play" components of the piece and the MP3 track would provide higher fidelity (or live) versions of the other parts."

But you're probably asking for a working falling note display for mp3 files. That won't work. But there is a way to convert mp3s to midis but it isn't very accurate and the result isn't good.
Dunya
Posts: 8

Post by Dunya »

Thanks for your reply
But what i am looking for is , if the mp3 can be played in the program so I can see the notes and know how to play it
That is only what i want
When i tried to open it any file with mp3 it didnot let me so
Mos
Posts: 183

Post by Mos »

Note, however, that you won't be able to see falling notes and sheet music unless you use midi. using mp3 will just add background like explained. So don't expect to play the game without midi files.
Rickeeey
Posts: 647

Post by Rickeeey »

Dunya wrote:Thanks for your reply
But what i am looking for is , if the mp3 can be played in the program so I can see the notes and know how to play it
That is only what i want
When i tried to open it any file with mp3 it didnot let me so
Ye I said that it won't work. It's not possible to get notes out of a mp3 file unless you listen to it and write it down yourself.
Rickeeey
Posts: 647

Post by Rickeeey »

Mos wrote:Note, however, that you won't be able to see falling notes and sheet music unless you use midi. using mp3 will just add background like explained. So don't expect to play the game without midi files.
Won't MusicXML files have the same function as midis when MusicXML support gets done for Synthesia?
Frost
Posts: 51

Post by Frost »

Unfortunately, MP3 files do NOT contain any note information, thus, there is no way to show which notes to play. You have to convert MP3 files to midi (or another format with note information) and currently there exists no robust way to do that automatically.
Celemony melodyne will be released in few weeks, maybe that can convert mp3 files to midi without *much* manual editing, but I wouldn't count on it. Just search for midi files, most classical and popular songs have midis anyway.
Dunya
Posts: 8

Post by Dunya »

Well i have even contact David himself and he said i don't have midi files :(
Frost
Posts: 51

Post by Frost »

if the author is willing to make a midi available (and has a digital keyboard), he can just play it once and record that playing to midi.
Mos
Posts: 183

Post by Mos »

Rickeeey wrote:
Mos wrote:Note, however, that you won't be able to see falling notes and sheet music unless you use midi. using mp3 will just add background like explained. So don't expect to play the game without midi files.
Won't MusicXML files have the same function as midis when MusicXML support gets done for Synthesia?
Yes :)
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

Dunya, technology is not that advanced yet, also Mp3 is a "mixed format" meaning it can contain any number of mixed sounds in it which is very difficult to separate accurately. However if you would use monophonic, single instrument recordings like a flute recording for example the results could be good enough. This can be already done now by following command-line sequence

1. find a monophonic .mp3/.wav to .mid converter and convert to .mid
2. load .mid into synthesia using its command-line feature.

In short, it depends which type of .mp3 files you want to use. Most normal mixed mp3 files would not work without any extensive filtering and processing work.
Dunya
Posts: 8

Post by Dunya »

Hi TonE
you said " Most normal mixed mp3 files would not work without any extensive filtering and processing work."
now i have converted how can i filter it ?
Mos
Posts: 183

Post by Mos »

Dunya wrote:Hi TonE
you said " Most normal mixed mp3 files would not work without any extensive filtering and processing work."
now i have converted how can i filter it ?
I think what TonE means is that in order to do a 'great' automatic conversion from MP3 to MIDI, you need a super signal processing filtering algorithems. the conversion programs from MP3 to MIDI out there attempt to do that but no program (so far) is perfect. So, i don't think you can create your own filter (a program to automatically convert MP3 to MIDI) unless you are a super genius :D

That being said, there are great developments in the area and I am guessing some programs are better than others. but you will still need to edit it manually if you want a good MIDI.

the best solutions for the MP3 problem:
- Find an already made MIDI for the song that you want
- try to compose the MIDI by ear. or give the MP3 to some one who can do that.
Rickeeey
Posts: 647

Post by Rickeeey »

Who knowns, maybe in a few years the mp3 to midi technology will be improved alot.
tommai78101
Posts: 766

Post by tommai78101 »

If you really consider that, I'm guessing it's way too farfetched for it to be true.

After a few more years, there will probably be having a new format that incorperates a new way of putting midi information inside MP3-quality music, or MP3-like quality music improvements for midi instruments, or a better way is to improve the sound card, so that pre-recorded CD-Audio quality notes can have the pitches altered to the tone of the midi note.

Anyway, it's too fantastic for us to describe that kind of technology. Wait about 50 or so years...
Hardware Information: Dell Alienware 15 R4, Intel Core i7-8750H @ 2.20GHz / 2.21 GHz, 16GB RAM, Nvidia RTX 2070 / Nvidia GTX 1060 dual-GPU, Roland FP-10, MIDI-OX + LoopMIDI combo.
Frost
Posts: 51

Post by Frost »

After a few more years, there will probably be having a new format that incorperates a new way of putting midi information inside MP3-quality music
it's not the technology that is lacking, it's the motive. there are high fidelity formats with different channels and it would be trivial to add metadata to it. there are musical pieces with creative commons license, that not hinder but actually encourage people to make derivative works. Some of them distribute pieces with traditional instruments separated into tracks and melody parts, so that you can easily edit and create modifications. the problem is the incentive part, most mainstream music would not accept such modifications to make it easier to edit/steal copyrights. hell, they make it even harder to just "play" it with DRM, I don't see such formats happening in near future. (plus there is the problem of non-note based music, most music is not at discrete frequencies, if you examine the harmonics scheme of the music.)
or MP3-like quality music improvements for midi instruments
that is much more possible, and actually is used today. the free/cheaper vsti's are much better than midi, and the higher quality sample based or sample+harmonics based ones are really good. Most film soundtracks, or symphonic additions are recorded inhouse, without any "actual" player, just the synthesizer plus vienna symphonic library etc.
the problem with that approach would be, well, traditional midi won't cut it, and actual music would still be released in waveform data. there is a huge amount of post-processing on the mainstream music that would make it impossible to translate to note based formats. plus, consider the nuances of music. even the piano an create such nuances and it only has key depression velocity and damper pedal that is continuous, not discerete from the perspective of how you can modify the instrument (timing is already in the midi info). thinking of violin; it can play the same passage in infinite variations. not just the different techniques, pizzicato etc., but the position and angle and velocity and acceleration and change on the acceleration of the bow... and the human voice...
If you are synthesizing the midi yourself, you can create different tracks for col legno, tremolo, pizzicato. but no way that it can take the actual recording. Synthesizing/generating music to map the lower dimensional sheet to the higher dimensional waveform is quite hard.

the first idea is actually more feasible, that is, transcription from waveform. you map from a waveform to a sheet music, which reduces dimensionality (sheet contains much less information), thus it is easier. For piano, the process is even eaiser, considering the sound is discrete notes with a very specific note onset (the loudest sound is created is when you first press the key).

There are many studies on that subject (both monophonic and polyphonic), from 60% to 95% note accuracy. However they are not 100%; they miss some notes, or can find the notes but can't find the octave correctly (mapping the notes to a dissonant one is actually very few). I believe in a few years we can see much more accurate ones; Zenph studios claim that their method can transcribe piano recordings with very high fidelity. they use this information to convert the recordings of the old masters to "xp midi" (higher information) to be played in disklaviers, they even replayed the exact performance of Rachmaninoff in a concert. http://musicthing.blogspot.com/2005/05/ ... music.html and http://www.zenph.com/

Also, the new melodyne version would have polyphonic transcription/modification and it looks much better than anything we have right now. see http://www.youtube.com/watch?v=jFCjv4_jqAY . Also watch the Frankfurt demo.

Considering, they are constantly shifting the release, it was supposed to "finally" come last year. But open beta is supposed to come this month, and released in september, so if they do not put off it further, we can test how it fares in actual music. there are some rivals as well; I belive we can see much better polyphonic transcription in few years.


However, most people would like to play piano parts of multi-instrument or non-piano based songs, and it would still require much manual editing. so, yes, we will see such requests constantly :)
tommai78101
Posts: 766

Post by tommai78101 »

Frost wrote:it's not the technology that is lacking, it's the motive. there are high fidelity formats with different channels and it would be trivial to add metadata to it. there are musical pieces with creative commons license, that not hinder but actually encourage people to make derivative works. Some of them distribute pieces with traditional instruments separated into tracks and melody parts, so that you can easily edit and create modifications. the problem is the incentive part, most mainstream music would not accept such modifications to make it easier to edit/steal copyrights. hell, they make it even harder to just "play" it with DRM, I don't see such formats happening in near future. (plus there is the problem of non-note based music, most music is not at discrete frequencies, if you examine the harmonics scheme of the music.)
Motive? That's a first. I'm thinking, that if we have a group of people, with the same motive, and enough fundings, we could start a project that involves open-source, modifications supported by various programs, and granted permissions from the music industries/companies. but, it's a thought. Of course, these formats you mentioned won't be happening in the near future, literally...(50 years later counts as "near future"?)

I really can't find any flaws with the copyrights. Because most mainstream music would not accept these due to the fear of having their works copied/edited/modified. It should be hard to find a format being acceptable by standard means...That I can't argue.

And wouldn't there be a chance of having such formats to exist after 50 years, even if it's 0.01% or even less? I believed in that.
that is much more possible, and actually is used today. the free/cheaper vsti's are much better than midi, and the higher quality sample based or sample+harmonics based ones are really good. Most film soundtracks, or symphonic additions are recorded inhouse, without any "actual" player, just the synthesizer plus vienna symphonic library etc.the problem with that approach would be, well, traditional midi won't cut it, and actual music would still be released in waveform data. there is a huge amount of post-processing on the mainstream music that would make it impossible to translate to note based formats. plus, consider the nuances of music. even the piano an create such nuances and it only has key depression velocity and damper pedal that is continuous, not discerete from the perspective of how you can modify the instrument (timing is already in the midi info). thinking of violin; it can play the same passage in infinite variations. not just the different techniques, pizzicato etc., but the position and angle and velocity and acceleration and change on the acceleration of the bow... and the human voice...
Hm...good point. I can't argue with that also.

Another thing, DirectX can handle VSTI perfectly. If Nicholas can implement that into Synthesia in some way, one or another, then Midi outputs might sound as close to CD-quality MP3 counterparts. In Windows 7, Microsoft Sam possibly have been updated to speak in a natural tone.

For human voices, ever heard of the Vocaloids? I have that. It utilizes human voices and converts them to waveforms, but the program itself uses a midi-GUI synthesizer. The metadata it uses has more information than a complicated midi file. So, maybe that's a good instance for you. The notes created by the synthesizer contains key elements for tracking down the correct waveforms pre-recorded.

It's goes like this: Middle C note --> Finds the metadata for it. --> Found the right waveforms with the metadata applied --> Output of the waveform.

Maybe the technologies in the "near future" might use Midi and make it, or them notes, to output the required formatted data, giving an authentic sound of the instruments.
Hardware Information: Dell Alienware 15 R4, Intel Core i7-8750H @ 2.20GHz / 2.21 GHz, 16GB RAM, Nvidia RTX 2070 / Nvidia GTX 1060 dual-GPU, Roland FP-10, MIDI-OX + LoopMIDI combo.
Post Reply