MOD/XM/S3M Support

Synthesia is a living project. You can help by sharing your ideas.
Search the forum before posting your idea. :D

No explicit, hateful, or hurtful language. Nothing illegal.
Post Reply
liero116
Posts: 5

Post by liero116 » 07-05-09 5:00 pm

It would be cool to have support for various module formats such as .mod or .xm. It is a tracked audio file format very similar to midi that is used to produce chip tunes. I have a very large collection of these and it would be awesome to be able to play them in this game.

Nicholas
Posts: 12343

Post by Nicholas » 07-05-09 6:50 pm

I have support for a couple other formats on my list (like MusicXML), but MOD might be a little tricky.

If I understand correctly, MOD not only contains the MIDI-like note signals, but also the fragments of wave data that should be played (at a particular pitch/volume/etc.) for each note.

In order to use those, I'd either need to find a flexible MOD library that could do the loading/playing (and allow interspersing user-played notes, too) with the appropriate software license, or I would have to make my own.

liero116
Posts: 5

Post by liero116 » 07-06-09 6:16 am

That does sound about right, though my knowledge only really extends to my use of the program ModPlug Tracker several years ago. On that note, if you were able to accomplish such a thing, would it also be possible to incorporate .flp readability? I am not 100% sure how the file format works, but it is a proprietary format used by Fruityloops. If I am not mistaken it is similar to these other formats only it references wav or vst plugins. I would imagine, however, that this would require a much more powerful computer as well as an ASIO capable card to reduce latency when using Midi inputs.

soryy708
Posts: 57

Post by soryy708 » 07-06-09 8:06 am

.flp? omg! he would need to buy a license to use it from imageline studios!
infact, why not making ur own fruity loops if u can make .flp reading programs? :o
Thats why it might turn pricey! maybe like a source code of a game engine... around 90,000,000 dollars! (Like unreal engine)

tommai78101
Posts: 762

Post by tommai78101 » 07-10-09 7:52 am

Can IT files may be a workaround?

There's a program called Modplug Tracker that uses the IT file. I wondered if it's the same as a Midi with Wave pitched in.

The source code is here at the bottom: http://sourceforge.net/projects/modplug/files/
Hardware Information: Windows Vista Home Premium SP1, 358MB Mobile Intel Graphics Media Accelerator X3100, Synthesia 0.7.1 preview r697, 2 GB DDRAM, 1.6 GHz Intel Pentium Dual-Core Processor T2330, Acer Aspire 5720-4126
New Hardware Information: Windows 10 Pro, 2GB Nvidia GeForce 860M, 8GB RAM, 1.7GHz Core-i5 4210U, Alienware 13 R1.

aria1121
Posts: 1505

Post by aria1121 » 07-03-12 10:33 am

Recently I've been thinking about this. Isn't it possible to route audio output to a program as VLC (has S3M support for sure). You could route pitch and stuff to Synthesia.
S3Msupport.png
S3Msupport.png (68.8 KiB) Viewed 8047 times

Nicholas
Posts: 12343

Post by Nicholas » 07-03-12 12:05 pm

In a (three year) newer topic over here, I've changed my tune and would actually like to include really nice support for MOD-like formats. Once the FluidSynth-based software synth goes in after 0.8.4, this shouldn't be that bad to add.

aria1121
Posts: 1505

Post by aria1121 » 07-03-12 5:35 pm

Nicholas wrote:In a (three year) newer topic over here, I've changed my tune and would actually like to include really nice support for MOD-like formats. Once the FluidSynth-based software synth goes in after 0.8.4, this shouldn't be that bad to add.
Nice! Which filetypes does that include?

Nicholas
Posts: 12343

Post by Nicholas » 07-03-12 6:11 pm

I have spent zero minutes looking at this to confirm if it's actually true, but looking at the source tree for the project, it looks like it supports:

669, abc, amf, ams, dbm, dmf, dsm, far, it, j2b, mdl, med, mid, mod, mt2, mtm, okt, pat, psm, ptm, s3m, stm, ult, umx, wav, and xm. :lol:

Now that doesn't necessarily mean all of those (e.g., WAV) would necessarily be compatible in Synthesia.

User avatar
stephenhazel
Posts: 223
Location: Seattle, WA stephen.hazel@gmail.com

Post by stephenhazel » 07-03-12 6:50 pm

wow - that thing has an audio rendering engine in it that's a lot like fluidsynth.
(interpolation, mixing, fx and all that math jazz)
and some assembly language :shock:

Ooooone thing to consider is how many of those mod files would be PLAYABLE by a human.
That's kinda Synthesia's thing (and my PianoCheater's).
It'll be cool to look at how the rendering engine does it's thing.
It'll be cool to =listen= to .MODs in Synthesia.

But I'm not sure how many .MODs have music NOTES that are playable by humans...
If i recall from my ooooooole Amiga days, there's a lotta "play this half bar of a full drumset" NOW
with THIS fx. Now swap to THIS half bar of full drumset mix NOW for the fill with THIS fx...
Great fun to listen to, but I'm not sure how many of those a human would PRActice... :mrgreen:
That'd be for a full 8 bars of drums - 16 snare hits on the half beat with one different at the end.

Once i get my sheetmusic decent (some century), I'm gonna build a softsynth from the ground up.
I've got a decent start at that. And this code will probably get me reeeally close to there :shock:


so GUYS, you guys that want to load .MOD and .ITs...
Do you ever play/practice those things? Or do you ONLY listen to em?
Have you ever converted one to .MID?
If I recall, most of those things were 8 bit samples, too... Hmmm...

Nicholas
Posts: 12343

Post by Nicholas » 07-03-12 11:32 pm

stephenhazel wrote:Do you ever play/practice those things? Or do you ONLY listen to em?
A personal interest of my own is ancient video game music. And I know a number of those formats are MOD-like. It's probably a little self-serving, but I would love to be able to plunk out those notes and have the resulting audio be true to the source material. That would be the best! (So the propensity for them to have 8-bit samples is actually a pro for me instead of a con. :D )

So I have to admit that a lot of my enthusiasm for MOD is that it will hopefully get Synthesia a little further along the path toward NSF or SPC. Niche? Absolutely. Is that going to stop me? Probably not. :lol:

Still, I know better than to do that work anytime soon. There are glaring holes that need to be filled first. Then I can move on to fun side-projects.

User avatar
stephenhazel
Posts: 223
Location: Seattle, WA stephen.hazel@gmail.com

Post by stephenhazel » 07-04-12 12:34 am

Yeah, hmm, I think I remember the Amiga's MED program having like 4 channels it'd do.
So, i GUESS you'ld only need 4 fingers max to play those things live.
It'd kinda be cheating cuz the samples are playing multiple notes for ya.
But it'd kinda not be cheating... Cuz that's what DJs do for a living...
And all's fair in love, war n music.
Hmmm... Kinda makes a guy not wanna work on sheet music notation :?

aria1121
Posts: 1505

Post by aria1121 » 07-04-12 2:16 pm

Nicholas wrote:
stephenhazel wrote:Do you ever play/practice those things? Or do you ONLY listen to em?
A personal interest of my own is ancient video game music. And I know a number of those formats are MOD-like. It's probably a little self-serving, but I would love to be able to plunk out those notes and have the resulting audio be true to the source material. That would be the best! (So the propensity for them to have 8-bit samples is actually a pro for me instead of a con. :D ) So I have to admit that a lot of my enthusiasm for MOD is that it will hopefully get Synthesia a little further along the path toward NSF or SPC. Niche? Absolutely. Is that going to stop me? Probably not. :lol: Still, I know better than to do that work anytime soon. There are glaring holes that need to be filled first.
I usually make sheets from S3M files from dem awesome games. Synthesia sure will help with that. If needed I also can rewrite the S3M so it is playable with Synthesia.
Nicholas wrote:I have spent zero minutes looking at this to confirm if it's actually true, but looking at the source tree for the project, it looks like it supports: (spoiler)
That would be fun if you added a new box that says

Code: Select all

Now with 669, abc, amf, ams, dbm, dmf, dsm, far, it, j2b, kar, mdl, med, mid, mod, mt2, mtm, okt, pat, psm, ptm, s3m, stm, ult, umx and xm (add upcoming filetypes here) support!
like all those other ("famous") software
Nicholas also wrote:Then I can move on to fun side-projects.
<interest> :P </interest>
stephenhazel wrote:But I'm not sure how many .MODs have music NOTES that are playable by humans...
Do it for the soundeffects!

User avatar
stephenhazel
Posts: 223
Location: Seattle, WA stephen.hazel@gmail.com

Post by stephenhazel » 07-11-12 5:15 pm

Hmmm, well, I couldn't help takin' a quick look through that libmodplug thingy.

I could be wrong, but it SEEMS to be sitting squarely on CSOUND for doin' it's audio...
And I think csound is GPL or whatever.
libmodplug is freeware sitting on top of csound which is gpl which i THINK reverts libmodplug to gpl...??
(I can't keep track of that dang license junk - blargh)

So although you can see how to load a .MOD file and .IT and allllll those other formats.
You're gonna need csound to dump out any audio i thiiiiiink :(

So if you really wanna see how to DO all those crazy fx things, you gotta dig into csound.
(And thaaaat doesn't sound too fun to me).

But way back in the late 80s I wrote a mod2mid c program. It was pretty easy if all you
care about is notes and spare teh fx and samples.
But then, a "note" could be a whole measure of the full drumtrack in a sample.

If you go thru the hell of writing a softsynth, a single .WAV instrument is a pretty
simple beast. So it's conceivable you could turn the notes into midi and the
samples into a soundfont (or whatever sort of "instrument" your softsynth uses).
But your fancy whizzbang fx are gonna be another matter.

I could be wrong about any of that...

Has anybody found a freeware SIMPLE AS PIE softsynth of some sort?

FluidSynth is GPL i think and isn't bad. But it's still pretty OVERLY complex.

BasicSynth is freeware i think, but I haven't been able to grok it so far.

CSOUND is supposed to be the bees' knees - but it's BIG - also can't grok it...

I'm hoping to write a softsynth that JUST plays wav instruments period.
Just interpolation, no envelopes,filters,fx,blah blah blah.
Well, after this godforsaken sheet music gets done, which'll be never... :?

Electrode
Posts: 187

Post by Electrode » 07-13-12 9:01 am

Some (not-so-small) issues to consider with MOD files and (anything under 16 tracks to be honest) are that:

- some samples may be tuned differently in the file - so a note that sounds as an A might be written as something completely different in the file because of the sample's tuning. MOD/S3M and older formats along those lines had strict tuning limitations that prevented some samples from being written in the file at sounding pitch.

- MODs have limited octave range, so multiple samples had to be used to convey range. A common example would be when Middle C with sample 2 would sound an octave higher than middle C with sample 1. Same sound, different sample, different octave.
- Especially in MOD format, one sample can contain an entire major or minor chord. These would only be written using one note in a MOD file, so how would you be able to extrapolate the notes?
- Percussion and arpeggiated chords can often be spread over several tracks/channels for aesthetic reasons (not visually, of course, but audibly).
- Tracks could have a melody instrument on them for one part of the song, and a percussion instrument on them in another part of the song. Track selection is not standardised as MIDI is. It's a very ad-hoc thing.

In short, this is a lot bigger than "It looks like a MIDI-style format so it should be easy".

These tricks are used even in files greater than 16 tracks to get the sound of far more tracks. (Realistically, if you wanted the best shot at implementing this, then Impulse Tracker's IT format is the only one that comes close to approximating the benefits of MIDI.)

Remember that these types of files came out of the demoscene in the late-80s/early-90s when coders were pushing limited technology to breaking point in order to do amazing things and have bragging rights! Even the musicians were thinking "I need a 64-piece orchestra but only have 16 tracks in my S3M. I need to get that feeling by using a whole string chord as one sample." (For example.) There had to be sacrifices somewhere, and it is these things that really make implementation of these formats a bit difficult.

Think about it. If MOD to MIDI converters aren't perfect (and they can never be because of the above limitations), Synthesia's support of MOD and similar (SPC etc.) formats won't be either. There will have to be compromises because the hardware limitations of the time forced people to think of and use workarounds that don't translate well to other uses.

Sorry to rain on the parade a bit, but this would need a lot of planning.

User avatar
stephenhazel
Posts: 223
Location: Seattle, WA stephen.hazel@gmail.com

Post by stephenhazel » 07-13-12 11:28 am

Just pulling out the notes won't be enough.
BUT, if your softsynth can load a .MOD's samples and use them as an instrument,
then you're in business.
The midi says play middle c for a whole note on program change 99.
The softsynth has loaded the .MOD sample and uses it for program change 99.
It sounds just like the .MOD, but maybe not quite due to fx.

Nicholas
Posts: 12343

Post by Nicholas » 07-13-12 3:00 pm

Electrode wrote:Think about it. If MOD to MIDI converters aren't perfect (and they can never be because of the above limitations), Synthesia's support of MOD and similar (SPC etc.) formats won't be either.
That seems reasonable though. Anyone that would want to play a .MOD in Synthesia would hopefully understand those limitations and what they mean, too. I'm not sure I'd worry too much about trying to "understand" what each sample meant in terms of it being a chord or whatever. Like you said: the best I could do is just show the notes. If the file happens to be amenable to the falling-note format, so be it.

Again, this is going to be a side, side, side project at some unspecified point far into the future. :D

User avatar
stephenhazel
Posts: 223
Location: Seattle, WA stephen.hazel@gmail.com

Post by stephenhazel » 08-17-12 1:47 am

Well, I just found the archive of Amiga SoundTracker sounds !!
10,641 samples !! (but all mono 8 bit :mrgreen: )

I found an archive of em converted to .WAV, too.
But the sample frequency was always 44,100 and there were no loop points :(
And so you wouldn't be able to USE em with, sayyy, your own hand coded softsynth.
Sooooooo, I dug out my ooooooold 8svx sample reading/writing code circa the '90s and
wrote a conversion program myself.

THAT is a satisfying thing to do :ugeek:

Post Reply