0.7.1 Preview r792, r797 (Release Candidates 1 and 2)

Archived development update discussion from past versions
Archived development updates.
maccer
Posts: 222
Location: Sweden

Post by maccer » 05-22-10 5:05 am

The new auto-track splitting probably works well most of the time, but for those times it doesn't, would it also be possible to force the tracks to be locked to a specific staff? In "real" sheet music, the reason for dividing the notes on the top/bottom staffs is that the top staff notes should be played with the right hand and the bottom staff ones with the left hand. Therefore, the notes need to be shown on specific staff even if it requires the usage of exceptionally many ledger lines.

I have attached the original notes as PDF for Sibelius Op. 76 Etude and a two-track MIDI transcribed from those notes. I've then loaded that MIDI into Synthesia. Compare Synthesia's output (the attached image) from the beginning of the song with the original notes. The circled notes that Synthesia draws on the top staff should be played by the left hand and be drawn on the bottom staff. And this is just in the beginning of the song - it becomes worse later on.
Start_of_Etude.png
Start_of_Etude.png (30.83 KiB) Viewed 14519 times
Attachments
Sibelius_Op76_Etude.mid
(7.51 KiB) Downloaded 211 times
Sibelius_Op76_Etude.pdf
(343.89 KiB) Downloaded 259 times
Songs learned using Synthesia:
CT: Wind Scene, The Trial | FF7: Prelude | SMB: Overworld, Underwater | Tetris: Theme A | Zelda: Lost Woods | Other: Für Elise

Nicholas
Posts: 12435

Post by Nicholas » 05-22-10 5:59 am

The problem in that one is that a couple times, the left-hand part goes very high. The first time is in measures 24-28. If you notice, the way the PDF sheet handles it is by switching to double-treble staffs.

If you were to force it to the bass staff, at those few high points, you'd need something like 4 or 5 ledger lines. I've never seen that many (between staves) in sheet before. The compromise in my algorithm whenever there is an absolute max note that high (or likewise, a minimum note too low on the treble staff) is to throw up its hands in frustration and revert to the old behavior. :D

This isn't quite as extreme as an example as, say, the hand-over-hand part in The Sims piece that ships with the game, but it's still pretty easily explainable away. If you were to jump into an editor and remove those high parts, the notes in the section you took a screen shot from would jump to the more correct positions on the bottom staff.

An alternative to reverting to the old behavior is to maybe start thinking about a double-"same clef" staff scheme, or something like 8va support.

maccer
Posts: 222
Location: Sweden

Post by maccer » 05-22-10 12:56 pm

Nicholas wrote:The problem in that one is that a couple times, the left-hand part goes very high. [...]
If you were to force it to the bass staff, at those few high points, you'd need something like 4 or 5 ledger lines.
Yes, this etude is a bit extreme and you would need 4 ledger lines, but why is that a problem? For me, the current display is more of a problem than the extra ledger lines (though I don't need Synthesia to learn this particular piece as I just learnt it from the sheet music). The ledger lines don't take up that much extra vertical space. And when you get the full-screen sheet music feature in there with multiple lines of sheet music per page, only those lines that need the extra ledgers would need to be extra high.

Programmatically it should be much easier since you don't need any algorithm at for the splitting, and UI-wise you could have a dropdown for each track where you could select "Auto" (your algorithm), "treble staff", "bass staff" etc. 8va support would probably make it easier to understand what notes those extremely high notes on the ledger notes actually are.
Songs learned using Synthesia:
CT: Wind Scene, The Trial | FF7: Prelude | SMB: Overworld, Underwater | Tetris: Theme A | Zelda: Lost Woods | Other: Für Elise

User avatar
FuzzyWuff
Posts: 9
Location: Russia

Post by FuzzyWuff » 05-23-10 5:27 am

I have been a lurker for quite a long time now and visited only the forum for Midi research.
But then I think it's time to be more useful and see if I can help on any possible way.
I am running 2 computers, one is a laptop. Both are running win7 with the latest release of Synthesia.

On the laptop, I am running it with a P840 at 2.26GHz on a 32bit system and 4GB (3 usable) of DDR2 ram, the card is a GeForce 9600m (see model ASPIRE 8930G)
You may want to know the laptop sadly is running on a 5200RPM hard-drive.

For the main computer, it's the 64bit version of win 7 and a custom build running a Q8400 at 2.66GHz and 4GB of ddr2 ram and the card is a GeForce GTX 260 EXO (special edition).

I am now downloading the preview version and will proceed with an extensive stress test on both computer. For information, I am using a Technics SX-P30 88 touch synthesizer including the sustain pedal. Old but extremely great to play with. ( I started Synthesia long ago with a Yamaha E213). Both are linked with a Cakewalk Roland UM-3G Midi Interface (out only, I did not found the necessity yet to link Synthesia with an IN Midi cable, does it affect anything in particularity anyway ?).

So the test begin this weekend (MAY 23th) for an extensive period of 4 to 7 hours per day. From there, I will post video, feedback and possible debug output if Nicolas want extended information on this stress test.

I will wait your reply to this post before starting the test if you have any suggestion you want me to add and I will be open for comment and suggestions as well.
I am using Gtalk, WLM or/and email if you are interested on a live feedback and it's with pleasure I will answer to any question you want to know.

Sadly, for the DirectX test, I am always updating my files. So I am afraid unless I uninstall for test purpose the drivers, I am unable to give you a proper feedback on this matter.

Best Regard,
Vinnie Shepard.

Code: Select all

This Signature is in Read-Mode only!

Nicholas
Posts: 12435

Post by Nicholas » 05-23-10 9:00 pm

maccer wrote:this etude is a bit extreme ... but why is that a problem?
I could expose the min/max value (right now it's 4 half-steps above or below Middle-C) in a registry setting or something. The real, not very good, answer is that right now there is a technical limitation in my code where I don't have good control over the staff spacing. You'll notice they're still in exactly the same places they used to be relative to one another. Once you get past 3 or so ledger lines in both directions, notes will start to overlap in ambiguous ways.

I'm guessing with the multi-line sheet feature that is coming up (next?), I'll be gaining that control. At that point, if you want a registry setting that will allow up to 15 ledger lines or whatever, it'll be able to do it.
maccer wrote:Programmatically it should be much easier since you don't need any algorithm at for the splitting, and UI-wise you could have a dropdown for each track...
At this point it's a non-issue because the algorithm is already finished and gives decent results most of the time. I am however becoming increasingly sensitive about the number of UI controls popping up all over the place these days. I'm trying to maintain something that is both very (very!) easy to use for new players, yet also satisfies the power-user types that tend to hang out here in the forums. Each time I add a new drop-down or whatever, it gets a little more complex and daunting for new players.

You can argue that extra stuff can be hidden behind "Advanced" menu options or someplace similar. Going down that road, you run the risk of having endless menus to navigate just to set up a song.

These are things that keep me up at night. ;)
FuzzyWuff wrote:I am now downloading the preview version and will proceed with an extensive stress test on both computer.
All of that sounds great. I'll be waiting to hear any information you can provide.
FuzzyWuff wrote:I will wait your reply to this post before starting the test if you have any suggestion you want me to add and I will be open for comment and suggestions as well.
Nothing extra is jumping to mind at the moment. If you find anything you think shouldn't be happening, certainly try to capture that and let me know. Thanks again!

TonE
Synthesia Donor
Posts: 1180

Post by TonE » 05-25-10 3:05 am

I also got the error message with the missing directx, after updating with the file you have given everything worked well. Anyway, I am normally using only the OpenGL drawing mode, as only this supports --windowed parameter.

Nicholas
Posts: 12435

Post by Nicholas » 05-25-10 3:50 am

Good to hear. At some point DX will support windowed mode too. I was super paranoid before about trying to do anything interesting. My DX wrapper layer was some Microsoft code from 2007'ish, shoe-horned to work with that old 2006 runtime, all building with modern platform files.

Now it's newest working with newest, building on newest. Having a clean environment like that boosts my confidence. :)

TonE
Synthesia Donor
Posts: 1180

Post by TonE » 05-25-10 10:32 pm

Regarding the song bookmarks feature:
Song Bookmarks (finished 02-02)

Finished: This task is complete and will be in the current or next development release.
Some MIDI files contain section "markers". These should be shown on the progress bar and input-assignable game functions should be added to jump to previous, jump to next, and add a new marker. New markers won't modify the MIDI file itself but will be saved as a song preference.
Is it possible to display the marker text on the falling notes view if they are strings in the set of {do,re,mi,fa,sol,la,si}. Then while learning the notes one could also sing to them correctly, if the midi files contain already "markers" as {do,re,mi,fa,sol,la,si} text strings.

Nicholas
Posts: 12435

Post by Nicholas » 05-26-10 2:42 am

I'm not sure I understand. You'd be creating markers every note? ...with the sung note name as the text?

TonE
Synthesia Donor
Posts: 1180

Post by TonE » 05-26-10 5:13 am

Nicholas wrote:I'm not sure I understand. You'd be creating markers every note? ...with the sung note name as the text?
Yes, for learning solfeggio it is great I think, actually together with that doremi.sf2 soundfont somebody posted here in the forum. How are the rules for the black keys? Do they also have any "string definition" when they are half tone higher or lower?

I wrote already a keykit code which can do this automatically some time ago, I can make a new package out of it, combined with AutoHotkey, for comfortable use, if there is interest. Then in one shot you could convert any set of .mid files from "no doremi marker midi files" to "doremi marker midi files".

Nicholas
Posts: 12435

Post by Nicholas » 05-26-10 11:41 am

If you've got a sample file that is encoded that way, I'd like to take a look at it.

Isn't what you're describing better suited to be handled by lyric events in the file instead of markers?

TonE
Synthesia Donor
Posts: 1180

Post by TonE » 05-26-10 5:41 pm

Nicholas wrote:Isn't what you're describing better suited to be handled by lyric events in the file instead of markers?
I can add both, as long as Synthesia supports it. I thought so far Synthesia supports only markers.

I added as an example Czerny 14 with DoReMi added as Lyrics and only Do's added as markers, not to overcrowd the bookmark view in Synthesia. Here I just added a Lyric string for each note, better algorithms should differentiate among important and non-important notes. Important notes can be those having longer durations and higher velocities, but mainly longer durations and higher pitches maybe. Or even considering the onset times and their phases, onbeat onset times and offbeat onset times could be regarded also as more important maybe?

As a summary again:
1. note duration
2. note pitch
3. notes onset time modulo 1/8th note duration == 0
Attachments
DoReMi.current.mid
Czerny 14 DoReMi
(6.08 KiB) Downloaded 222 times

Nicholas
Posts: 12435

Post by Nicholas » 05-26-10 8:02 pm

I see them.

Both lyrics and markers are a little tricky because they're not associated with any note in particular -- just a time point.

In my ancient MIDI editor, that ends up looking like this:
lyrics.png
lyrics.png (4.8 KiB) Viewed 14423 times
Because they're not associated, the falling note view wouldn't be able to do much better.

What you're trying to accomplish is just the Movable-Do version of the existing note labels, right? If you ensure that all the key signature events are correct in the file, shouldn't we be able to generate the voicing automatically and expose it as a new note label mode?

Nicholas
Posts: 12435

Post by Nicholas » 05-27-10 12:30 am

PREVIEW r797 - 0.7.1 Release Candidate 2
Download from the pink box above.

There is a good chance this will become the official 0.7.1 release.

Changes in r797 since r792:
  • IMPORTANT: That DirectX thing still applies. See the first post in this topic!
  • Added a short delay between successive song previews on the Song Library screen. (Previewing too quickly sent device resets faster than some devices were able to handle.)
  • The Song Library will now always advance after pressing Enter -- whether the song is still loading or not. (Well, alright, not if the song ends up loading incorrectly. ;) )
  • Updated 3 of "Harder" songs to have the split left/right hand parts. (Thanks again you guys!) Just Jana-Gana-Mana is left now.
Try to break it if you can.

TonE
Synthesia Donor
Posts: 1180

Post by TonE » 05-27-10 6:53 am

Nicholas wrote:What you're trying to accomplish is just the Movable-Do version of the existing note labels, right? If you ensure that all the key signature events are correct in the file, shouldn't we be able to generate the voicing automatically and expose it as a new note label mode?
I did not understand really what you mean here, sorry for my lack of English understanding.

1. "Movable-Do version"?
2. "generate the voicing"?

If 2. means, finding out where the melody line is, I would say this can be always different, sometimes as top notes, sometimes as middle notes, sometimes as bass notes.

TonE
Synthesia Donor
Posts: 1180

Post by TonE » 05-27-10 8:21 am

Nicholas wrote:Both lyrics and markers are a little tricky because they're not associated with any note in particular -- just a time point.
Yes, but if frost would release one day the fingering algorithm, we can map to each finger a separate midi channel, having at maximum 10 fingers, thus midi channels, at the same time, then adding the lyrics events in each of those midi channels. Then every note would have its unique, own lyric event. I am assuming this for piano songs.

For the finger to midi channel mapping I would suggest

- for the right hand: midi channels 1..5 in reverse order
- for the left hand: midi channels 11..15
(see attached image)

Midi channel 10 would be still be free for potential drums in the song. The rest of the songs arrangement tracks would go then into the midi channels 6,7,8,9,16.
Attachments
CrossedPianoHands.jpg
crossed piano hands and midi channel to finger mapping proposal
CrossedPianoHands.jpg (416.2 KiB) Viewed 14409 times

maccer
Posts: 222
Location: Sweden

Post by maccer » 05-27-10 10:47 am

TonE wrote: For the finger to midi channel mapping I would suggest

- for the right hand: midi channels 1..5 in reverse order
- for the left hand: midi channels 11..15

Midi channel 10 would be still be free for potential drums in the song. The rest of the songs arrangement tracks would go then into the midi channels 6,7,8,9,16.
If MIDI information (and not pure metadata) is going to be used for fingering, it seems a bit wasteful to use one midi channel per finger. Wouldn't it be better to let all fingers use a single channel, for example channel 15? You could then use different notes for the different fingers. C3-E3 for right hand 1-5, C2-E2 for left hand 5-1, see the image.

You could even have a "smart" fingering system that associates each note with a finger: C4 -> finger 1, D4 -> finger 2, E4 -> finger 3, F4 -> finger 1 etc, and those notes would keep the same fingering until told something else by the fingering info. In that case, only half of the fingering information shown in the image would be needed; after you have played the scale upwards, Synthesia would have made all the note-finger associations it needs and would use the same info when playing the scale downwards.
notes_and_fingerings.jpg
notes_and_fingerings.jpg (129.44 KiB) Viewed 14404 times
Songs learned using Synthesia:
CT: Wind Scene, The Trial | FF7: Prelude | SMB: Overworld, Underwater | Tetris: Theme A | Zelda: Lost Woods | Other: Für Elise

TonE
Synthesia Donor
Posts: 1180

Post by TonE » 05-27-10 11:21 am

maccer wrote:...it seems a bit wasteful to use one midi channel per finger.
Not really, as the notes can still stay in their original midi channels, only the created lyrics events would be placed on different midi channels. (Btw. I also do not know exactly if it is allowed to have a lyric event in a midi channel without having there any notes? Some programs might ignore midi channels if there are no notes.)

Using real notes for encoding fingering information would have a disadvantage that you would need to mute those channels, e.g. midi channel 15 as in your example.

maccer
Posts: 222
Location: Sweden

Post by maccer » 05-27-10 12:09 pm

TonE wrote:
maccer wrote:...it seems a bit wasteful to use one midi channel per finger.
Not really, as the notes can still stay in their original midi channels, only the created lyrics events would be placed on different midi channels. (Btw. I also do not know exactly if it is allowed to have a lyric event in a midi channel without having there any notes? Some programs might ignore midi channels if there are no notes.)

Using real notes for encoding fingering information would have a disadvantage that you would need to mute those channels, e.g. midi channel 15 as in your example.
Ah, now I got the part that you suggested using lyric events to store the info instead of notes.
I think it depends on how the fingering editing will be done. If Nicholas introduces an easy "click on a note in the falling notes and type a number" type of system, it doesn't matter where the fingering info lives, but if the community needs to enter in the fingering info themselves using MIDI editors, I believe that note editing is faster and easier to visualize compared to editing lyrics events on 10 different channels.
Songs learned using Synthesia:
CT: Wind Scene, The Trial | FF7: Prelude | SMB: Overworld, Underwater | Tetris: Theme A | Zelda: Lost Woods | Other: Für Elise

Nicholas
Posts: 12435

Post by Nicholas » 05-27-10 1:31 pm

Moving forward, I'm going to try and keep Synthesia out of the business of modifying MIDI files.

I'm kind of a control freak when it comes to my data... and if some viewer/practice app started opening my files and changing them out from under me, I would be unhappy.

So, if an in-game fingering editor is introduced, it'll certainly only be writing out to meta-data.

Of course, if some other built-into-the-MIDI standard exists, there isn't any reason Synthesia couldn't read that too -- like MIDI markers.

(The plan is to have an easy-to-use editor built right in, too. Imagine the bookmark editing interface. It'll be another paused mode where you can scroll around, click notes, and assign fingers.)

As for Movable-Do, this is my understanding:

Fixed-Do: "Do" is always sung for C.
Movable-Do: "Do" is always sung for the root of the current key.

So, if the key signature events are in there properly (i.e., the "C Major" at the bottom of the screen doesn't just say C Major all the time ;) ), we should be able to auto-label notes based on the current scale's root.

Locked