Variable tempo with expression pedal

Synthesia is a living project. You can help by sharing your ideas.
Search the forum before posting your idea. :D

No explicit, hateful, or hurtful language. Nothing illegal.
Post Reply
diggidoyo
Posts: 9

Post by diggidoyo »

Thanks for all the fantastic development that's gone into Synthesia!
I'm not sure if this one made to it to the task list already.

When playing a song in rhythm or recite mode, it'd often be incredibly useful to have dynamic control over the tempo with an expression pedal.
This would allow us to slow down or even pause the song using a variable controller without having to use our hands!

A clear example of where this could be useful would be for a fermata.
Synthesia eats fermatas for breakfast, maybe dropping the tempo by 1/3 for that note before trucking on a tempo ad infinitum.
With an expression pedal bound to the tempo, we could push it all the way down for 0% speed, and release only we are ready to proceed.
This would also let us throttle the tempo with our foot to keep the music in pace with our actual performance.

It can often be pleasant to draw out certain phrases or linger on spicy chords for dramatic effect, but so far the only hands-free method to keep the MIDI playback in line has been to bind the step backward and step forward to my soft and sostenuto pedals. This at least puts the music back where I need it, but it doesn't allow for direct control over the tempo.
The shortcuts do provide options to speed up and slow down the tempo in discrete steps, but I haven't yet found a practical method of using this feature for precise fine-tuning of the tempo on the fly.

No pedal: 100% speed
Full pedal: 0% speed

Feels like this feature could be fun to play around with for dynamic interpretations as well as for practicing complex sections.
Any chance of seeing something like this in Synthesia's future?
Peyton15
Posts: 1

Post by Peyton15 »

Hello,

It's true that the idea is pretty good. We'll have to see if it comes true in the future.
My website : Botnation
Nicholas
Posts: 13132

Post by Nicholas »

diggidoyo wrote: 10-15-22 2:59 amSynthesia eats fermatas for breakfast...
:lol:
diggidoyo wrote: 10-15-22 2:59 amWith an expression pedal bound to the tempo, we could push it all the way down for 0% speed, and release only we are ready to proceed.
That does sound very cool.

There is one other hands-free method I can think of: Score Following.

That would make it completely interaction-free and you could simply play at whatever speed you like at any time and Synthesia would follow your lead.

That's a bigger feature that would take more work, though. And yours seems very straightforward to implement and test in an afternoon, so I wouldn't want to dismiss it out of hand. Still, there is a certain sensation of one requiring the human to be trained to work around the computer's preferences and the other requiring the computer to be trained to follow the human's preferences. Getting used to using an expression pedal to adjust the speed of the music is a skill that transfers to exactly zero other situations... even if it does solve the problem here. Hmm.
diggidoyo
Posts: 9

Post by diggidoyo »

Funny enough, I shared your same reservations regarding the non-transferability of the skill at first, but was ultimately convinced by the fact that this feature is really only intended to address the source of the true issue:
With sheet music, we can innately speed up and slow down the tempo by just reading faster or slower.

Sheet music is an incredibly useful standard that's been used for centuries to communicate information across the broad spectrum of instruments.
The falling notes in Synthesia... is essentially just sheet music flipped on it's side and reversed, then presented to us like we're running down the mirrored staves in a kind of minimalist Beatsaber/Temple Run. This dimensional translation is performed in our heads every time we read sheet music to play piano, but Synthesia does this all for us which results in a display that more closely matches the actual physical configuration of the instrument we are playing.

But this assistance comes with a price.

Our view of the score is confined to a tiny window that moves along at a pace we have little control over.
As a result of this tunnel vision, tempo control and reading ahead are examples of skills that are currently non-transferable into Synthesia.

The expression pedal is the most practical method I could think of to emulate some level of the missing tempo control.
This advanced feature would certainly involve a learning curve to use properly and would be unique to Synthesia, but perhaps this could be seen as a benefit and competitive advantage?

I also considered using a controller like a pitch wheel.
Binding the wheel from 0% to 200% with 100% in the middle could be a fun DJ tool all on it's own. Maybe there's even a negative tempo setting we can use to play the song backward and forwards? Could be lots of options for the tempo wheel, but we need our hands to operate it effectively. To go hands free, we really need these controls on the floor.

The most basic configuration of the tempo pedal from 0-100% seems like something Synthesia is almost already capable of doing, I will just need to work on some macros to try to get the shortcuts to provide the intended effect. If I'm ever able to achieve this kind of control over the tempo, then the next step would be to try to add another pedal for further control over the 100%-300%+ speed range. Speeding up the song can also be useful at times and having full manual control of the tempo whenever it's needed would simply help us calibrate with the MIDI playback. Our left foot needs something more to do anyways, so giving it a couple tempo pedals might make it feel more included.


Score following would definitely be the ideal solution that solves most of these problems for the widest audience.
Thank you for considering this as a stop gap until then.
revilo2
Posts: 135

Post by revilo2 »

Not sure to have understand :

Expression pedal is sustain pedal ?

In this case, if used to modity the tempo, there will be no sustain when playing ?


:?
Nicholas
Posts: 13132

Post by Nicholas »

diggidoyo wrote: 10-21-22 1:33 amThe most basic configuration of the tempo pedal from 0-100% seems like something Synthesia is almost already capable of doing...
It's already built in, today.

On the shortcuts screen there is a "Speed (variable)" shortcut. The default binding is Ctrl+Mouse Wheel, but there is some special handling depending on the type of "axis" of the control you bind it to. A mouse wheel is what I call a "relative axis" internally: there's no beginning or end, it can spin forever in either direction, so the only input you get from it is how much it has changed since the last input event. Other types of axes like pitch bend wheels or CC knobs on MIDI controllers provide "absolute" data where you know the min/max values. (I.e., the default position of the pitch bend wheel is at the 50% position because that's where the springs hold it and input always comes in somewhere between 0 and 100% so you know exactly where the wheel is.)

On the play screen inside Synthesia, whenever that shortcut is triggered, it checks which kind of axis is associated with whatever triggered it. For relative axes (like the default binding), it just increments or decrements the value, because that's all it can do.

For absolute axes, it sets the song speed to that value (I think on a scale from 0% to 200% speed).

It looks like I even went so far as to include a small 100%-speed dead-zone (between 48% and 52%) for loose pitch-wheels. (This was added many years ago and I actually forgot that part was in there.) :lol:

If you can find an expression pedal that delivers its input as MIDI "CC" control messages, that should work to control the song's speed right out of the box. You may need to re-map its range to avoid having to hold it right in the middle for 100% using a third-party MIDI app like MIDI-OX and a virtual port like LoopBe1 or loopMIDI... or, it looks like some of them (this one?) have a hardware adjustment built in so you can just turn a knob to control the range. Combined with something(?) that can convert the analog pedal signal into a series of MIDI messages (any MIDI controller with an 1/4" expression input) and that should allow a 0-100% pedal with completely off-the-shelf hardware and software, today. (I may pick one of those pedals up to experiment with it.) :D
Bavi_H
Posts: 115

Post by Bavi_H »

revilo2 wrote: 10-21-22 7:34 amNot sure to have understand : Expression pedal is sustain pedal ?
A sustain pedal usually only works as an "off or on" switch.

An expression pedal has a see-saw shape. You can use your foot to adjust the see-saw from completely down at the bottom to completely down at the top.

Some MIDI keyboards have both a jack for a sustain pedal and a jack for an expression pedal.

The sustain pedal will usually cause a Sustain (CC64) message to be sent with value 0 for off or value 127 for on.

The expression pedal will usually cause a Volume (CC7) or an Expression (CC11) message to be sent with any value from 0 to 127.

Image: three kinds of pedals
Left: An expression pedal has a see-saw shape. Middle and right: A sustain pedal is usually just an "off or on" switch.
Last edited by Bavi_H on 11-07-22 5:15 am, edited 2 times in total.
diggidoyo
Posts: 9

Post by diggidoyo »

Nicholas wrote: 10-21-22 2:05 pm If you can find an expression pedal that delivers its input as MIDI "CC" control messages, that should work to control the song's speed right out of the box.
I've tried playing around with the Speed (variable) setting but couldn't get it to work properly.
The mouse wheel works fine as the shortcut, but when I set it to my auxiliary pedal, it registers as "CC Axis 64 ch1" and any amount of input on the pedal instantly sets the tempo to 400%. The soft and sostenuto pedals also set the tempo straight to 400%.

I do actually get some control when I set the variable speed shortcut to the sustain pedal, which registers as "sustain + CC Axis 64 ch1".
This gives it an interesting gas pedal effect, with the tempo going from 400% with the pedal fully down and 0% when it's up.
That's almost there already! I just wasn't sure if my pedals were the problem and I couldn't find a way to change the variables.
Good to know all I need is the actual expression pedal and the MIDI adapter, I just ordered them both now and will test it out later.
Thanks for your insight!
Bavi_H
Posts: 115

Post by Bavi_H »

diggidoyo wrote: 10-21-22 5:57 pmI've tried playing around with the Speed (variable) setting but couldn't get it to work properly. [...] when I set it to my pedal "CC Axis 64 ch1", any amount of input on the pedal instantly sets the tempo to 400%. [...] I've tried using a Roland DP-10 set to continuous and a Triple Pedal both plugged into a Yamaha P515.
It looks like Synthesia lets you use the Sustain switch as a shift key when setting a shortcut. In other words, you can assign a shortcut to be pressing down Sustain switch plus moving some other control or button. This shift key ability seems to prevent Synthesia from using the Sustain message (CC64) as a continuous controller in a normal way.

See if your Yamaha P-515 can change the type of message that is generated when you press your continuous pedal. Using another CC message besides CC64 seems to work better with Synthesia.

Also, if you do not need to use any buttons on your MIDI device for shortcuts, consider unchecking the advanced setting "System.IgnoreZeroValuedCc" described below. This will let you change the tempo down to 0%.

However, as you have already discovered, the "Speed (variable)" shortcut maps the minimum value to the slowest speed, and the maximum value to the fastest speed. You wanted the reverse mapping with half the range: map the zero value to 100% speed, and the maximum value to 0% speed.


Testing CC messages as a "Speed (variable)" shortcut

I tested sending Control Change messages into Synthesia 10.8.5686 or preview-10.9.5850 on Windows 8.1 with Pocket MIDI and virtual MIDI cable loopMIDI. I saw strange behavior when trying to use CC64 messages for the "Speed (variable)" shortcut.
  • In the window to add a new shortcut for "Speed (variable)", while I twiddle the CC64 value knob, sometimes the shortcut shows up as "Sustain + CC Axis 64, ch1" and sometimes it shows up as just "CC Axis 64, ch1".
  • When the "Speed (variable)" shortcut is "CC Axis 64, ch1", then during song playback:
    • If I send CC64 value 0 then value 1, the tempo changes to 1%.
    • If I send CC64 value 0 then value 127, the tempo changes to 400%.
    • Any other CC64 values have no effect on the tempo.
  • When the "Speed (variable)" shortcut is "Sustain + CC Axis 64, ch1", then during song playback:
    • Changing CC64 values smoothly from 0 to 127 causes tempo changes from 1% to 400%.
    • Toggling the CC64 value immediately between 0 and 127 doesn't change the tempo.

If I set the "Speed (variable)" shortcut to another CC message like CC4 ("CC Axis 4, ch1"), then during song playback:
  • If the advanced setting "System.IgnoreZeroValuedCc" is checked, CC4 values from 1 to 127 change the tempo from 1% to 400%. CC4 value 0 has no effect.
  • If the advanced setting "System.IgnoreZeroValuedCc" is unchecked, CC4 values from 0 to 127 change the tempo from 0% to 400%.

Advanced setting "System.IgnoreZeroValuedCc"

In Synthesia's Shortcut settings, most of the items are for "button press" shortcuts. But there are a few items that can react to continuous values from a slider, wheel, knob, or continuous pedal: The items for "Speed (variable)", "Song Position Jog", and "Absolute Song Position" have a comment that says "This shortcut works with an axis like a pitch bend wheel, a control knob, or mouse scroll wheel."

There is an advanced setting that can affect how MIDI message shortcuts work. Hold the Shift key as you start Synthesia to get the Synthesia Configuration Window. Find the setting called "System.IgnoreZeroValuedCc".


Check the "System.IgnoreZeroValuedCc" setting if you want to:
  • Map a shortcut to use a button or footswitch that sends a Control Change message with a value of 127 when you press it and value of 0 when you release it.
However, when this setting is checked:
  • If you map a shortcut to use a button or footswitch that sends a Control Change message with a value of 127 the first time you press it, and sends a value of 0 the second time you press it, the shortcut only activates every other time you press the button or switch.
  • If you map one of the continuous value shortcuts to use a controller that sends a Control Change message with continuous values from 0 to 127, when a 0 value is sent it does not activate the shortcut.

Uncheck the "System.IgnoreZeroValuedCc" setting if you want to:
  • Map one of the continuous value shortcuts to use a controller that sends a Control Change message with continuous values from 0 to 127, and you want the 0 value to activate the shortcut.
  • Map a shortcut to use a button or footswitch that sends a Control Change message with a value of 127 the first time you press it, and sends a value of 0 the second time you press it, and you want to activate the shortcut every time you press the button or switch.
However, when this setting is unchecked:
  • If you map a shortcut to use a button or footswitch (other than CC64) that sends a Control Change message with a value of 127 when you press it and value of 0 when you release it, the shortcut will active twice each time you press the button or switch. (The shortcut will activate when you press down, and then activate again when you release.)
    • Except, if you map a shortcut to use a button or footswitch that sends CC64, the shortcut will activate only one time time each press. (I guess CC64 works differently because Synthesia is always monitoring CC64 for potential use as a "shift key"?)
diggidoyo
Posts: 9

Post by diggidoyo »

Bavi_H wrote: 10-22-22 12:22 am See if your Yamaha P-515 can change the type of message that is generated when you press your continuous pedal. Using another CC message besides CC64 seems to work better with Synthesia.
Ah, thank you!
Yes, the auxiliary pedal was set by default to operate as a sustain pedal, but the system settings on the P-515 lets you change the output to "Pitch Bend Up" or "Pitch Bend Down"!

I set the output for this expression pedal to Pitch Bend Down and reversed the polarity from Make to Break. It now works exactly as described, with full tempo 100% speed without input and 0% speed when fully pressed! The Pitch Bend Up setting even goes from 100%-400%!
Now I just need the MIDI adapter to decouple this function from the pitch bend controls, but it's good to know the feature works with the right hardware.
Thanks again for the help!
Nicholas
Posts: 13132

Post by Nicholas »

I grabbed that M-Audio pedal (I already had a MIDI controller with an expression pedal port I'd never used before), and gave this a try.

My first impressions are that it's a lot of fun. :lol:

Probably the only thing missing (in general) from the "axis"-style shortcut binding is a little more in the way of flexibility. I could see adding a kind of calibration feature to specify how the input range should be mapped to the output range. (In this case, 400% speed isn't very useful, so it'd be nice to be able to say something like "I want the top of the pedal to mean 120% speed", etc.)
Bavi_H wrote: 10-22-22 12:22 amAlso, if you do not need to use any buttons on your MIDI device for shortcuts, consider unchecking the advanced setting "System.IgnoreZeroValuedCc" described below. This will let you change the tempo down to 0%.
I wonder if that setting should be ignored in the case of axis-based controls (vs. single-event triggers)? For those it feels natural to always read zero as a valid input. Would you agree?
Bavi_H wrote: 10-22-22 12:22 amI guess CC64 works differently because Synthesia is always monitoring CC64 for potential use as a "shift key"?
This is correct for CC 64, 66, and 67. They are treated as special cases, ignoring some of the other processing, so they can also behave as modifier keys for shortcuts.
Bavi_H
Posts: 115

Post by Bavi_H »

Nicholas wrote: 10-26-22 10:57 amI wonder if [the "System.IgnoreZeroValuedCc"] setting should be ignored in the case of axis-based controls (vs. single-event triggers)? For those it feels natural to always read zero as a valid input. Would you agree?
It sounds okay, but I can't tell if could cause any unintended effects in some cases. I started thinking of other ideas for improvements, but held back on mentioning lots of ideas for something I don't personally use.

For example, instead of a fixed "System.IgnoreZeroValuedCc" setting to ignore CC values of 0, when assigning a shortcut, while the window is waiting for input, if it only sees CC value 127 then 0, it could assume you pressed a button or foot switch and decide a CC 0 value shouldn't trigger the shortcut. If it sees other CC values, it could assume you touched a slider/knob/wheel/continuous pedal and allow the CC 0 value to trigger the shortcut. But I couldn't tell if those ideas could have any unintended effects in some cases. And what should it do if it sees a lone 127 value or a lone 0 value? This might happen if you press a toggle button -- a button that sends 127 on the first press and 0 on the second press. I started getting carried away wondering about ways to detect and handle toggle buttons, or mapping custom ranges and reverse directions, and so on.
Nicholas
Posts: 13132

Post by Nicholas »

Bavi_H wrote: 10-26-22 7:23 pmI started getting carried away wondering about ways to detect and handle toggle buttons, or mapping custom ranges and reverse directions, and so on.
I know how easy it is to get carried away with those sorts of ideas! My paragraph above about flexibility and mapping the input and output ranges is already starting to get there. At some point I'm going to have accidentally reimplemented all of MIDI-OX across Synthesia's various menus. :lol:

I just tried out zero-valued CC's still counting as "axis" data and it feels right. Now you can drop the song speed to 0% with a pedal or knob or whatever, but you still don't get double-triggering on regular shortcuts. If there's a case I missed, I'll cross that bridge when we come to it. ;)
Post Reply