I’ve checked with my Jolla 1 (SFOS 3.4) and it wasn’t quite how I remembered it.
Once there is a media app (Media or Video player) running in SFOS 3.4, whether active (maximized) or not (minimized), whether playing a media or not - the volume button changes media volume. Otherwise it changes ringtone volume.
In SFOS 4 (at least on my phone, a Sony Xperia X), the volume button always changes the ringtone volume, no matter how many media apps are running - unless I’m playing a media, then the button changes media volume.
Yes, I guess it’s a regression bug.