Sega Saturn Model 2 - Ever notice Hard Panning of L-R Sounds?

Been meaning to post about this for a bit just never got around to it.

I notice when playing some games (for example Street Fighter Zero 3 or Radiant Silvergun) that my sound is very hard panned left or right. Not that I would say sound is dropping out or staticky at all. It almost sounds like the left sometimes is louder than the right by a significant margin AND vice versa (in other words it's not always the left that is louder). I can play SFA3 on PS1/2 on the same setup and I do not run into the same finding.

My Saturn setup consists of a Model 2 with phantom universal mod chip installed (not that it should have any affect on this). Where it gets complicated is that I have it hooked up through an XRGB Mini and the sound passes from the Framemeister HDMI into my LCD TV then from there into the TV's optical out into a dedicated stereo DAC then into a stereo receiver pushing two big floorstanders. I've tried a different DAC and the effect is the same.

Is this normal for this kind of setup? I'm wondering if maybe the problem is between the Saturn's horrible a/v out jack and the SCART cable (ps2 mentioned above btw is not SCART but my SNES is SCART and it sounds fine). I've unplugged and replugged it in a few times, same issue so if it is indeed the cable maybe it is internal. If it's a Saturn internal cap issue wouldn't the problem be more along the lines of no sound or crackly static?

I want to say BTW that it's harder to hear the hard left or right panning if using my TV built in speakers for the audio rather than the external stereo setup. I need to do more testing to verify this one though.

Thanks in advance!