Dot-pattern averaging: Hacks to turn this on?

So I found my old source code, and I didn't apply a non-linear adjust to Y, in order to match the real system and the patent entries. I add a slight bias for rounding and I scale Y by a small amount. But it's a constant, so it's linear. So the stepping pattern doesn't appear to be some gamma based side effect, but more of just the way they did rounding/turnication.

//R/G/B input are 3bit values. Not scaled to 8bit
y=(1.33*r)+(2.61*g)+(0.49*b);
v=(2.14*r)-(1.8*g)-(0.34*b);
u=(-0.72*r)-(1.42*g)+(2.14*b);

//U and V are not 8bit values. Offset by 16 instead of 128
u+=16;
v+=16;

//for rounding/turnication
y+=0.5;
u+=0.5;
v+=0.5;

// round y/u/v to remove decimal parts. Convert to int and back again to double.


//scale to 8bit range
y=y*8;
u=u*8;
v=v*8;

//scale Y to get stepping
y*=1.134;

And then I convert back to 24bit RGB, and clamp accordingly for range. Mednafen has an option to allow custom RGB maps for PCE. I loaded it and ran Ccovell's 512 color demo. It had the same stepping as the capture card test of his demo on the real machine. But, that was just for Y. I didn't figure this out for U/V.

A pic of captured video from the real system (I/Q traces are cut, so only Y/sync/CB are output): http://www.pcedev.net/pcetech/vce/luma_scale_capture.png