doctorxyz, what were the unexpected values returned by strtol?
lee4, a MAGH of 1 is not a good idea. That's essentially telling the read circuit to read 2 lines worth of pixels per line, and that's only if the original framebuffer is 640 pixels wide. I'm not actually sure what would be the effect of doing that, though. The width can be anywhere from 256,320,384,512,640 giving you MAGH values of 9,7,6,4, and 3, respectively. The lower the resolution of the framebuffer, the more it needs magnification. It looks like doctorxyz has it right, though it's limited only to a single supported width of 640.
Games tend to use a 512x512, 640x448, and 640x512 or 512x256x2, 640x224x2, and 640x256x2. It depends on the interlacing mode. For all the NTSC/PAL modes, conversion to/from and interlacing on/off, the magh value should be left alone, since the screen's width or framebuffer width isn't changed when changing any of those options.
I've seen some posts concerned with the problem with having less height when converting a game from NTSC to PAL. For the DH value, increasing the height from NTSC to PAL height won't help with some games, as they set the scissoring test area to the NTSC's dimensions. The pixel coordinates are tested to see if they are within the dimensions, and pixels aren't processed further if they're not. The only way to increase the dimensions is to send a dma packet to correct the values, but it'd require knowing when the register was written. The best option is probably just to center it so it's letterboxed using the DY value.