lately I've been doing a lot of research and there are some things I'd like to share.
Let's skip the part listing all the reasons why progressive vmodes are by far the best way to display PS2 games, especially 480p (and to a degree 576p for PAL releases without 60Hz mode). As you all know, there are two outstanding problems which hinder GSM's 480p compatibility, namely the black border issue and the pixelation issue. Both problems are caused by "abnormal" (but frequently applied) frame buffer usage. Here is what we've got so far:
1. Black border issue (enforcing 480p results in a horizontally squished image with aspect ratio approx. 1:1)
- Occurs when the game uses a frame buffer with reduced horizontal resolution, e.g. 512x448 (instead of the full resolution 640x448).
- The horizontal scaling for 480p (1280x480 VCK units as opposed to 2560x480 VCK units for NTSC) doesn't fit because the MAGH value must be an integer.
- Games with a 512x448 frame buffer and a built-in 480p mode (e.g. GoW1&2, RE4, SotC, SSX3) switch to 640x448 when the 480p mode is activated.
- Current solution approaches: HEX-editing the frame buffer size to 640x448, which AFAIK in most cases causes many problems. Or finding a way to somehow get analog control over the scaling. While this would be great, it might be that this isn't possible at all because of hardware restrictions.
2. Pixelation issue (enforcing 480p results in losing half the vertical resolution)
- Occurs with games using the "field rendering" technique, i.e. only half-frames are rendered at a time (with 0.5px vertical offset between them).
- The 480p mode enforced by the Xploder HDTV player doesn't suffer from pixelation as obviously, but from a jittering image.
- Games which do field rendering for 480i and offer a 480p mode as well (e.g. GT4, T5) switch to a full-frame buffer when 480p is activated.
- Current solution approaches: None - 480p not feasible for field rendering games?
In these statements some key details are missing. The most important one is that - since (all?) PS2 games use double buffering - there isn't one "frame buffer", but at least two which need to be distinguished by their function:
This is the draw buffer, i.e. the buffer used for rendering. It contains color, alpha and z-buffer data. The color data is copied to the front-buffer.
This is the buffer used for displaying. It contains only the color data.
When copying the "image" (the color data) from back- to front-buffer, it often gets scaled using the following tool:
This is a GS hardware feature. It can be used for scaling when copying from back- to front-buffer. Both down- and upscaling is possible.
- gives FSAA (full-scene anti-aliasing)
- used to adjust the front-buffer size
When the image arrived at the front-buffer, it's the CRTC's turn:
CRTC (Cathode Ray Tube Controller)
This is the hardware responsible for the video signal. The data is read from the front buffer and converted to a (analog) video signal. The CRTC contains two read circuits which can be set in different ways, depending on the video mode (interlaced, non-interlaced). The CRTC also offers a free (i.e. without computional cost) flicker filter.
Nearly all PS2 games use some sort of flicker filter. This is important here because different kinds of flicker filters imply different frame buffer usages (and vice versa):
A flicker filter is used to reduce the flickering/jaggies that are inherent to all interlaced vmodes (most notably to field rendering games). Some games even have an option to enable/disable their flicker filter, sometimes called "soft/sharp" (e.g. FFXII, T4&5, DT Racer, some Sega Ages games). There are different kinds of flicker filters, which also can be combined:
- CRTC line blending: this is basically a 2-sample blur filter (free)
- GS bilinear filter: 2x vertical FSAA by downsampling a full-height back-buffer to a half-height front-buffer (not free)
- [Motion blur: blending consecutive frames together]
To come back to our two problems (namely the black border and the pixelation issue), let's take a closer look at how PS2 games typically set up their back- and front-buffers:
Problem Nr.1: Back-buffer widths
1. Full width (e.g. 640x448)
- Everything fine here (no black border issue).
2. Reduced width (e.g. 512x448)
- The reduced back-buffer width has been advised by Sony to save VRAM; reducing the back-buffer means not only less color data, but also less alpha and z-buffer.
- For 480i the front-buffer stays 512x448. Horizontal scaling is done with MAGH which saves VRAM and apparently looks better than upscaling with the bilinear filter.
- Games with a built-in 480p mode:
- When 480p is activated proper scaling with MAGH isn't possible, therefore the front-buffer is upscaled to 640x448 with the bilinear filter.
- The real rendering resolution (i.e. the back-buffer) remains 512x448. Changing the back-buffer size would introduce lots of problems!
- Often the color depth is reduced to 16bit to compensate for the increased front-buffer resolution.
- God of War 2 has a 512x448 back- and front-buffer in 480i mode. When activating the 480p mode the front-buffer is increased to 640x448, while the back-buffer stays 512x448. When activating the hidden "HD mode" (hold down L1, L2, L3, Square and Circle during game boot; if performed correctly, the "SCEA Presents" text appears in purple instead of white) the back-buffer is increased to 640x448 as well. While both "480p mode" and "480p HD mode" work without black border issue, the sharper look of the latter proofs the different render resolutions.
- Jak 2 has a 512x416 back-buffer which is upscaled to a 640x448 front-buffer.
- Hitman Bloodmoney doesn't upscale its front-buffer (512x448) - and suffers from the black border issue.
- When activating some game's built-in 480p mode, one effective method to find out if the width of the back-buffer really increased (or if only the front-buffer is upscaled) is to count the "jaggies" in the vertical plane.
Problem Nr.2: Rendering types
1. Full-frame rendering
- Characterized by a full-size front- and back-buffer (e.g. 640x448)
- Fully 480p compatible!
2. Field rendering
- Characterized by a half-height front-buffer (e.g. 640x224) containing one field (i.e. a half-frame) with 0.5px vertical offset
- The game must run at constant 60fps (fields per second), slowdowns result in very noticeable pixelation.
- Field rendering effectively is a trick to achieve the full resolution while only rendering at half resolution.
- Use of field rendering has been advised to the developers by early SDK versions. It has been designed specifically for interlaced displays and does not work well for progressive displays, since consecutive fields are rendered at a different points in time and hence don't fit together properly when there is movement (result: sawtooth effect).
2.a) Half-height back-buffer (e.g. 640x224)
- Back- and front-buffer change roles every 1/60 second ("page flipping")
- A flicker filter is not possible in this case, which results in severe interlace artifacts.
- True 480p is probably not feasible (changing rendering resolution causes lots of problems)!
- Only a few games used a half-height back-buffer (mainly Japanese launch titles).
- Examples: Virtua Fighter 4, Ridge Racer 5, Dead or Alive 2
2.b) Full-size back-buffer (e.g. 640x448)
- The game renders a 640x448 back-buffer which is downsampled to a 640x224 front-buffer (flicker filter).
- While incompatible with GSM, Xploder and ps2force480p, it is quite possible to enforce 480p! See the next post!
- The majority of field rendering games actually render a full-size back-buffer.
- Examples: Virtua Fighter 4 Evolution, Soul Calibur 2 (480i mode), Jak & Daxter, Gran Turismo 3 (apparently some of the effects like smoke are rendered half-height), Devil May Cry series. The Snowblind Engine games, for example Baldurs Gate Dark Alliance, are a special case. They use a 1280x448 back-buffer downsampled to a 640x224 front-buffer, which gives 2x horizontal FSAA besides the flicker filter).
That's it so far... keep in mind that the info posted above is by no means guaranteed to be true!
If you find any errors or know something that should be added, let me know.