Source code of USB ADVANCE Loader for PS3

[ifcaro]

Hi, I have published the source code of "USB Advance Loader for PS3" under the GPLv2 license.

I hope so other developers can improve the compatibility if that is possible.

Also remember that any improvements also works for the PS2.

The source code is in http://code.google.com/p/usbld/

================================================
INSTRUCTIONS FOR COMPILING USB ADVANCE LOADER
================================================

To compile the USB Advance Loader you need to make some previous actions.

First you will need a copy of USB ADVANCE unpacked completely (3MB)

To "enable" loader's patches is necessary to edit the original USB ADVANCE's elf that
once implemented it will return control to the loader for adding any improvements.

For this we need to replace a jump which is located in 0x000016e8 offset by a
JAL to offset the "parches" function of the loader.

I programmed a little utility to patch the executable knowing the offset of the function
'parches'. You can download it at: http://ps2dev.ifcaro.net/archivos/USBADVN_PATCHER.zip

To know which is the offset of the function patches follow one of these methods:

* Search with PS2DIS in our elf once compiled (ee-strip disabled in makefile) the function
'_Z7parchesv' or similar and look at offset (0x00800298 in my case)

* Run in PCSX2 (EE/IOP recompiler off) and looking at the text log
"Patches injection at 0x00800298"

* Comment call to Reset() function in main and use ps2link to search in the log the string
"Patches injection at 0x00800298"

Now USB Advance will give the step to Loader to add new features.

Finally copy the USB ADVANCE elf renamed to USBADVN_ok.ELF to the emb folder

NOTE: In the source code appears the name of some modules that I have extracted from the
USB ADVANCE elf. These modules gave them the name that occurred to me at that time.

These modules are:

* PARCHE_EE: It is not a module but it is a piece of code for EE is loaded just before
to launch the game. (This patch is copied to the memory address 0x000A8000)

* MSD.IRX: (MASS-DRIVER) This is the module responsible for the manege USB disk filesystem.

* RAMDISK.IRX: Its a RAM DISK for IOP updating with USB ADVANCE modules.

* CDVDMAN.IRX: It is a replacement driver CDVDMAN to read from USB.

* USBLOADER.IRX: IOP part of USB ADVANCE.

[_zaphod_]

Interesting.

[dlanor]

Hi, I have published the source code of "USB Advance Loader for PS3" under the GPLv2 license.

Not to 'rain on your parade', but is any homebrewer really entitled to 'license' this stuff ?
If so, then it must be a brand new creation, not in any way based on the commercial product.

I hope so other developers can improve the compatibility if that is possible.

I'm sure several would be interested in trying, myself included.

Also remember that any improvements also works for the PS2.

That is of course a prerequisite for my own interest.
(I have no PS3 and zero interest in getting one.)

The source code is in http://code.google.com/p/usbld/

Yep, apparently it is, but it's not available for normal download.

The only form of access I was allowed was to browse the source tree with my web browser, which is completely useless. I am not going to download an entire source package by iterating the Firefox "Save page as..." command.

Why don't you just post a normal ZIP attachment instead, to make things a little bit easier.
Or if you insist on using SVN methods only, then provide us with a directly useful SVN link.
(Preferably allowing read access also for 'non-members'.)

Best regards: dlanor

[ifcaro]

Here is a zip with the source from SVN: http://usbld.googlecode.com/files/sv...t_26-04-09.zip

Google code page says:

# Non-members may check out a read-only working copy anonymously over HTTP.
svn checkout http://usbld.googlecode.com/svn/trunk/ usbld-read-only

[dlanor]

Here is a zip with the source from SVN: http://usbld.googlecode.com/files/sv...t_26-04-09.zip

Thanks. This is much more convenient for everyone.

Google code page says:

# Non-members may check out a read-only working copy anonymously over HTTP.
svn checkout http://usbld.googlecode.com/svn/trunk/ usbld-read-only

Yes, I know. But the only way of 'checkout' through that link is to visually check the source out on HTML pages, with the added feature of being able to use "Save link as..." to save the 'raw' files (ie: the ones you posted in the ZIP) one by one, which is a very tedious process. There is no support on those pages for making a full package download. (At least not any that I could find.)

But enough about that, as the real package is available here now, which is what counts.

I have already downloaded some of the files (by the tedious method), and immediately spotted some major problems for most people who'll want to work on this.

The primary problem is that you assume the presence of a USBAdvance file in a form that most people don't have access to. (That unpacked 3MB file)

The closest most of us can get to it is with the 'clean' (but still packed) USBADVN.ELF for use with the USBAStrip skinning program. But there is no easy way to unpack that USBADVN.ELF file, since ps2-unpacker simply crashes on an attempt to do so.

Possibly some special version of it or another program is needed, possibly with special arguments too, and unless you simply tell us how it is to be done each of us may have to waste a lot of time on finding that out.

The secondary problem is the assumption that everyone can easily produce those five files that you extracted from embedding in the large elf, which would be a lot easier if you gave the correct start address and length of each such file section, for use with a hex editor like WinHex or similar.

At this point I'm not even sure how or if those are needed as separate files for this source project though, as I can't find them in the Makefile... They seem to be referred only in "parches.cpp". But I think they really need to be explicitly extracted for best results.


In the long term the greatest improvements can probably be achieved after analysis and modification of those embedded files. The CDVDMAN replacement is obviously crucial to all game compatibility, since this is one of the parts that must remain resident during the gameplay, and even enforced for continued residence when the game attempts to reload drivers. Another crucial part is of course that RAMDISK file, which I assume serves as a replacement for IOPRPxxx.IMG or DNASxxx.IMG files of a game. So this too may be crucial for improved game compatibility.

And of course the basic USB device compatibility will be mostly affected by the USBD and USBHDFSD modules placed in the 'emb' folder for embedding.

You never did mention the USBD or USBHDFSD (MSD.IRX ?) equivalents as embedded in the existing elf (again that 3MB file), but I assume that you somehow overwrite those by re-embedding the new ones provided in the 'emb' folder as mentioned above.

Best regards: dlanor

[ifcaro]

The primary problem is that you assume the presence of a USBAdvance file in a form that most people don't have access to. (That unpacked 3MB file)

The closest most of us can get to it is with the 'clean' (but still packed) USBADVN.ELF for use with the USBAStrip skinning program. But there is no easy way to unpack that USBADVN.ELF file, since ps2-unpacker simply crashes on an attempt to do so.

Possibly some special version of it or another program is needed, possibly with special arguments too, and unless you simply tell us how it is to be done each of us may have to waste a lot of time on finding that out.

I decompressed USBADVN.ELF with USBAstrip using:

USBAstrip /x  USBADVN.ELF USBADVN_ok.ELF

The secondary problem is the assumption that everyone can easily produce those five files that you extracted from embedding in the large elf, which would be a lot easier if you gave the correct start address and length of each such file section, for use with a hex editor like WinHex or similar.

At this point I'm not even sure how or if those are needed as separate files for this source project though, as I can't find them in the Makefile... They seem to be referred only in "parches.cpp". But I think they really need to be explicitly extracted for best results.

This source code does not use any of the embedded modules from usb advance elf. only replaces the usbd.irx, when it will be loaded.

Previously, I used it the module MSD.irx for the option of importing usbd and language file to memory card. But then I've replaced by usbhdfsd to avoid posting copyrighted code.

Yet, usbhdfsd is only used to import files to the memory card and is not used for usb advance

In the long term the greatest improvements can probably be achieved after analysis and modification of those embedded files. The CDVDMAN replacement is obviously crucial to all game compatibility, since this is one of the parts that must remain resident during the gameplay, and even enforced for continued residence when the game attempts to reload drivers. Another crucial part is of course that RAMDISK file, which I assume serves as a replacement for IOPRPxxx.IMG or DNASxxx.IMG files of a game. So this too may be crucial for improved game compatibility.

These modules have not replaced, just dumped and tried to see their operation. if you want, I can upload them in megaupload or similar

I've been working some time studying how USB ADVANCE works and I saw that the EE patch "parche_EE" makes a hook in loadexecps2 to load USB ADVANCE modules.

RAMDISK is similar to ps2img module. Its called this by way "RD0:001BD800.0001AE40"

I think it means.

RD0 = RAMDISK0 (virtual drive)
001BD800 = IOP memory offset where is custom ioprp
0001AE40 = size of ioprp image

I'm not sure but I believe that this works in this way

And of course the basic USB device compatibility will be mostly affected by the USBD and USBHDFSD modules placed in the 'emb' folder for embedding.

You never did mention the USBD or USBHDFSD (MSD.IRX ?) equivalents as embedded in the existing elf (again that 3MB file), but I assume that you somehow overwrite those by re-embedding the new ones provided in the 'emb' folder as mentioned above.

USBD.irx is replaced with an old free version that works also in PS3 (newer versions do not work) This version came with Launchelf 3.41 I think

MSD.irx can not be replaced by usbhdfsd because it seems that MSD.irx has special features that usbhdfsd haven't. It would be good to reverse these functions and use usbhdfsd but I have not managed to know, what they do these functions.

forgiveness for my bad English I have been helped by google translator

[dlanor]

I decompressed USBADVN.ELF with USBAstrip using:

USBAstrip /x  USBADVN.ELF USBADVN_ok.ELF

I see. The funny thing is that this is not what USBAStrip itself says that this option will do, when you run the program without arguments. The help text you then get only states that the '/x' option is to be used to extract various picture files, which is obviously wrong, since no such files are produced by that command.

I did what you said and got the 3MB file, so I'm all set now.

Yet, usbhdfsd is only used to import files to the memory card and is not used for usb advance

I don't get this. For writing on a memory card usbhdfsd is not needed, but only when you need to access something on USB, like when copying a file to MC from USB, but that is an operation which USBAdvance does not support as far as I know. Or does it ?

These modules have not replaced, just dumped and tried to see their operation. if you want, I can upload them in megaupload or similar

That would be nice, though a simple list of their start and end addresses (or length) in the 3MB elf file will also do fine. It is then quick work with WinHex to extract each separate file.

I've been working some time studying how USB ADVANCE works and I saw that the EE patch "parche_EE" makes a hook in loadexecps2 to load USB ADVANCE modules.

Yes, that makes sense. This would be part of the methods used to interfere with the games' attempts to reinit IOP with the modules from the game disc.

RAMDISK is similar to ps2img module. Its called this by way "RD0:001BD800.0001AE40"

I think it means.

RD0 = RAMDISK0 (virtual drive)
001BD800 = IOP memory offset where is custom ioprp
0001AE40 = size of ioprp image

I'm not sure but I believe that this works in this way

This could well be right, and should be possible to verify in part by checking the routine that loads this module.

USBD.irx is replaced with an old free version that works also in PS3 (newer versions do not work) This version came with Launchelf 3.41 I think

Even so, it is unthinkable to limit a future improved version to those USB drives which work with such obsolete driver versions. It would be better to make it load an external driver file instead, so that the driver can be replaced to suit the user.

Another approach would also be to make modifications to the current USBD driver so as to make it compatible with a PS3 too, though I have no idea what it is that makes the present version incompatible. (I'm a developer for PS2, not PS3...)

Either way, PS2 users will insist on a newer USBD.IRX, since the old ones fail to work with lots of USB devices.

MSD.irx can not be replaced by usbhdfsd because it seems that MSD.irx has special features that usbhdfsd haven't. It would be good to reverse these functions and use usbhdfsd but I have not managed to know, what they do these functions.

Well, maybe later then. The chances are better once more people have access to this stuff.

forgiveness for my bad English I have been helped by google translator

No need to apologize. I had no problem following your meaning in this post, though a few technical details still seem a bit vague to me, but that is not due to language problems.

Best regards: dlanor

[ifcaro]

I don't get this. For writing on a memory card usbhdfsd is not needed, but only when you need to access something on USB, like when copying a file to MC from USB, but that is an operation which USBAdvance does not support as far as I know. Or does it ?

[...]

Even so, it is unthinkable to limit a future improved version to those USB drives which work with such obsolete driver versions. It would be better to make it load an external driver file instead, so that the driver can be replaced to suit the user.

[...]

Either way, PS2 users will insist on a newer USBD.IRX, since the old ones fail to work with lots of USB devices.

This program has a configuration menu where you can copy an usbd.irx from the USB drive to the MC for using instead of old usbd embbeded. This was done for the people who use the drivers from Sony in the PS3.

You can also change the language and lock the format button in this menu. In this video you can see these new options.

[youtube]tnsqOwLTYII[/youtube]

That would be nice, though a simple list of their start and end addresses (or length) in the 3MB elf file will also do fine. It is then quick work with WinHex to extract each separate file.

Is not so easy. they have an compression or encryption, and can not be extracted in RAW. I dumped them from memory just after been extracted. That's why I told you upload, but I do not know if I can put the link here.

[dlanor]

This program has a configuration menu where you can copy an usbd.irx from the USB drive to the MC for using instead of old usbd embbeded. This was done for the people who use the drivers from Sony in the PS3.

Yes, I understand better how the loader is to be used now, having had time to look into the source a bit more than earlier. There are still a few things that escape me though, and I'll add some questions about those things at the end of this post.

As for the USBD selection, this can of course be used for more than one purpose, including the choice of the latest homebrew drivers that PS2 users will want, so as to gain compatibility with more USB devices (eg: the old 1-4 GB Kingston Data Travelers which never work with the old USBAdvance driver.)

----- re: posting embedded files

Is not so easy. they have an compression or encryption, and can not be extracted in RAW. I dumped them from memory just after been extracted. That's why I told you upload, but I do not know if I can put the link here.

Well, I'm not sure, but I think it should be OK. USBAdvance elfs have been considered 'fair game' attachment-wise in the past, so any files extracted from them should also be OK, as long as they are not affected by Sony copyrights. (Hardly the case for USBAdvance )

If they are embedded in encrypted form, even in the unpacked elf, then it can be tricky to modify them of course. It would be necessary to either re-encrypt them after the modifications, or to change those parts of the main elf that load and decrypt those files, so as to make them simply read the raw files instead. Either way should work, but the latter may be easier as it could be hard to reimplement the same encryption method without any difference.


Two things are still unclear to me:

1: In parches.cpp I find the following comment:
* This function applies all patches to original USB ADVANCE executable
* is called by changing SifIopReset offset in the USBADVN elf.
But the USBADVN.elf file referred to here is the file that was repacked by the ps2-packer command in the Makefile, so making any direct changes in that file is not possible, and I can not find anything that modifies the elf before packing it either.

I have no problem with the patches made by the 'parches' function, as it is clear to me that this is intended to be called through the 'bent' SifIopReset call after launching the USBADVN elf, so that it has already unpacked itself in RAM before 'parches' is called.

So it is only the 'bending' of that SifIopReset call that I still haven't found...


2: When I attempted to compile the source using "make all", I got an error message complaining about a missing "libgs.a" file, which should have been stored in the "lib" subfolder. I never used libgs before, so I'll have to download that from the SVN repository and compile it before I can proceed with compiling the USBAdvance loader.

So this is as far as I've gotten for now.
Hopefully I'll be able to test it for real before my next post.

Best regards: dlanor

[dlanor]

@ifcaro:
Like the title implies, I think I may have the wrong revision of libgs.
I downloaded the latest from svn.ps2dev.org and compiled it error-free, using both 'make all' and 'make release', to ensure that the binaries also were added to the ps2sdk lib collection.

After this the compilation of 'usbld' proceeded quite well until the final stage, where I got tons of errors as follows:

ee-gcc -mno-crt0 -Tstartup/linkfile -D_EE -O2 -G0 -Wall  \
        -o main.elf /usr/local/ps2dev/ps2sdk/ee/startup/crt0.o obj/parches.o obj
/main.o obj/loader.o obj/config.o obj/gs.o obj/pad.o obj/usbadvn.o obj/usbd.o ob
j/usbhdfsd.o obj/fuente.o -L/usr/local/ps2dev/ps2sdk/ee/lib  -ldebug -lpad -lmc
-lsupc++ -lgcc lib/libgs.a -lc -lpatches -lc -lkernel
obj/gs.o(.text+0x88): In function `Graphics(int, int)':
src/gs.cpp: undefined reference to `gsDriver::setDisplayMode(unsigned, unsigned,
 unsigned, unsigned, unsigned, unsigned, unsigned, unsigned, unsigned, unsigned)
'
obj/gs.o(.text+0xa0):src/gs.cpp: undefined reference to `gsPipe::setAlphaEnable(
int)'
obj/gs.o(.text+0xd4): In function `flip()':
src/gs.cpp: undefined reference to `gsPipe::Flush()'
obj/gs.o(.text+0xdc):src/gs.cpp: undefined reference to `gsDriver::DrawBufferCom
plete()'
obj/gs.o(.text+0x114):src/gs.cpp: undefined reference to `gsDriver::DisplayNextF
rame()'
obj/gs.o(.text+0x11c):src/gs.cpp: undefined reference to `gsDriver::setNextDrawB
uffer()'
obj/gs.o(.text+0x160): In function `cls(unsigned long)':
src/gs.cpp: undefined reference to `gsPipe::setZTestEnable(int)'
obj/gs.o(.text+0x188):src/gs.cpp: undefined reference to `gsPipe::RectFlat(int,
int, int, int, unsigned, unsigned)'
obj/gs.o(.text+0x194):src/gs.cpp: undefined reference to `gsPipe::setZTestEnable
(int)'
obj/gs.o(.text+0x2c4): In function `DrawRect(int, int, int, int, unsigned long)'
:
src/gs.cpp: undefined reference to `gsPipe::RectFlat(int, int, int, int, unsigne
d, unsigned)'
obj/gs.o(.text+0x3a4): In function `DrawRectGoraud(int, int, int, int, unsigned
long, unsigned long)':
src/gs.cpp: undefined reference to `gsPipe::RectGouraud(int, int, unsigned, int,
 int, unsigned, unsigned)'
obj/gs.o(.text+0x4c4): In function `DrawTriangleGoraud(float, float, float, floa
t, float, float, unsigned long, unsigned long, unsigned long)':
src/gs.cpp: undefined reference to `gsPipe::TriangleGouraud(int, int, unsigned,
unsigned, int, int, unsigned, unsigned, int, int, unsigned, unsigned)'
obj/gs.o(.text+0x6ac): In function `DrawImageAlpha(IIF*, int, int, int, int, int
)':
src/gs.cpp: undefined reference to `gsDriver::getTextureBufferBase()'
obj/gs.o(.text+0x6d8):src/gs.cpp: undefined reference to `gsPipe::TextureUpload(
unsigned, int, int, int, int, unsigned char const*, int, int)'
obj/gs.o(.text+0x6e4):src/gs.cpp: undefined reference to `gsDriver::getTextureBu
fferBase()'
obj/gs.o(.text+0x710):src/gs.cpp: undefined reference to `gsPipe::TextureSet(uns
igned, int, gsTexSize, gsTexSize, unsigned, unsigned, unsigned, unsigned, unsign
ed)'
obj/gs.o(.text+0x790):src/gs.cpp: undefined reference to `gsPipe::RectTexture(in
t, int, unsigned, unsigned, int, int, unsigned, unsigned, unsigned, unsigned)'
obj/gs.o(.text+0x960): In function `DrawTextBox(char*, int, int, int, unsigned l
ong, int)':
src/gs.cpp: undefined reference to `gsDriver::getTextureBufferBase()'
obj/gs.o(.text+0x980):src/gs.cpp: undefined reference to `gsFont::uploadFont(gsF
ontTex*, unsigned, int, int, int)'
obj/gs.o(.text+0x9b8):src/gs.cpp: undefined reference to `gsFont::Print(int, int
, int, int, unsigned long, gsFontAlign, char const*)'
obj/gs.o(.text+0xa4c): In function `__static_initialization_and_destruction_0(in
t, int)':
src/gs.cpp: undefined reference to `gsDriver::~gsDriver [in-charge]()'
obj/gs.o(.text+0xa6c):src/gs.cpp: undefined reference to `gsDriver::gsDriver[in-
charge]()'
collect2: ld returned 1 exit status
make: *** [main.elf] Error 1

I'm not sure if this means that I have the wrong libgs revision (it has recently been updated), or if it is my main ps2sdk (of an older revision needed for uLE) which lacks something that libgs needs to compile properly.

I hope the error messages I posted can tell you what is wrong, so I can correct it.
My libgs SVN revision is 1583 (last committed at 1576)
My main ps2sdk revision is 1420 (required for fully functional uLE)

I'm perfectly willing to revert libgs to any revision you use, but I can't compromize about the main ps2sdk revision. This has to remain unchanged for the sake of uLE.

Best regards: dlanor