HD Homebrew Project Developement (Old Thread) [Archive]

Krevnik

02-28-2005, 11:38 PM

For those of you just joining the discussion, I decided to outline what is happening. The original post is still here, just quoted below.

Anyways, this has developed into a full-fleged open source project called 'HDDemolisher' (a bit of a jab at my own confidence about this project). The code is available under the GPL, and instructions for accessing the CVS server can be found here (http://www.ps2-scene.org/forums/showpost.php?p=235889&postcount=76).

The current status is that I have working CD/DVD detection, which was a test to see if code I wrote for my DVD9 ripping tool on MacOS X would port cleanly to the PS2's EE. It does, which means HDDemolisher will be able to rip and play DVD9 games without flattening them. Next up is interfacing with the HDD and with the HDL database.

What you can do to contribute:
- I need a splash screen, and a menu background. So artists/etc can help out in this regard, even if you don't know programming.
- I need information on how the IOP acts on the IOP reset command, and which IRX module is responsible for handling the reset. Any links/resources/documents to this end would be helpful.
- If you can program, get a copy of the source from CVS and read the TODO file. Anything marked as not finished is fair game. Submit the patches directly to me for inclusion into the CVS tree.

With homebrew apps like HDL Dump now available, and some of us with an understanding of how DVD9s work for PS2s (including myself), there is a chance that those ambitious enough could write a replacement for HD Loader / HD Advance which can solve two of the nagging problems I personally have with HD Advance:

1) Lack of built-in DVD9 support. It isn't quite as hard to detect a PS2 DVD9 as one would think. I use a short-cut for a personal DVD9 ripping tool on MacOS X because Virtual PC forces FAT32.

2) Lack of support for storing saves onto the HD. When Battlefront, Star Ocean, and Dark Cloud 2 can eat up 4MB of space on their own VERY EASILY, being able to store saves on the HD directly would help in not having to constantly weed out saves from space-hungry games like Dark Cloud 2.

The tricky parts to this project would be a custom CDVD driver, and for #2, a custom MCMAN driver (the first is required just to trick games into running off the HD, and DVD9 support cannot be done using HD Advance's CDVD driver). The easy part would be the code which reads the CD or DVD image onto the HD, the menus, etc. Also, I would like to incorporate the HDL Dump server into the app (this would be a GPL project, so it would be possible), and provide any patches we make to the server back to the project.

I am confident that I could achieve this on my own, although having an extra hand would greatly speed up the development. I plan to have a project roadmap and milestones laid out by the end of the week, so the earlier someone with PS2 programming skills decides to join up, the more input to the roadmap they can provide.

Anyone feel as ambitious as I do?

omegatron

02-28-2005, 11:42 PM

this is a way bigger project that i could ever imagine.... but so is ps2 dos LOL

Krevnik

03-01-2005, 12:02 AM

It isn't nearly as big as what I had to write during my last term in college: 2 projects, one C/C++ 3D engine with Serial I/O for a VR suit totalling 5000 lines of code, the other a Java telnet game server (similar to a MUD) totalling 4300 lines of code. If I can do both of those single-handedly in under 15 weeks, this will be possible, but I prefer to work in a group to speed up development and let me focus on the key problems that I have leads on solving.

Deeve

03-01-2005, 12:20 AM

Sounds like you are truly determined...I hope you decide to start and at least learn some stuff from trying.

heh...I gotta try at least....maybe a good project to start would be a "simple" win32 app that will copy .elf, .mp3 and rom files to organized directories from the PC with the PS2 drive attached.

omegatron

03-01-2005, 12:22 AM

well go for it trust me no of us will stop you... Good luck

pstar

03-01-2005, 09:23 AM

looking forward to see the development of this project... Good luck

Kormann

03-01-2005, 10:05 AM

1) Krevnik, if you don't mind you could *try* to make a combo of buttons (something like start+select+r1+r2+l1+l2) to go back to your homebrew hdloader main menu.. of course you could put a time delay like 5 secs so it wouldnt be incompatible with games that already reset (in game) pressing this kind of combo for 1-2 seconds.
2) And maybe, just maybe, you could add a hdd browser so you could put elfs of homebrew programs into folders (in some partition) and organize them realtime like ps2menu does, and add fakehost (like ps2menu) so any homebrew programs would run of the hdd. (we know its in your low priority list but thats an idea :D)

Good luck dude, we are all here to support your cause :)

[]'s
Kormann

Krevnik

03-01-2005, 10:20 AM

Kormann, on your two ideas:

1) Difficult, as this would require knowing where to reload the old elf, and patching or rewriting PADMAN. Because I am already working on the difficult task of writing a custom CDVDMAN, this just cannot be a priority, at the moment, but I can put it at a higher priority over storing save files onto the HD.

2) This would be possible, and actually not that difficult, since we could build from PS2MENU on that front.

The best way to support me: help out with coding experience. Even if it isn't much, someone who helps write the interface for the user, and provides a means to write debug output to the screen in a nice fashion would still save me a great deal of time. I am not looking for someone to help with the driver writing, but rather all the gruntwork that needs to be done so I can focus on the stuff that is really gonna suck.

Kormann

03-01-2005, 10:55 AM

Hmmm thats ok :)
I'm not a coder.. All I could do is create some concepts of the interface, because writing an interface means coding and I'm not able to.
Another thing, don't disable network adapter (like hdloader does with the new games) thing for online playing.. it would be nice to play new games from hdd thought.
I know hdloader doesnt disable network adapter. There is a theory that it conflicts with a module used for DN4$ and so will be read as no network adapter instaled.

A proof would be: you can play games online against other ppl using xlink (games that are able to play in lan mode) - SW: BF, TimeSplitters, Twisted Metal Black Online. And some older games that has internet play (but no DN4$) like Socom1, Tribes Aerial Assault, Automodellista, Twisted Metal Black Online.

[]'s
Kormann

Krevnik

03-01-2005, 11:32 AM

Yes, but one thing to consider is where does the dev9/atad/smap tree branch out from the 'official' IRX drivers? If we could build a setup that only uses a custom atad driver, and a custom cdvdman, then we /might/ be able to do this. However, it is not on my todo list at the moment since it is rather tricky to solve, and would be more useful for those who pirate the games, rather than the owners. So I am not likely to actually do anything on this front.

c0d3x

03-01-2005, 01:37 PM

waiting for the project roadmap ;)

ale275

03-01-2005, 01:40 PM

the biggest problem with mcman is that every game uses a different one or use it in different way so u can't do an universal patch.... :(

Deeve

03-01-2005, 01:53 PM

ya know it really doesn't seem like you have that bad of a chance in doing this. I fully support this in any way I can. Maybe we will have a compatibility list for this app at www.ps2hd.com one day.

geez..ya got me all excited to see people interested to try.

HypERSoniC

03-01-2005, 06:54 PM

With homebrew apps like HDL Dump now available, and some of us with an understanding of how DVD9s work for PS2s (including myself), there is a chance that those ambitious enough could write a replacement for HD Loader / HD Advance which can solve two of the nagging problems I personally have with HD Advance:

1) Lack of built-in DVD9 support. It isn't quite as hard to detect a PS2 DVD9 as one would think. I use a short-cut for a personal DVD9 ripping tool on MacOS X because Virtual PC forces FAT32.

2) Lack of support for storing saves onto the HD. When Battlefront, Star Ocean, and Dark Cloud 2 can eat up 4MB of space on their own VERY EASILY, being able to store saves on the HD directly would help in not having to constantly weed out saves from space-hungry games like Dark Cloud 2.

The tricky parts to this project would be a custom CDVD driver, and for #2, a custom MCMAN driver (the first is required just to trick games into running off the HD, and DVD9 support cannot be done using HD Advance's CDVD driver). The easy part would be the code which reads the CD or DVD image onto the HD, the menus, etc. Also, I would like to incorporate the HDL Dump server into the app (this would be a GPL project, so it would be possible), and provide any patches we make to the server back to the project.

I am confident that I could achieve this on my own, although having an extra hand would greatly speed up the development. I plan to have a project roadmap and milestones laid out by the end of the week, so the earlier someone with PS2 programming skills decides to join up, the more input to the roadmap they can provide.

Anyone feel as ambitious as I do?

I'm not sure about #1, Krevnik, but you are spot on with your #2. You have 4 posts here and you immediatly recognise that MCMAN and CDVD are the programs you must target? You cannot tell me your are new to the ps2-scene...

There is a GPLd CDVD.IRX available over at PS2DEV. It kick's Sony's CDVD.IRX's ass all the way to Japan supporting Joliet and ISO level 2, etc etc. You can check out the source on the ps2dev cvs. If you want contribute to the homebrew CDVD.IRX project then adding support to HDLoader (i think) would be trivial. To get it to load your driver onto the IOP you could just replace the inbuilt CDVD.IRX inside the hdloader executable. (unless hdloader uses sony's driver in the ROM, but since sjeep was a major contributor to many ps2dev projects, i think he would have stored the free version in his executable)

Replacing MCMAN in the ROM would be tricky.. There is a modchip which can redirect calls from rom0:MCMAN to another of your choice, the 02. I'm not a PS2 developer so i'm not sure if you need a modchip to achive what you are trying to do, or if you could just run a program to redirect calls from rom0:MCMAN to your own file (on the MC or something). There is, infact, someone who wrote his own(or modified Sony's) MCMAN over on the 02 forums, which allowed the ps2's browser to access USB memory directly(it just saw them as really really big memory cards)

You could add your MCMAN to games easier though. Most games use their own MCMAN on the game's disc, so you could just replace it and reburn.

www.ps2dev.com <--- invaluable resource

GOODLUCK!

omegatron

03-01-2005, 11:39 PM

hell this is almost news worthy

Best of luck to you

Krevnik

03-02-2005, 12:18 AM

I'm not sure about #1, Krevnik, but you are spot on with your #2. You have 4 posts here and you immediatly recognise the areas of the PS2 which deal with your idea? You cannot tell me your are new to the ps2-scene...

Well, #2 is trickier, since it is stated by another that various apps do it. #1 /is/ possible. The raw access of a DVD9 in pretty straight-forward, so is checking for it (as I said, I wrote a tool for MacOS X which does a raw read of the DVD and can detect PS2 Dual-layer discs, and rip them correctly). PS2 DVD9 discs don't comply with the ISO specs for defining multiple filesystems on a disc, but is still really easy to detect.

Ironically, I can say I am pretty new, but I learn quick, and have multiple years of experience with computer and embedded scale development, and did a little tinkering with PSOne emulation bits awhile back. I think there is enough information to do this at this point, but not without relying on PS2SDK and other projects. Hence why I want it open source, to ensure that it adds to the community, rather than leeches.

There is a GPLd CDVD.IRX available over at PS2DEV. It kick's Sony's CDVD.IRX's ass all the way to Japan supporting Joliet and ISO level 2, etc etc. You can check out the source on the ps2dev cvs. If you want contribute to the homebrew CDVD.IRX project then adding support to HDLoader (i think) would be trivial. To get it to load your driver onto the IOP you could just replace the inbuilt CDVD.IRX inside the hdloader executable. (unless hdloader uses sony's driver in the ROM, but since sjeep was a major contributor to many ps2dev projects, i think he would have stored the free version in his executable)

I have been searching for the CDVD IOP sources, but have only found stubs which call the kernel, and a EE library to access the stubs (of course this is PS2SDK's CVS tree). I may be wrong though, but this doesn't seem right, or I am looking in the wrong part of of the CVS tree for the project.

Double Post Edit: I found what you were referring to, although it is really just an IOP server which uses the official CDVDMAN to read from the DVD. However, this, along with PS2SDK's libcdvd, gives me quite a bit of information on what CDVDMAN needs to do for other IOP drivers and EE, and provides a good start for me to work with. I could probably write up a HD version of CDVDMAN for other IOP code in a couple of days (that meet the requirements for the module's exports). Getting it to interact with the EE properly /might/ be a bit longer, as I am not too familar with the SIF yet.

Replacing MCMAN in the ROM would be tricky.. There is a modchip which can redirect calls from rom0:MCMAN to another of your choice, the 02. I'm not a PS2 developer so i'm not sure if you need a modchip to achive what you are trying to do, or if you could just run a program to redirect calls from rom0:MCMAN to your own file (on the MC or something). There is, infact, someone who wrote his own(or modified Sony's) MCMAN over on the 02 forums, which allowed the ps2's browser to access USB memory directly(it just saw them as really really big memory cards)

You could add your MCMAN to games easier though. Most games use their own MCMAN on the game's disc, so you could just replace it and reburn.

www.ps2dev.com <--- invaluable resource

GOODLUCK!

Here is the thing, you don't need to. If that was the case, then HDLoader wouldn't work, since it loads CDVD.IRX of its own, and then prevents the game from loading the one on disc. If it thinks MCMAN is already loaded, it won't reload it, and effectively replaces it until an IOP reset (which is how HD Loader hijacks everything). The first thing to do when loading this app is to reset the IOP and start loading modules needed for HDD access. Then the hacked CDVD and MCMAN can be loaded before executing the ELF. This way, the hacked MCMAN support can be enabled on a game-by-game basis. This would probably break support with HD Loader at this point, so I want to make sure that it has two things first:

1) DVD9 support in addition to the feature-set required by a loader.

2) 100% GPL or some other open source license.

#2 is key for me, as it is what hurts HD Loader-type development on the PS2 currently. Not to mention that attempting to make money on HD Loader like the HDA team did to Sjeep, rubs me the wrong way. Homebrew apps like this should remain open so that people can learn from it. ;)

omegatron

03-02-2005, 12:22 AM

come ppl step up and say yes i will help you or i know someone who will...Get this thing rolling

liknsmak

03-02-2005, 12:32 AM

good luck. if you can get this thing rolling it going to be amazing. sounds like you know enough to do the job too.

Krevnik

03-02-2005, 01:08 AM

Well, someone asked for the Roadmap, and here it is. 5 stages, with actual game playing being stage 3. Since I don't know exactly what the compatibility modes in HDL/HDA /do/ at the moment, I likely will not have support for them. Anyone that can provide insight on this would be pretty helpful. The idea I have is to be as accurate as possible on the CDVD driver so that compatibility modes are not as needed as before, but it might have its own set of modes as I discover glitches with my own games. Probably at Stage 3 to 4 is where I plan to break away from supporting HDL Loader's way of doing things, as it would be able to do right about everything HDL/HDA can do at that point.

SephirothTNH

03-02-2005, 01:58 AM

Incase you wanted to know about the modes...
Acording to Zer0-X
Mode 1: Enable slower HDD access. Passes data from the HDD slower for games that rely on the speed the data is coming from the CDVD.

Mode 2: Enable DVD9 support. (games larger than 4 gigs)

Mode 3: Remove the Loader from the memory after the game is patched. Some games flush the location where the loader is (because some cheat programs use the same location), and if after that the game resets the IOP (when the loader kicks in to patch stuff) and the system is still patched to enter the loader it crashes.

Krevnik

03-02-2005, 02:47 AM

Okay, Modes 1/3 I can definitely see getting supported then. Mode 2 will likely be obsolete if my work is successful, as it would be able to autodetect the type of disc image stored, and even allow natural DVD9 accesses to the image.

lazyj

03-02-2005, 05:35 AM

It must be good to be able to do this stuff :)

HypERSoniC

03-02-2005, 07:02 AM

oh, also Krevnik, in this way you would be able to add hdd support to the browser by simply mcman.irx into the iop, then just run the browser from the ROM...

I read your roadmap. I think you have summed it up quite well. Consider, however, that you will not be able to access the source code to hdloader(unless you reverse engineer and rewrite it).However there are alot aspects that are very hard and will require alot of work and knowledge. You have a very bumpy ride my friend, and it is definitnly not as easy as it seems. here are my thoughts:

Milestone 1 - Possible

Milestone 2 - Possible

Milestone 3 - Perhaps difficult. HDLoader (as i understand it) resets the IOP *after* it has been launched, which will ofcourse remove your IRX's from the system. Unless of course you edit the machine code and replace the driver with your own(not sure how hard this is. It has been done before with byte for byte exact replacement) And also, games reset the IOP when they are launched. Some games even reset the IOP many times at different stages during the game. This is why the 02 coder had trouble making his replacement for games, but had no trouble with the browser.

Milestone 4 - Easy to implement, but perhaps tricky to code. It has been done before, so you may want to consult with the original author). To make the ps2browser see the hdd, loading custom MCMAN into the IOP then running ODSYS on the EE. To make games see it, simply replace MCMAN in the original game image. This howerver is not that simple. Many games these days store the irx files inside their compressed/crypted datafile. So for these games, unfortunatly this is impossible.

Milestone 5 - HDD Homebrew loader - dead easy, just cut and paste.

Revert to hdd demolisher, possible. Sjeep already has done this in his own testing environment.

DNAS support - im 90% sure that has already been achieved. Only people who know how are the ones who will benefit (the info was promptly removed from this site)

In conclusion,, i think that you are underestimating the difficulty of this project.

sincro

03-02-2005, 09:20 AM

Krevnik

03-02-2005, 10:51 AM

little consideration:

you don't need to write a custom cdvd.irx...why? for do that you need to write a custom cdvdman module instead of rom0:CDVDMAN, not need a custom cdvd.irx. You must to write rpc server module that return to any (commercial or homebrew) cdvd.irx information from hdd and not from cd.

Not simple to make....

Well, either route can be taken, and I might wind up taking both to see which gives better compatibility. The trick is, does Sony's CDVDMAN have calls into the driver that we don't know yet? If so (which seems unlikely though, since IRX exports are clearly marked), CDVDMAN will be the better choice. Otherwise, we might get better compatibility from writing a fake CDVD driver. It could be due to replacing CDVDMAN instead of the driver itself that causes issues with some of the games.

However, you are right and I should start with CDVDMAN, as there is already something like that to start from. ;) (On a side note, I should probably refrain from posting so late, my concepts get muddled)

Milestone 3 - Perhaps difficult. HDLoader (as i understand it) resets the IOP *after* it has been launched, which will ofcourse remove your IRX's from the system. Unless of course you edit the machine code and replace the driver with your own(not sure how hard this is. It has been done before with byte for byte exact replacement) And also, games reset the IOP when they are launched. Some games even reset the IOP many times at different stages during the game. This is why the 02 coder had trouble making his replacement for games, but had no trouble with the browser.

*snip*

In conclusion,, i think that you are underestimating the difficulty of this project.

Nah, I am just oversimplifying the key issues in my roadmap. Why else would game play be its own milestone? The rest is EASY up to this point because what I don't already know, is easily accessible at this point. Milestone 3 is where the fun begins and I find out if I can finish the last mile. Milestone 4 is seperate because honestly, it is the last task, and could be just as tricky as #3.

The IOP resets will be the tricky part as I understand it, as it would quite easily ruin my day. Finding the method (which has already been found, obviously), to execute my function when the IOP Resets will be the fun part. The easier part is after that.

sincro

03-02-2005, 11:03 AM

Well, either route can be taken, and I might wind up taking both to see which gives better compatibility. The trick is, does Sony's CDVDMAN have calls into the driver that we don't know yet? If so (which seems unlikely though, since IRX exports are clearly marked), CDVDMAN will be the better choice. Otherwise, we might get better compatibility from writing a fake CDVD driver. It could be due to replacing CDVDMAN instead of the driver itself that causes issues with some of the games.

However, you are right and I should start with CDVDMAN, as there is already something like that to start from. ;) (On a side note, I should probably refrain from posting so late, my concepts get muddled)

I think for rewrite a cdvdman you must start from libcdvd, and make a rpc server respond of libcdvd (from ps2sdk). I think all callback are include to libcdvd. If you rewrite a cdvd driver and most game use a specified cdvd driver from IRX folder? You must to patch original elf game?

I would not know as the compatibility could help rewriting the CDVD driver...

omegatron

03-02-2005, 11:06 AM

oh when (not how) was dnas ( of the new library) possible from the hdd...It never was?!

Krevnik

03-02-2005, 11:30 AM

I think for rewrite a cdvdman you must start from libcdvd, and make a rpc server respond of libcdvd (from ps2sdk). I think all callback are include to libcdvd. If you rewrite a cdvd driver and most game use a specified cdvd driver from IRX folder? You must to patch original elf game?

I would not know as the compatibility could help rewriting the CDVD driver...
Well, some games that include it seem to be including an /updated/ version of the BIOS CDVD driver, replacing it after the IOP reset. If done correctly, we can sneak a different one into the IOP that won't get removed.

sincro

03-02-2005, 11:53 AM

Well, some games that include it seem to be including an /updated/ version of the BIOS CDVD driver, replacing it after the IOP reset. If done correctly, we can sneak a different one into the IOP that won't get removed.

ok but how you make it?

Krevnik

03-02-2005, 12:44 PM

That is a good question, there is enough information to know which calls are commonly used, and what they take in/return. However, it seems not all the exports from CDVD.IRX are known at this point, unless there is intentional obfuscation on the part of Sony. Hence the starting with the RPC module, rather than the driver module.

omegatron

03-02-2005, 12:50 PM

sence it is home brewn why is allowing dnas support bad i mean it doesnt envoke piracy any more that being able to put rented games on the hdd. You have to pirate a game to play a pirated game online so you are doing nothing wrong by allowing dnas support

Deeve

03-02-2005, 02:02 PM

I've said it before....contrary to others statements about it, but I think having dnas support would be as simple as not coding a DNAS detection routine like HDLoader does at some point.....I don't think it has conflicts with Dev9 when using the NA and HD at the same time because it is possible with some games and even a few with pre-271 dnas (I think) libraries have worked from hd. And I don't think DNAS itself has a built-in protection to disable the NA in anyways. But this isn't even important right now.....it's about getting the first milestones clear and understood first.

Krevnik - no like pm's? :0

Krevnik

03-02-2005, 02:34 PM

Well, there are two reasons why I have /no/ plans to support DNAS. Firstly: it circumvents explicit copy protection in the PS2. The CD check in the PS2 is hardly copy protection, and loading a game off the HD /could/ be defended in court (rather weakly under the space shifting court decisions in the US), as it does not violate any clear part of the DMCA.

Secondly: It is going to be GPL software. Want someone to add the support? Get someone to do it. I don't plan on purposefully disabling DNAS, but if it doesn't work, I will not put any effort into making it work. Simple as that.

Writing an app like this is walking a fine line. I don't intend to bait the sharks by doing something blatantly illegal in the US (even if the law is stupid).

Double Post Edit: Oh, and don't expect me to check my PMs often, I didn't notice until just now, and I will be slow getting back to them.

Status: I am currently writing up the CD/DVD/DVD9 detection code, porting what I need from my MacOS X tool (irony), and plan to have a really cheesy text screen that lets me know what disks are being put in. This way I can reach the first milestone this week or next.

omegatron

03-02-2005, 03:18 PM

ok so you will do yor part to make it so that you "CANT" play dnas games off hdd or you will just code your app and if it works cool if not no dnas
Are you physically going to code dnas protection

Krevnik

03-02-2005, 03:42 PM

As I said, I am not going to go out of my way to disable DNAS, but if it doesn't work without me doing anything, I will leave the support broken. If I go out of my way to cripple it, another user will just uncripple it, wasting my effort entirely. If I go out of my way to ensure that it works, I get eaten by the $harks. ;)

DOUBLE POST EDIT:

A small status update, I have my build scripts setup along with a chunk of the code linked together into a small binary, which means it accepted my ported DVD9 detection code. Now the task is to see if it really works, but I have things coming up, so it won't be until early next week I have some code to post.

v1p3r

03-03-2005, 06:50 AM

Good luck with the project, have you considered putting up a CVS so people can see what your doing? :)

Krevnik

03-03-2005, 09:53 AM

I could run the server off my home server, and am thinking about doing so, although I would at least like to reach the first milestone before doing so. This means the code would be in your hands about the same time either way. But since this will be a project under the GPL, it isn't like I have anything to gain by hiding the code once there /is/ something to show for it. ;)

v1p3r

03-03-2005, 10:24 AM

What's wrong with sourceforge? :)

adavidm

03-03-2005, 01:46 PM

Or ps2dev.org? It might be worth asking Pixel or ooPo what they think.

adavidm

sincro

03-03-2005, 02:54 PM

Or ps2dev.org? It might be worth asking Pixel or ooPo what they think.

adavidm

I think ps2dev not really like this project....

for Krevnik: if i can i help you, now i'm still busy with new os ...

omegatron

03-03-2005, 03:13 PM

"new os"??

Also what is savesupport is fully working mean (

(sorry to get of topic)

Psumoni

03-03-2005, 05:34 PM

I'd be happy with the Loader being capable of just auto-loading MC directories from the HD to the MC prior to execution of the specified game. The loader can make a note of what game was loaded, and upon re-execution of the Loader, via reboot or the possible "return to loader" option, will then (unless prevented by user) copy the directory from the MC back to the HD, and remove the directory from the MC.

Of course you could embellish this option a little (or a lot) so that you could just leave it on the MC, useful for playing the same game over and over, instead of moving the data back and forth each time the Loader loads. Futher options could be to allow multiple copies of a game's directory to reside on the HD, each to a different user, profile-style, just select one as the game is selected to play. A huge feature would be to add a "learn this game" mode - have the Loader compare the MC directory structure before and after the game was played to allow it to give a best guess as to what directory belongs with the game you just played. Natually, you could manually tell the program the MC directory for a game.

Guaranteed 100% compatibility, and it would add a few seconds to boot, but that time is much shorter than doing the manual backup routes we have now.

Krevnik

03-03-2005, 09:21 PM

I think ps2dev not really like this project....

for Krevnik: if i can i help you, now i'm still busy with new os ...

Oh yeah, PS2Dev would smite my user account on their forums faster than you can say 'HDLoader'.... well, maybe AS fast, since you have to mention it first before they actually smite user accounts. The official stance of their mods is that HDLoader and its ilk are warez tools, and nothing but. They have a similar stance on 'backups'. I do have one burned game, and that was because I ripped the PAL version of Worms Forts, NTSC patched it, and then burned it so I could play it in the US from my UK copy. However, the warez people /have/ made life difficult by learning how to hide in public and use terms like 'backup' in place of 'copy I got off that internet thingy'. It is thanks to greed and these folk that the US is subjected to the stupidity of the DMCA (by giving the BSA, RIAA, and other acronyms the excuse they need for such draconian legislation).

Okay, enough of the soapbox... about HD memory card support, I will likely not be using a method of copying files around to get better support. This would add more than just a few seconds to transfer ~8MB around, and burns through the limited write ability of the flash EEPROM of the memory card.

mastaalien

03-04-2005, 12:02 AM

didn't understand the half of the posts, sience im not a coder.
(to bad i got noone to teach me over here)
but your project seems amazing. hopefully you will do the trick and i hope also many coder will help you.

Psumoni

03-04-2005, 01:15 AM

Okay, enough of the soapbox... about HD memory card support, I will likely not be using a method of copying files around to get better support. This would add more than just a few seconds to transfer ~8MB around, and burns through the limited write ability of the flash EEPROM of the memory card.

I agree that's a lot of EEPROM writes, and would cut the expected lifetime down. But it would not be doing the whole 8meg each time, just the directory for the game being launched - from just 100k up to a couple megs for very large games.

Kormann

03-04-2005, 07:53 AM

Does someone here has a PS2 with Sony's updated browser (1.0 or 1.1) installed?
If yes (or no, whatever) you might know there is a folder where you can backup your MC saves to hdd.. When your create a folder for your MC backups it is created on "pfs/0/ hdd:__common", for example if I 've created "PS2 Saves" while in hdd browser it will appear "pfs/0/ hdd:__common/PS2 Saves" when you try to browse it with exectftps.
Why I had post this useless info? Its not that useless, if you want to make your HD act as a big memorycard (just like MM16 memory cards for example) you could create a folder to store your saves with your HDDemolisher or with Sony's Browser (but must be created at "pfs/0/ hdd:__common/" for ppl that has "sony's 1.0 or 1.1 browser" to browse their saves with its browser).
I know that for ppl that doesnt have Sony's browser it would work too because these folders aren't used by PS2 anyway, so no conflict and excelent compatibility.

What do you think ppl? For me its a very cool idea lol :P.. if you have installed PS2 SOny's browser you can browse your saves while in browser too so it would be good for the user when wanting to backup the saves back to MC (so no time wasting using ps2menu to transfering from folders that arent visible by ps2's browser)

[]'s
Kormann

Krevnik

03-04-2005, 01:34 PM

Well, I seem to be getting closer to figuring out how I should approach the IOP reset problem. Here is what appears to need to be done:

- Detect the IOP reset command as it comes in, and let the EE know. Preferably AFTER the reset is complete and the IOPRP image is loaded (if there is one). This will require some patch to either part of an IOP module (since modules have to be loaded in from ROM to run, any Sony module can be patched), or the game itself. MIPS is great for this, since every instruction is specifically 32 bits wide.
- "Unload" any CDVDMAN/ATAD module that currently exists (this can likely be achieved by removing the reference to the module from the linked list), and load a new one in its place from our little part of the world. They must exist in RAM somewhere or be on the MC in order to load.

Now, there are a couple of nice things: we can store the HDDemolisher IRXs onto the MC (as part of the install process) and use a lightweight patch function written INTO the game's memory space after loading the ELF, but before executing it. This way, games that attempt to clear KSEG0 will fail to clean out the patch.

Angelus3X

03-04-2005, 01:43 PM

Goodluck Krevnik! Great project your working on! I know it may be imposible to say but when do you expect to complete a beta of this app? 6 months? 1 year?

I'm sure many people cant wait to try this out!

Good luck tackling those tricky parts, dont give up!

Krevnik

03-04-2005, 07:35 PM

Well, some very good news, I am about half-way finished with Milestone 1 (the hard part of it anyways). I just confirmed that my code correctly detects the type of PS2 disc inserted into the drive. This has been tested with multiple PS2 Single-Layer DVDs, Xenosaga I (it even reports the correct length and layer break!), and a PS1 CD. I don't have any valid PS2 CD games at the moment, so I cannot test CD code very well. Attached is the ELF for anyone interested in seeing REALLY fast scrolling text. I am working on getting my CVS server configured for public access (the reason why I am using my own server is that I have quite a few projects that CVS would help manage, but sourceforge isn't for em all). Once that is done, and I have the copy of the GPL license in place within the source tree, I will make the CVS server publically accessible for those interested in the DVD9 detection code.

Attached Files:
DEMOLISHER.ELF - Very prelim app, sits in a loop detecting the disc type. Might need to rename it before loading it into your PS2. Plus, you must wait for 'Valid PS2' to report YES before the information on the disc being dual-layer is valid. I intend to address this before Milestone 1 is reached, so that it sets a flag stating that the disc is still being detected.

Roadmap.doc - Updated roadmap. Added a section which discusses issues with development and how they are being addressed. The first issue discussed is reading/detecting DVD9 discs.

Additional:
If someone can confirm that this does detect other DVD9 games like GT4 and the like, please do so. Do remember that you need to wait about 5 seconds before the detection is truly accurate (and you can eject the disc again to get the full info on the disc without the scrolling).

romz

03-04-2005, 08:31 PM

First, this program crashes under NapLink (sad). But also good news - I tried it with Chinese Gran Turismo 4 (HK Silver) and it works (Dual Layer DVD). Xenosaga Episode I (english) also detected as DVD9.

It seems this program tries to read discs when drive still isn't ready to do it. You probably have to add "cdDiskReady(0);" to your code.

Krevnik

03-04-2005, 10:03 PM

Yeah, I know, but I was more concerned with it reporting EVERY DVD as a dual-layer, which stemmed from me not checking for errors from a non-blocking cdRead()... so used to synchronous programming. I personally use ExecFTPs to copy the ELF onto the memory card and execute from there, as I don't have things setup for software like PS2Link or NapLink (yet).

So that is still good news though that GT4 is detectable in the same way, which means that is the format that all DVD9 games are likely to follow (since I only have Xeno to test with for DVD9 support). Still, I would like to see someone test with some other games (HK Silvers /may/ behave differently from the rest). Anyone have Champions of Norrath or the like to test with as well?

While people get a chance to see how it works, I will be making changes to the disc detection code to finalize it. I will also be working on the code to work with the HDD and possibly making progress on the second milestone by accessing the HDL database. Once it can interface with the database (with some help from the HDL Dump source), I will start work on the initial UI and set things up to provide ripping ability of games (hopefully without the 'you have run out of space' glitch).

RyDaWg2k1

03-04-2005, 10:08 PM

Way to go Krevnik, keep up the great work! its nice to see some development on a homebrew hd loading app and the progress so far is fabulous :) Lookin' forward to future updates

P__S__2

03-04-2005, 11:57 PM

nice work Krevnik..keep up the good job :)

omegatron

03-05-2005, 12:11 AM

this app role screens for me i am ntsc running it from thumbstick
with spyro the dragon psx in does a dvd nine game have to be in for it to display right

Stefy2

03-05-2005, 01:26 AM

esojz

03-05-2005, 01:29 AM

Is there anything that those of us that don?t know anything about programming can do to contribute to this project? (Besides give you lots of praise!)

v1p3r

03-05-2005, 08:25 AM

People with ph33r photoshop skillz could contribute some graphics for the menu :)

CrysDark

03-05-2005, 11:55 AM

Am I mistaken, or did krev throw this together in about a week???? :wow: Good job man, perhaps when you finish this project we can get you to tackle a media player. :lol:

Krevnik

03-05-2005, 12:13 PM

it's because it's too fast to recognise cd.
anyway i tested it with xenosaga 2 dvd and it recognise it right valid ps2 , dvd single layer.
with ar max evo ntsc also it's recognised right as cd and valid ps2
but if i insert a neo geo cd it's still recognised as valid ps2.
the same with a dvd containing only avi

Are you using a modchip? If so, that would explain the odd behavior with it saying everything is a valid PS2 disc. It is how it manages to get the PS2 to boot burned CD-Rs and DVD-Rs, afterall.

This is NTSC but should work on PAL and NTSC systems, it just constantly reads, so you get the scrolling text issue, I hope to solve this and release an ELF tonight (US west coast) without that issue, so it only reads a particular disc once, and only when it is ready to read.

Anyways, there are currently two things that people can help me with:
1) Once the new ELF is ready, take it and test to make sure it reports discs correctly. As in, reports DVD9s and DVD9s and DVD5s as DVD5s. So break out your copies of whatever DVD9 games you do have, and make sure that they are detected correctly.
2) Yes, help with graphics. I /could/ do something myself, but I would rather focus on code. Be aware that the work that you contribute to the project in terms of art will fall under the GPL.

SiNER

03-05-2005, 01:28 PM

Great progress you've made so far. I have my copy of GT4 ready for testing once you release the updated version of your .ELF tonight.

Krevnik

03-05-2005, 01:33 PM

Great progress you've made so far. I have my copy of GT4 ready for testing once you release the updated version of your .ELF tonight.

Here is your chance, I just tested it, and it works a lot nicer now. Pop in a disc and it will tell you what type it is... once. (Basically a loop that waits for the disc to be ready, then waits for the disc to be ejected before going through again)

Anyways, here is the updated ELF, but the CVS server isn't ready yet (rejecting my anonymous logins still).

Tec

03-05-2005, 01:49 PM

DVD game backups return:
ValidPS2: No, discType: Unknown

V7, no modchip PS2.

Krevnik

03-05-2005, 02:44 PM

Well, let me start by saying that the ELF doesn't support the swap trick, so burned discs will not work. Since you have to open the tray, guess what: the PS2 checks the disc and you get the result you saw. Eventually, it might wind up supporting swap tricks simply because I cannot prevent it, but that is for us to find out at Milestone 2. Right now I am only looking to see the results from originals. ;)

v1p3r

03-05-2005, 03:23 PM

Working great here on non-chipped V9 :)

snake3

03-05-2005, 03:59 PM

Reported:

MGS2 : Substance as DVD9

Smack down Vs. Raw as DVD5
Devil May Cry as DVD5
X-Men Legends as DVD5

Maximo as CD
RPG Marer 2 as CD
Code Breaker 9.0 as CD
Code Breaker 8.1 as CD
HD Loader as CD

All of the results were correct

It returned all my disc as valid even the burns because i have a mod chip

spawner_brasil

03-05-2005, 04:04 PM

Any possibilty to transfer DVD Movies and maybe PS1 games?

Krevnik

03-05-2005, 04:20 PM

Heh, not likely. I honestly have no idea how PS1DRV works nor the DVD player, and so until I get a chance to do a rom0 dump, I cannot say if it would even be possible. I would practically need one of the prominent BIOS hackers helping in order to approach those tasks.

spawner_brasil

03-05-2005, 04:40 PM

And any possibilty to support DVD5 and DVD9 movies?

E P

03-05-2005, 06:13 PM

Krevnik, your demolish.elf works fine when ran through PS2LINK via inlink.

Exploit PS1 trigger disc
Disc Read:
validPS2: No, discType: CD

Gran Turismo 4 NTSC USA - rental
Disc Read:
validPS2: Yes, discType: DVD, Dual-Layer
Layer Break: 1317056, Total Length: 2594960

Sorry, but I don't own any DVD9 games.

SiNER

03-05-2005, 07:05 PM

Guess you beat me to it. BTW, Dual Layer = DVD9.

romz

03-05-2005, 09:00 PM

Many useful PS2BIOS related thing can be found nearby at http://ps2dev.ps2-scene.org/
And PS2BIOS dump tool can be found there. I used it to dump rom0 via PS2Link. But DVD player is not there, it's located in encrypted rom somewhere in rom1.

zabolyx

03-05-2005, 10:35 PM

well doesn't the driver for DVD play load from the MC once it is installed (with the update)? So could one just have the app rewrite this driver there?

I'd love to help with testing and some dev (teaching myself C/C++) but I just moved and my PS2 got some major abuse and won't read any CD/DVD discs... tried cleaning and realigning the sled but look dead to me...

spawner_brasil

03-06-2005, 12:07 AM

I don´t know if transfer files betweeen PC and PS2 are slow (0.7kbs) because everybody are using a old source to make transfers using ExecFTP, Ps2vfs, hdl server... but if possible to transfer files from PC to PS2 using full ethernet capacity is possible, it can be a good upgrade. It is possible?

Krevnik

03-06-2005, 02:01 AM

And any possibilty to support DVD5 and DVD9 movies?

My earlier post says it all: not likely, too much effort and beyond the scope of the project. I would rather work on getting PS1DRV working with the HD than DVD video playback, and I am not quite good enough with reverse engineering to be able to say that I could get the PS1DRV working.

On the slow transfer speeds, that can likely be explained by how the I/O bus in the PS2 works between DEV9 and the rest of the SBUS. 0.8MB/sec is pretty good (About 10Mbit, once overhead is accounted for), and realize this: When reading from the network to the HD, you read data from SMAP, send it over the SBUS to the IOP, and then send it back over the SBUS to the ATA bus (through DEV9 again). It is hardly surprising that the network speeds really suck, especially if you sync things up after every read and write. The connection to DEV9 probably isn't that great.

Stefy2

03-06-2005, 03:26 AM

yes i have a modchip as you can read from the signature, anyway non ps2disc are being detected as dvd video and at boot up as not valid ps2 disc obviusly and psx games as psx games also before bootup, anyway hdl detects them correctly as not valid

Krevnik

03-06-2005, 12:20 PM

Gran Turismo 4 NTSC USA - rental
Disc Read:
validPS2: Yes, discType: DVD, Dual-Layer
Layer Break: 1317056, Total Length: 2594960

This is interesting, because this hints that GT4 is actually pretty small (5GB roughly). It would almost fit on a DVD5 by itself.

HDL does more extensive detection than I do currently, so of course it will be more accurate. I am just returning what the PS2 claims the disc is, plus my extra detection for DVD9 discs.

UPDATE:

I got proper access setup to the CVS server now (the build bundled with MacOS X 10.2 is broken, whoops), which allows only anonymous read-only access through the pserver. If you have a unix-like (Linux, BSD, MacOS X) environment, or the command-line client for Windows, you can checkout the latest source with the following commands:

cvs -d :pserver:anoncvs@mzed.homeip.net:/usr/local/cvsroot login
(It will ask for a password, just hit enter)
cvs -d :pserver:anoncvs@mzed.homeip.net:/usr/local/cvsroot co HDDemolisher

If you use a graphical client, the server is at: mzed.homeip.net. The user is: anoncvs. There is no password (blank) for the user.

I have taken measures to ensure that this machine can remain available, and secure enough that I can be confident in making it public. Do not abuse the server, or I will wind up taking it down and finding some other way to host it. Be good. ;)

ADDITIONAL ADDITIONAL:
Another thing you can find in the CVS tree is the latest ELF builds. Under the dist/ directory is the latest build I have produced before committing changes to the CVS tree, so it will always be as recent as the source code available.

spawner_brasil

03-06-2005, 06:39 PM

If we can install Final Fantasy XI Online on the HDD and run from the HDD without the disc using a Final Fantasy XI Online copy and a modded PS2 (Matrix or DMS), okay, we will have the problems to connect, but I´m just talking about installation of FFXI Online and execution from the HDD, why we cannot execute programs like HDLoader from the browser? The only way is having ELFs hidden for the browser and executing from Memory Card or hd0: using DMS Explorer... Why not execute from the browser if with the ATAD patch from the modchips, it is 100% compatible? I hate that.

leidout

03-07-2005, 06:33 AM

spawner_brasil

03-07-2005, 09:21 AM

No.
Sorry, but for me it is lack of ability.

agentboolen

03-07-2005, 11:09 AM

Phreaker47

03-07-2005, 11:49 AM

This is interesting, because this hints that GT4 is actually pretty small (5GB roughly). It would almost fit on a DVD5 by itself.

HDL does more extensive detection than I do currently, so of course it will be more accurate. I am just returning what the PS2 claims the disc is, plus my extra detection for DVD9 discs.

Can't help much here other than being a cheerleader of sorts, but I can confirm that DVD Decrypter also detects that same size for GT4 USA:

(From my original US copy)

Sectors: 2,594,960
Size: 5,314,478,080 bytes

But I would also caution against using this particular game as a benchmark for your application, due to some nasty protection it has (a working backup solution has not yet been found and it's been out well over two months in Japan.) I don't think anyone has even successfully decrypted the files yet that we know of, and for all we know it could have a spoofed TOC (polyphony had said a while back the game would be 9+ gigs but maybe they were blowing smoke.)

Anyway, probably something like Xenosaga EP1 or Champions of Norrath would be better test subjects... but of COURSE, we'd also like to see what could be done with GT4. A lot of us legitimate owners of the game would love to run it off HDL, especially since they ditched online anyway (LAN games through KAI will still work!)

Keep up the good work.

spawner_brasil

03-07-2005, 11:54 AM

Nothing. I think is good to run HDDemolisher from ps2link or pressing L1 or using ps2menu/PS2OS or maybe booting a CD. Nothing... yeah. I really don't know why turn on my ATAD patch.

Krevnik

03-07-2005, 12:42 PM

GT4 does pose problems, BUT... they cannot get away with hiding the files outside a valid ISO volume, especially since the CDVD.IRX requires data to be on valid volumes or it will not read the sectors. Hence why you see things like VFS used on Star Ocean. So there is probably just some really nasty protection ripper teams haven't found, but not really my bag...

I have gotten a BIOS dump from my V9 PS2 and have been doing a little disassembly. I have managed to reverse engineer the process that starts the IOP reset from the EE, which is f'ing great news. The IOP sets up a small thread which is used to setup the command handler for the IOP reset, and then waits on a flag to be raised by the command handler. Once the flag is raised, it knows that it just received a reset command from the EE and calls modload::ReBootStart(). So every route to reset the IOP manually goes through modload::ReBootStart(), probably to assist in loading the new modules (since modload is used for all of that).

This means I should be able to whip up a module that either reacts to the IOP reset, or a patch to modload which will allow me to redirect part of the loading process. I have to do some major work with modload before I can understand which route would be best. In the CVS, I will be writing up some files which appear to be source code files, but are C/C++ versions of my BIOS reverse engineering to help provide understanding of what various parts of the BIOS are doing.

ps2feen

03-07-2005, 05:11 PM

keep up the good work Krevnik. i hope to see this powerful applciation when its done..

Krevnik

03-07-2005, 10:37 PM

I have successfully managed to get almost all of the functions in modload related to the IOP reset analyzed. The only one I haven't analyzed is rather long, and I already know what it does, so I didn't bother going through it just yet. (The function does a simple search of the BIOS for the start address of the IOPBOOT 'module')

I am working on the REBOOT module tomorrow, in order to figure out everything it actually does. It is pretty small, so it shouldn't be too bad. IOPBOOT actually handles the reset itself (since it is triggered in ROM directly, it can load other modules safely, using the IOPBTCON file list), and will be the focus of my efforts for awhile (when not writing code for the HDD and the like). Once I understand what is going on, I can decide the best course of action.

Golden Helmet

03-08-2005, 06:30 AM

Stefy2

03-08-2005, 06:42 AM

for you to know the pfs drivers are bugged and can't support more than 4gb partitions, as far anyone correct this it's not possible

zabolyx

03-08-2005, 08:59 AM

but the design of the PS2 allows new versions of drivers to load from a game.... would it be possible for the CDVD.IRX they supply have code that allows for "illegal" drive access....

romz

03-08-2005, 08:59 AM

I have a question about your app...

I asked Sjeep the same question previously, but got no answer...

Is it not possible to store installed games on the HDD as raw files, in directories, as opposed to ISO image partitions?

Basically, by doing relatively simple file copies?

If I've missed something, then let me know, cheers ;-)
Why do you ask for it? Do you really think every game can be copied by "by doing relatively simple file copies"? :) Look at the disc contents of some games like Final Fantasy X or Ratchet & Clanck - only some small files can be found there. But actual disc size in much bigger, not just few megabytes. Many games use streaming and have assumptions about physical file location on disk (lba).

Golden Helmet

03-08-2005, 09:32 AM

Krevnik

03-08-2005, 10:46 AM

Ah, but as a PS1 and PS2 developer, 99.9% of the time the LBA stuff doesn't matter a jot. The only time it matters is when ensuring a movie is of a certain length, a form of protection if you like. IE in the code you make sure you specify the length of the movie, if the movie is replaced my a shorter version then it will not play.

If you were a developer, then you wouldn't have to ask the question if we could do it, now would you? And if it was 99.9% like you say, then we would probably have a higher compatibility rate with HD Loader right now. Not all incompatibilities are because of protections.

The actual start position on the disc is pretty much irrelevant.

How do you think PS2 games are developed? on HDD's :-)

There is the problem, the start position is not irrelevant in some CD games. The reason being that direct access to the LBA of the file is faster than loading the TOC, searching it and then seeking to the LBA. And loading/seeking from CD is pretty slow. (Hence the LBA table tutorials on backup threads) Another thing is that while it is developed in a manner similar to PS2Link for awhile, they do have to start testing on final media at some point, and this is when the LBA tables, protections and the like are inserted into the code.

With regard to your actual question through: No. No pfs. First off, I want this to be compatible (mostly) with how HD Loader does things. Secondly, using an ISO filesystem retains more compatibility and simplifies my code (since I can forgo having HDD libs on the IO Processor, just the ATAD/DEV9 libs). So when a read starts, I can just add the 4MB offset of the odd partition header HD Loader uses, and then read. (Not to mention some disk checks work by doing raw reads without searching for a file)

romz

03-08-2005, 12:36 PM

In spite of games developed using hdd many of them change into weird forms in retail version. As far as I know it is serious task to make game work from cd or dvd even if it works fine from hdd already. Besides low level file system support in "cdvdman" module operates with logical sectors so it's easy to work with disc image or at least I think so. Such functions as sceCdSearchFile() and sceCdLayerSearchFile() return lsn for specified file which will be used with sceCdRead(). Why should HDL software use files instead of images then?

For example take a look at Final Fantasy X-2 (JAP):
3700636 - SLPS_252.50
57 - SYSTEM.CNF
7353 - MCSERV.IRX
90533 - MCMAN.IRX
151721 - IOPSOUND.IRX
27173 - LIBSD.IRX
43861 - PADMAN.IRX
6161 - SIO2MAN.IRX
9197 - DEV9.IRX
11781 - ATAD.IRX
30405 - HDD.IRX
49665 - PFS.IRX
247 641 - IOPRP234.IMG
13 files 4 376 184 bytes
There are only 13 files visible on disc. So do anyone think this game fits in 4 megabytes ? :)

Krevnik

03-08-2005, 03:59 PM

Well, just did another large commit to the CVS, no new ELF yet (other than some bloat added by merging the atad, hdd, and dev9 IRXs into the ELF). This new commit has the entire REBOOT module disassembled and turned into a nice readable C-form. I also did a little work on SIFCMD and created a decompiled form of sceSifDelCmdHandler(). Very interesting, as this means I might be able to hijack the entire reboot process via a single patch to modload's export table, and patching the Sif Command handler table to use my module instead of REBOOT. The only catch is how to let the IOP reset, and then reload all my stuff afterwards.

One method that comes to mind is that I could get my routine setup on the EE, catch the IOP reset... and then unhook my IRX from the IOP and repatch it to function correctly. Then I could trigger an interrupt to the EE which my routine sits, which will then continue the IOP reset on behalf of the game, and goes to work in interrupt mode. Since the game still needs to sync, I can sync while in interrupt mode, cause my modules to get loaded, and then do what I need to do to exit.

The only really tricky part to all this is that I cannot do my work entirely inside the IOP. Theoretically, I could, but that would mean implementing a version of IOPBOOT from scratch and overriding the entire reset functionality with my own, but it will be the most compatible way to trick it... ouch.

Golden Helmet

03-08-2005, 04:01 PM

I only asked guys :-)

Your reason for using ISO, and explanation were enough to answer my original question, thank you.

romz

03-08-2005, 07:56 PM

Just a note: reversed module sifcmd.c can be found in file "modules08.zip" at ps2dev.ps2-scene.org

Krevnik

03-08-2005, 09:02 PM

Funny thing is that there was also a reversed reboot.c in there. ;)

Anyways, this was still good practice for me, and I managed to write up an IRX, that is fairly large because of all the IRX import tables, but still works. Anyways, what it does is hijack the reboot command coming from the EE, and then starts up the reboot anyways, sending an interrupt over to the EE first. This interrupt is then caught and handled on the EE (although the ELF currently locks up at the end of the SBUS handler).

For those curious, you can checkout this updated source now, which includes the latest ELF which demonstrates that it is indeed possible to hijack the IOP Reset command and insert your own functionality in place of it. Currently, the only difference is that my code uses a semaphore instead of event flags to pass information, and that I trigger an SBUS interrupt which is caught by the EE and starts executing code there (and then promptly FREEZES, whoops). So, I can actually send some sort of signal to the EE and execute code there, and I can even sync up to the IO processor as well. The catch really seems to be that my function doesn't quite work right. Anyone know a little more about the SBUS interrupt handler routines and if they need to have an Assembly wrapper to function properly?

HypERSoniC

03-09-2005, 08:29 AM

Zer0-X

03-09-2005, 08:52 AM

IMO something like HDL would be way easier than a mediaplayer. You only need to understand how to pass data from place a to place b. With mediaplayer it's all about getting the best working codecs and then adapting them to a new environment and lots and lots of optimizing. :)

<G>

03-09-2005, 09:25 AM

I love how some stupid n00bs are asking questions in this thread completely unrelated to any kind of dev in this project. Keep that crap out of this thread from now on.

Krevnik

03-09-2005, 09:55 AM

hey krevnik.. this hdloader is too much wizzely doodah, way to hard man. i hope you know how to reverse engineer,, cause.. man.. hdloader is very hardcore shit. you could make a killer media player. dead set. and that would kick ass. nice open xvid player... you could borrow from ps2mp3. man.

Uh, you haven't been paying attention, have you? I reverse engineered the REBOOT module, a chunk of MODLOAD, and a piece of SIFCMD (mostly for verification)... BEFORE looking at the stufff [RO]man has already done. I really should have checked out his work first before spending 2 days dumping and disassembling.

Not to mention that I have /successfully/ hijacked the RebootByEE command for my own purposes. From here, I have two choices: I can write up my module loader on the EE which I am currently testing out, or I can write up a new reboot sequence on the IOP. If I do the second one, it will cause problems with games that attempt load an IOP image, if I do the first, it will cause problems with games that clear out kernel memory. The magical third option is that I create a sort of reinforcing duo that allows me to reload the EE code, triggered by the IOP before the reset, and the EE code reloads my hijacker. I could also patch my EE code into the game's memory space on the stack or the heap. (depending on how big the stack can be) This way, the game can't clear it out and I get higher compatibility than HDAdvance.

This is hard, yes... but not impossible. Talking to the HDD for ripping games and displaying what is on the HDD: really easy. Ripping games: easy now, since I have working DVD9 detection. Playback? Well, that is trickier, since I need to load two modules at least: CDVDMAN (my own version), and ATAD (from ps2sdk). The hardest part of all is actually getting the modules injected during a game-induced reset. I think I have that one in the bag now, it is just a matter of time. ;)

Phreaker47

03-09-2005, 11:22 AM

Correct me if I'm wrong, but doesn't HDL use memory reserved for PS1 emulation to hide itself and remain resident during game execution, or am I just way off here?

romz

03-09-2005, 12:17 PM

Krevnik

03-09-2005, 05:07 PM

To Krevnik:

Are you sure all of this allowed in SBUS interrupt handler routine ?

void ee_sbus_handler(unsigned int cause) {
dprintf("Interrupt Found.\n");
_ee_sbus = cause;
// ee_load_modules();
wait_until(SifIopSync());
dprintf("Synced up, ready to go!\n");
}

Especially I am afraid of doing something like loading modules before the "SifIopSync()". May be should sbus interrupt handler just set event flag and wake up special thread on EE?

PS: Detailed information about memory layout on the EE and the IOP can be extremely useful.

Realize that the call is currently commented out, and I added the sync /after/ I commented out the ee_load_modules call. That particular function is REALLY messy and pretty poorly written. I mostly just wanted to make sure things worked properly. Oddly enough, the EE locks up after the interrupt finishes, so something dicey is happening, but I am not sure exactly what yet. It is quite literally next on my list of things to do.

With regards to hiding itself, that only works if the game decides not to wipe that area of RAM out. As posted earlier by someone describing the 3 compatibility modes, some games wipe the memory to get rid of any resident software like cheat apps (but a lot of cheat apps do their work and then sit idle, so it doesn't really affect them). However, we attempt to stay hooked in, so the game could quite nicely wipe us from existance at the worst possible time.

Actually, it isn't crashing in the SBUS handler, it is locking on SifInitRpc()... and I generate a TLB exception on the EE reading from the mapped IOP RAM after the reset, so I am not causing a clean reset... hmmm....

FINAL RESULT: Figures, it was the dprintfs that were crashing the interrupt handler when it went to return. Whoops. It shouldn't be too hard to use a Semaphore to trigger a thread, since I can give it a high priority, and then disable interrupts to prevent the game from getting control back. Load the modules, and then re-enable interrupts once I am done.

Krevnik

03-10-2005, 12:25 AM

Some interesting tidbits: HDAdvance only seems to use the Vblank interrupt, where all it does is increment a counter. Seems that I am taking an approach fairly different than Sjeep and others have. I don't know if this is good or bad yet. ;)

HypERSoniC

03-10-2005, 08:26 AM

hdloader..

sorry everytime i hear you say hdadvance i shudder :)

v1p3r

03-10-2005, 10:45 AM

How to get the latest CVS version of HDDemolisher on Windows:

NOTE: Peerguardian can cause problems when installing cygwin. Disable it now if you use it.

Installing Cygwin.

1. Grab the Cygwin installer here (http://cygwin.com/setup.exe)
2. Run setup.exe, and choose to 'Install from Internet'.
3. Choose where you can cygwin to be installed, and where packages should be kept for future use.
4. Choose a mirror close to your physical location.
5. Under the 'Devel' node, choose to install 'cvs'.
6. Install and run :)

Using CVS

1. Paste
cvs -d :pserver:anoncvs@mzed.homeip.net:/usr/local/cvsroot login
into the Cygwin window (click on the top left icon then Edit to paste into a command line).
2. When asked for a password just hit enter. Don't wory if you see an error, that always seems to happen the very first time you use CVS.
3. To download HDDemolisher, paste this command:
cvs -d :pserver:anoncvs@mzed.homeip.net:/usr/local/cvsroot co HDDemolisher
and wait :)
4. When its done, look in cygwin_dir\home\username\HDDemolisher\dist for the very latest build of HDDemolisher.

:dance:

zabolyx

03-10-2005, 10:50 AM

There is the problem, the start position is not irrelevant in some CD games. The reason being that direct access to the LBA of the file is faster than loading the TOC, searching it and then seeking to the LBA. And loading/seeking from CD is pretty slow. (Hence the LBA table tutorials on backup threads) Another thing is that while it is developed in a manner similar to PS2Link for awhile, they do have to start testing on final media at some point, and this is when the LBA tables, protections and the like are inserted into the code.

Correct... if you think about it a pressed disc is the same as a computers RAM... all memory locations are mapped out... so a frame reference is much more acurate and faster than using the TOC to access files and data.

Say all the games levels are contained in one 200MB file... would it be faster accessing the literal location for the second level than to start at the beginning of the file an work your way through it. Save on memory usage and time.

But with the games that have been shrunk to take up less space on the HDD this doesn't seem to be the case.

I retrack that statement... seems that some games with inflated LBA have problems being read after changing it.... Looks like possible direct frame access to me and not likely disc protection. So it would seem that some developers use it and others don't.

I still think that the OS idea is the way to go to improve development...

omegatron

03-10-2005, 12:05 PM

i understand that hdloaders source has not and probabley will not be released.... BUT have you tried to contact sjeep to see if he can give you some advice. He doesnt have to release his source code just help you work out some of the bugs

dlanor

03-10-2005, 12:48 PM

Omegatron:
First, it should be very clear (from other content in these forums) that Sjeep would not be interested in helping anyone else to compete with his commercial venture...

Second, I think it is also clear from the content of this particular thread that Krevnik is not really in need of such help, and might even find help from that direction a hindrance, since he would then also risk 'inheriting' the bad feelings many in the 'scene' have towards Sjeep. It is probably better to make (like he has) a clean start without any direct connections to the old HDLoader project.

Krevnik:
I have followed this thread from its start, and have just checked out the HDDemolisher module from your CVS. I think you are digging into the PS2 functionality in exactly the right way to achieve success with this project. A good grasp of the interrupt interactions of the PS2 processors is definitely a key element in what you need to do, and from the work you've already done I'm sure you'll shortly get around the present difficulty.

All in all, I'm quite impressed with the level of ambition and rapid progress of this project.
(And being impressed is not really something I'm accustomed to... :) )

Best regards: dlanor

omegatron

03-10-2005, 01:04 PM

Krevnik,
...
....
.....
I really hope you didnt take offence to that. You dont need help, and i hope that this is put on the presses (as they say) soon as possible which im sure it will be

HypERSoniC

03-10-2005, 05:14 PM

Krevnik

03-10-2005, 11:13 PM

Wow, I forget to check and a lot happens in the day... so I will address things as they showed up:

CVS: You /can/ use a graphical client, just make sure it is pretty up-to-date. I built the CVS server from the most recent sources simply because the one bundled with OS X 10.2 is borked, and doesn't have a functional pserver.

LBA Tables/etc: You are right zabolyx, some developers use them, and some don't. A great example is Worms Forts. It is a straight port from the PC, so they didn't bother to add LBA tables and the like, even though it would have really helped (it uses a ZIP file for the data, ICK! Compressed loading!). Also, a lot of DVD games don't use it because of the lower seek latency and higher throughput. (Worms3D is on a DVD in US simply for better throughput from the disc)

About HDLoader/HDAdvance/Sjeep: Unfortunately, I missed out on all the drama surrounding the two projects and Sjeep (I don't even know if he was the guy behind HDL, HDA or if he is involved with both!). However, I am more inclined to agree with those who are hinting to stay away from direct involvement with either project. Two reasons for this: I am inclined to agree that Sjeep, while pretty helpful to the community is probably not going to be very helpful to someone attempting to create something of this nature. What is more bizarre is that PS2Dev relies on Sjeep's work and tools, and their forums ban discussion of work related to a project such as this. Not sure the reasons behind it, but oh well. The other reason is just that if I take a different approach to the solution because I am not aware of the existing solution, then I might wind up making something that works better with problem games that HDL/HDA can't work that well with. Then again, I might not. However, experimentation is key, and very helpful to myself and others in the long term as that helps the general understanding of the hardware, rather than grabbing parts of a solution already made.

Regarding some of the analysis I /have/ done on HDA (since it is the only one I have accessible right now): It contains a segment of code which is run normally, and another segment which is copied into 0x88000. (Programs normally start at 0x100000, giving it about 512K to play with for the module) Interestingly enough, HDA /appears/ to use a EE-only solution which grabs the DMA request /before/ it leaves for the IOP. Interesting, but keep in mind I may be off base. HDA is nearly impossible to disassemble without doing a lot of work as part of a full-time effort to manually split modules out and analyze them. Still, an interesting approach if true.

Regarding the current Progress: Development might start to slow a bit after this week, as I am also in the middle of the post-college job hunt. It is just that I have been really itching to get some of the key tech issues out of the way. However, I have a build that isn't incorporated into CVS that works. It is a HUGE mess right now, and I should do some more testing to see if I can streamline things for expandibility in the future before committing the changes. Plus, during my bugfix spree, I managed to tie my module directly into REBOOT, which I would like to see if I can untie it again. From here (working interrupt), the interrupt handler could quite easily suspend all threads except mine, and signal it. OR, I give my thread ultimate priority: 1. Either way, when I signal it, it can't get interrupted by the game thread. Then just sync back up with the IOP, load up modules (unloading some first, if needed), and then disconnect.

Regarding future ideas: In an earlier post, I described how I considered a few methods where I could install myself into the game itself. It hit me last night that since you can override syscalls, I can get an idea on how the game maps the 32MB of RAM. If I do this, I can find out if there is enough space between the heap and stack. I might be able to experiment with an installer which uses 0x88000 initially, but then re-installs just under the stack. But that is once it starts actually loading games off the HDD.

crazyc

03-11-2005, 12:18 AM

>HDA /appears/ to use a EE-only solution which grabs the DMA request /before/ it leaves for the IOP.

It might do some patching to the executable before running it, but for the most part, it seems to me to be a straight up replacement of cdvdman.

>HDA is nearly impossible to disassemble without doing a lot of work as part of a full-time effort to manually split modules out and analyze them.

It's not that hard. Anyway, it cool that you are able to inject code into the IOP after SifIopReset. The way HDloader does that seems to be black magic.

Stefy2

03-11-2005, 01:22 AM

sjeep isn't behind hd advance. hdadvance is a copy from hdloader with changed text and gfx and with patches released in this site, then putted on two disc (cd and dvd) with a large toc and here you are hdadvance, anyone can do an hdadvance with the available tools. and after hdadvance there is bs2ownz and one of their sponsor.

dlanor

03-11-2005, 01:37 AM

thats not true... sjeep isnt the robot that everyone makes him out to be :p Im sure he would be happy to help if you asked.
I should perhaps make it clear that I have no hard feelings toward Sjeep myself. I've never had any direct contact with him at all, so all I know about him is either by hearsay or from contact with his works. (Mainly HDLoader and some PS2Dev stuff, all of which seems good to me.)

However, the fact remains that many others in the PS2 'Scene' do have hard feelings against both Sjeep and his HDLoader project, and I too must agree with one complaint. It's never a good idea to use open-source libs to launch a closed-source commercial project.

The new project is both non-commercial and open-source, which puts it in an entirely different class, as I see it.

Best regards: dlanor

romz

03-11-2005, 04:15 AM

>HDA /appears/ to use a EE-only solution which grabs the DMA request /before/ it leaves for the IOP.

It might do some patching to the executable before running it, but for the most part, it seems to me to be a straight up replacement of cdvdman.
magic.
I believe that HD loader not just replaces "cdvdman" module but rather hooks some its functions. There are two reasons that make me think so:
1) cdvdman contains some some functions like sceCdReadClock, sceCdRI, etc which aren't closely related to C/DVD emulation;
2) other module "cdvdfsv" (which supports libcdvd calls from EE) not only uses "cdvdman" functions but also accesses C/DVD hardware registers.

romz

03-11-2005, 05:31 AM

Btw, one interesting fact.

I think most of you know game "Tenchu: Fatal Shadows". I personally didn't install it on my HDD but I saw a patch for that. I realized that patch simply replaces "cdvdman" string in game's main ELF with zero bytes so I started to learn what's going on there.

I discovered very small IOP module inside the program file and "cdvdman" string actually was no other than part of dynamic links to "cdvdman" module's function. The game loads that irx via sceSifLoadModuleBuffer to do some odd job. That small module just calls sceCdSC(-9, &param); and returns some exit code which depends on result. Return value of sceCdSC with 0xFFFFFFF7 а code is just version of "cdvdman" module.

It is definitely interesting why the game do this. May be it's an anti-HDloader trick or may be it's just some kind of weak point in HD loader "cdvdman" emulation&hooks.

Stefy2

03-11-2005, 07:19 AM

dlanor you should think also about other thing then:
-o2loader it uses barely all opensource things and it's "commercial" being usable only with o2 and being closed source
-ps2reality media player closed source but at least not commercial
-someone can blame me for telling this but it's the truth boot manager is closed source and it's "commercial" also it because it don't work on other/no chip.
-exiting from the scene of the ps2 a simple example lindows or linspire it's completely closed source and completely commercial, you can't download a installable version without paying.

and anyway sjeep used for the most part it's own thing or thing where he was involved and just followed AFL licensing terms, as you can see in toxic os there is a disclaimer file with all the people involved in the project used

this isn't going to be a flame post, if this seems so i'm sorry. i was only going to precise some things, and i can be wrong after all...

Krevnik

03-11-2005, 10:14 AM

When I said HDLoader seems to use a EE-only solution, I meant from the standpoint of catching the IOP reset command. My way isn't the only way, if you can catch the command on its way out and raise a signal to a local thread on the EE, you can achieve the same effect. I just chose a method that I could expand to replace the entire reboot if I decided to do so, even though I very likely /won't/ do such a thing.

With regards to the information on Tenchu: That is rather interesting, since I don't have the game, I can't take a peek myself, but do you know what sort of route it takes with that version code? Depending on the version, does it load a new one? I am just curious as to how it responds to unfavorable version codes, so I can think of ways around it.

Well, I could probably write a module that patches only the CDVD hooks it needs to to function properly... HOWEVER, it appears HDAdvance does have a fully functioning CDVDMAN in the ELF, so it might be overriding it completely. I could also gain better compatibility by hooking into an existing CDVDMAN (and modload to notify my module of a new CDVDMAN being loaded), letting the game do what it wants with the module, but then just overriding it on the IOP exclusively. However, that is much more work.

crazyc

03-11-2005, 10:29 AM

>1) cdvdman contains some some functions like sceCdReadClock, sceCdRI, etc which aren't closely related to C/DVD emulation;

Some of those functions are reimplemented in hdloader. They even missed that cdvdman contains two copies of sceCdReadClock thus causing some timestamps on the memory card to be wrong. If hdloader just patched cdvdman, that problem never would have happened. Also remember, the clock, ilnkid and machineid are located in the mecacon.

>2) other module "cdvdfsv" (which supports libcdvd calls from EE) not only uses "cdvdman" functions but also accesses C/DVD hardware registers.

I could be missing something, but a quick look at hdloader's cdvdfsv, I don't see any place where it accesses the mecacon registers.

>That small module just calls sceCdSC(-9, &param); and returns some exit code which depends on result.

Interesting.

>When I said HDLoader seems to use a EE-only solution, I meant from the standpoint of catching the IOP reset command.

Oh, I misunderstood.

Krevnik

03-11-2005, 01:49 PM

Another CVS commit a little earlier today, this one freezes the hdreboot.irx module. It works, it does what it needs to do, and I am not gonna touch it again unless I absolutely have to. This last mess I waded through was pretty nasty and it took me 30 minutes just to clean up the code once I was finished. Ick. The good news is that the entire module is now under 5K in size, without stripping the symbols. Not bad, eh?

I think the next step is to start interfacing with the HDD (finally), and get things together for an actual interface. Initially, it will be compatible with HDLoader, but I may decide to remove compatibility with it at some point so that I can have higher compatibility rates and more stable operation. Any input on if I should eventually split from HDL-compatibility or should I expend the extra effort to keep it?

omegatron

03-11-2005, 02:01 PM

ok i just used demolish.elf and it scrolls the saying
Disk Read:
validPS2: Yes, disk type dvd, dual layer then a block symbol
Layer Break 2050738, Total Lenght 4062160

is that correct the disk was champions of norrath

Krevnik

03-11-2005, 02:12 PM

Well, it is an older elf that you used, but yeah... Champions of Norrath is a DVD9, and that proves it read correctly. I can pretty much say with confidance that DVD9s will be supported in this app.

vinniev2222

03-11-2005, 02:21 PM

Krevnik - When you talk about the taking out "HDL compatibility", do you mean that once we load your app, we would need to completely reformat and reinstall the games on the HDD? If so, maybe initially that should be there, simply so more folks would be willing to try and test your app without having to wipe their HDD game collection thus far. If you can increase the game compatibility while keeping HDL format there, (so if someone wanted to switch between the two I guess?), that may be a drawing feature for alot of people to use it. However for myself, I would rather see you do your own thing, especially if it addresses the game compatibility issue, once a good stable environment is acheived. Awesome project man. Wish I could go this far

Krevnik

03-11-2005, 02:29 PM

Angelus3X

03-11-2005, 02:32 PM

I suppose I wouldnt mind reinstalling all my games to a HDDemolisher partition but I think initially (if possible) it would be good to retain HDL compatibility but if that contrains your app at all I would just drop the HDL compatability!

omega_sc

03-11-2005, 02:38 PM

Hey Krevnik, awesome work.
hmm let me see... seems to me that you're claiming you can keep HDLoader games, and still be able to install Demolisher ones on the same HD without reformatting right? Then it would be just a matter of using the correct app to play the games (did I get it right?). I think you can wipe hdloader compatibility out and keep all effort on your own format/ideas.

dlanor

03-11-2005, 02:42 PM

I think the next step is to start interfacing with the HDD (finally), and get things together for an actual interface. Initially, it will be compatible with HDLoader, but I may decide to remove compatibility with it at some point so that I can have higher compatibility rates and more stable operation. Any input on if I should eventually split from HDL-compatibility or should I expend the extra effort to keep it?
My opinion on this subject is inevitably split, since there are some different forms of compatibility involved.

When it comes to the physical disk format, and the internal structure of the partititions, I see no reason at all to drop backwards compatibility. But even so, that standard may need to be extended to cover true PS2 dual-layer discs, as they have two iso file systems. ('Flattening' methods as presently used will not work with all games.) Apart from that change (if/when), I think the partition format should remain as-is. One big bonus from that choice would be automatic compatibility with games already installed, as well as with all the third-party HDL installers (HDL_Dump, WinHIIP, etc).

When it comes to internal methods of HDDemolisher, there is absolutely no reason to maintain ANY compatibility at all with HDLoader, except coincidentally, where similar needs may require similar methods. This is the area of change that can bring the greatest improvements, so don't let any old design hold you back.

When it comes to the user interface, I have no strong feelings either way. The current methods suffice, but there is always room for improvement, and a pretty obvious one would be to add LaunchELF-like capability to browse PS2 media and launch ELF programs. But that's for a longterm wishlist, when all the main stuff (replacing HDL functionality) has been implemented.

One more thing that needs to be mentioned is the mode flags. As long as they still remain relevant, their meaning should not be altered:
1 = Attempt to emulate true CDVD timing (slow but compatible)
2 = Allow huge disks (ignored/always allowed with your code !?!)
3 = Attempt to hide patcher from game code

Most likely these flags are stored somewhere with room for more bits that are currently unused, so if you need mode flags of your own, then those are the bits to use for them. However, adding such could break some of the compatibility with HDL installers, so that should be taken into consideration.

Best regards: dlanor

romz

03-11-2005, 02:48 PM

With regards to the information on Tenchu: That is rather interesting, since I don't have the game, I can't take a peek myself, but do you know what sort of route it takes with that version code? Depending on the version, does it load a new one? I am just curious as to how it responds to unfavorable version codes, so I can think of ways around it.

This is what I know as this moment:
IOP module code is pretty simple amd module name appears to be "ck01"

int start (int argc, char *argv[])
{
int *param;

return (_sceCdSC(0xFFFFFFF7, &param) == 1) ? 5 : 1;
}

At EE side there is code like this:

int run_ck01(int *result)
{
int rc;

/* some initialization code skipped */

rc = sceSifLoadStartModuleBuffer(iop_mem, 0, 0, result);
*result &= 0xFFFFFFFC;
sceSifFreeIopHeap(iop_mem);

return rc;
}

Function run_ck01() is called from main() and these is a branch much later in the main() which depends on whether or not IRX exit code (result) is equal to zero.
What's going on later I can't understand yet. But I don't think the game tries to reload "cdvdman" since its in IOP image.

"cdvdman: from "rom0" doesn't support sceCdSC() call with such code while "xcdvdman" from "rom0" does. All the "cdvdman" modules from IOP images I dug in support for sceCdSC call with 0xFFFFFFF7.

dlanor

03-11-2005, 02:59 PM

Well, it wouldn't reformat the drive, but it would make HDL-loaded games inaccessible and vice versa. There are a few things I would like the freedom to try, but keeping HDL support in the long term would only make a mess out of where I have to store things when I start breaking away from HDL's feature set. I intend to use HDL's layout for now, but I am only doing that to speed up initial development until I have desires/needs outside of what HDL's database can currently do.
I'm not sure exactly what you mean by 'database' here.

If you mean some internal database of each game, to keep track of stuff within that ISO, then we do have a real problem.

If you mean the collective database where HDL keeps track of all the games, then I'd say there is no problem at all. Unless I'm misinformed that is erased anyway when using PC-based installers, and then rebuilt when restarting HDL, so that database has nothing to do with the compatibility of the game images installed.

Of course, if you truly see a real need for changes that break compatibility, then so be it. But personally I don't think we should assume that this will be necessary until such a need is known.

Best regards: dlanor

omegatron

03-11-2005, 03:07 PM

i think that launchelf is about the best ps2 app ( no chip) out thier so if can also boot elf how could it ever be beat LOL

Krevnik

03-11-2005, 03:10 PM

When it comes to the physical disk format, and the internal structure of the partititions, I see no reason at all to drop backwards compatibility. But even so, that standard may need to be extended to cover true PS2 dual-layer discs, as they have two iso file systems. ('Flattening' methods as presently used will not work with all games.) Apart from that change (if/when), I think the partition format should remain as-is. One big bonus from that choice would be automatic compatibility with games already installed, as well as with all the third-party HDL installers (HDL_Dump, WinHIIP, etc).

Here is the problem. The current database layout for HDL/HDA is restrictive, and I already have to resort to some other way of storing information for my database. Specifically, I don't have any information on how to flag a partition as being a DVD9 partition to make my search easier. (HDL seems to store all its information on disc type/etc in its database) Also, the partition format is kinda bizarre because not only is each partition an ISO by itself, but it has a 4MB header whose contents I am not entirely familiar with yet. Conforming to this is tricky at best. As a compromise, I might simply have support for importing from HDLoader, but not for exporting /to/ HDLoader. That way games that need my special attention to work (DVD9s, games that might work under some new compatibility mode) don't wind up in some horrid mess.

When it comes to internal methods of HDDemolisher, there is absolutely no reason to maintain ANY compatibility at all with HDLoader, except coincidentally, where similar needs may require similar methods. This is the area of change that can bring the greatest improvements, so don't let any old design hold you back.

Not planning on it.

romz

03-11-2005, 03:13 PM

>1) cdvdman contains some some functions like sceCdReadClock, sceCdRI, etc which aren't closely related to C/DVD emulation;

Some of those functions are reimplemented in hdloader. They even missed that cdvdman contains two copies of sceCdReadClock thus causing some timestamps on the memory card to be wrong. If hdloader just patched cdvdman, that problem never would have happened. Also remember, the clock, ilnkid and machineid are located in the mecacon.

>2) other module "cdvdfsv" (which supports libcdvd calls from EE) not only uses "cdvdman" functions but also accesses C/DVD hardware registers.

I could be missing something, but a quick look at hdloader's cdvdfsv, I don't see any place where it accesses the mecacon registers.

"cdvdman" modules from IOP images don't support all that functions thier "old brothers" from system BIOS can provide. I noticed versions from redistributable IOP images don't allow to write iLinkID or MachineID and call "ReadAudio" function or set new system time also. But they have new functions for DualLayer DVD support and some more interesting things. So its easy and hard to reimplement "cdvdman" functions at same time since program rather aimed just to support games. :)

Just look at "cdvdfsv" a little closely. It can be found even in few RPC handler callbacks. Module accesses memory locations 0xBF402005 and 0xBF40200A at least.

romz

03-11-2005, 04:05 PM

My thoughts about compatibility with HD loader&HD Advance:
I believe people do not want a revolution.
1) Many people have decades of games already installed on their HDDs, so who wants to reinstall them all ?
2) There are many game compatibilty databases (supported games, HDL modes, patches and so on) around the world, so who wants to use totally new unknown application then ?
3) Many third party software for HDL like FapLink and others are available already but why it can't be used with new application ?

I think Demolisher should support all HD loader installed games as well and do it as good as it can be done at a moment. All extra features should be implemented to not interfere HD loader's duty. Users should have option to install games in two modes: compatible with HDL for most games and so called native mode (if it really needed) for games incompatible with HDL by default such as DVD9 games.

If someday most people consider to use only Demolisher instead of HDL then it will be time to split Demolisher from HDL compatibility by starting new branch. But nowdays HD loader users community is too large and can't be ignored.

PS: Personally I even want Demolisher's support for lba48 will be optional feature since I use old version of HD loader and want my HDD to be compatible with few games which has native hard drive support. ;)

crazyc

03-11-2005, 08:16 PM

int start (int argc, char *argv[])
{
int *param;

return (_sceCdSC(0xFFFFFFF7, &param) == 1) ? 5 : 1;
}

That's weird, hdloader's impementation of sceCdSC (assuming its cdvdman:50) is

int sceCdSC()
{
return 1;
}

Just look at "cdvdfsv" a little closely. It can be found even in few RPC handler callbacks. Module accesses memory locations 0xBF402005 and 0xBF40200A at least.

I see that Sony's cdvdfsv accesses those registers, but I don't see where hdloader's does.

zabolyx

03-11-2005, 09:46 PM

Here is an idea.... a running database file that can be downloaded for the program to use for auto shrinking ISO images... the database tells the program what files to remove and so on.... thus saving automagically craploads of HDD space...

romz

03-12-2005, 08:38 AM

That's weird, hdloader's impementation of sceCdSC (assuming its cdvdman:50) is

int sceCdSC()
{
return 1;
}

So it must be an anti-HDL protection then. I doubt any standard "cdvdman" has version number 0x1 because it usually begins with 0x100. I have more to say, there is another game with such stuff found recently. Its a "Neo Contra". I didn't extact that irx yet but it looks very same in hex-editor.

I see that Sony's cdvdfsv accesses those registers, but I don't see where hdloader's does.
Oops. I thought you talked about Sony's cdvdfsv.

crazyc

03-12-2005, 09:25 AM

So it must be an anti-HDL protection then. I doubt any standard "cdvdman" has version number 0x1 because it usually begins with 0x100. I have more to say, there is another game with such stuff found recently. Its a "Neo Contra". I didn't extact that irx yet but it looks very same in hex-editor.

But, according to your code, it's expecting a return value of 1 with the version number in the param field?

Oops. I thought you talked about Sony's cdvdfsv.

It really isn't an important point, but to show that hdl's cdvdfsv is a very simple wrapper that does nothing but forward calls to cdvdman.

Krevnik: How are you planning on handling cdvdman, by patching it's export table or replacing it altogther?

gchild19

03-12-2005, 11:03 AM

forget hdloader... people will change do demolisher if you have a good reason to start...
if u have like Gran Turismo 4 workin or a game like that than people will gladly switch over...

romz

03-12-2005, 11:49 AM

But, according to your code, it's expecting a return value of 1 with the version number in the param field?

Parameter doesn't matter, only value of "v0" register is used:

return (_sceCdSC(0xFFFFFFF7, &param) == 1) ? 5 : 1;

Second argument is formal.

crazyc

03-12-2005, 01:19 PM

Parameter doesn't matter, only value of "v0" register is used:

Do think that most games with this check would be satisfied if hdl was patched to return say, 0x225 from sceCdSC?

romz

03-12-2005, 03:23 PM

Do think that most games with this check would be satisfied if hdl was patched to return say, 0x225 from sceCdSC?
Currently I know only two games with such protection-like code inside. I quickly checked out few new games such as Nanobreaker (JAP), Project Altered Beast (JAP), Dragon Quest VIII (JAP) - nothing like "ELF" or "cdvdman" was found there but actually I didn't install them. So I think such patch could be helpful for "Tenchu: Fatal Shadows" and "Neo Contra" at least. And main idea of all this mess :) - do not forget to implement such stuff in the Demolisher. ;)

crazyc

03-12-2005, 03:47 PM

So I think such patch could be helpful for "Tenchu: Fatal Shadows" and "Neo Contra" at least.

If anyone wants to try it, patch, on a decrypted hdloader, offset 699468 from 01 00 to 25 02. I have none of these so it's untested.

snake3

03-12-2005, 05:08 PM

Currently I know only two games with such protection-like code inside. I quickly checked out few new games such as Nanobreaker (JAP), Project Altered Beast (JAP), Dragon Quest VIII (JAP) - nothing like "ELF" or "cdvdman" was found there but actually I didn't install them. So I think such patch could be helpful for "Tenchu: Fatal Shadows" and "Neo Contra" at least. And main idea of all this mess :) - do not forget to implement such stuff in the Demolisher. ;)
project altered beast (jap) works fine off of hdl

romz

03-12-2005, 11:04 PM

Some time ago I noticed an unknown disk type code in some newer versions of "cdvdman" module. It can be read from register 0xBF40200F ie sceCdGetDiskType() can return it but I don't know what type of disc it is. If you look into PS2SDK you'll find no such code here. Nearest known code is "CDVD_TYPE_CDDA = 0xFD" but I found code 0xFC. I wonder what can it be and I am sure its a DVD disc. May be it is for DVD-Audio or at least for SACD. Can someone figure out what is it since I even can not get such media at now?

bryan4203

03-13-2005, 03:14 AM

Trying neo contra now with crazyc patch.Will report back to confirm working or not

bryan4203

03-13-2005, 03:32 AM

[QUOTE=bryan4203]Trying neo contra now with crazyc patch.Will report back to confirm working or not.It works w no modes enabled.This is usa release.Great job guys.Looks like you have already gotten a few more games 2 work.

crazyc

03-13-2005, 09:13 AM

If you look into PS2SDK you'll find no such code here. Nearest known code is "CDVD_TYPE_CDDA = 0xFD" but I found code 0xFC.

Interesting.

Looks like you have already gotten a few more games 2 work.

This patch may break other games, as sceCdSC returns many things based on the input parameters. On the other hand, these games are the only ones I've heard of that call it.

Krevnik

03-13-2005, 10:34 AM

Well, after doing some actual /research/ on what HDL does when it stores information, I can pretty safely say that HDL compatibility won't be nearly as big an issue as I thought. All information about a game for HDL is stored in the first 1024KB of the partition, where normally about 4MB of space is left. It would be pretty simple to implement an extended header that marks itself as an HDDemolisher extended header that exists later in this open space. That way I can extend the data record without explicitly breaking compatibility. However, you guys will likely have to enter compatibility modes by hand for each.

Also, the trickiest part is yet to come: installing a custom module. I plan on a full replacement, rather than hooks. Some other devs I talked to agree that using hooks can get pretty messy. Not entirely sure what method I am expecting to use just yet, however. I need to do a little more research and disassembly to find out.

romz

03-13-2005, 12:51 PM

I have no experience in hooking module's functions on IOP but it seems to be pretty simple:
Just copy eight bytes of code from begining of exported entry than place jump-code there:

j hook_proc
nop

When hook subroutine want to pass control to the original entry it must execute first 8 bytes (copied) of original entry followed with:

j orignal_8
nop

where "orignal_8" is address of original entry point plus eight. I believe such hooks should work well since size of most opcodes is equal to four bytes.

Btw, my friend brought me MGS2:Substance (HK-Silver) today. It was succesfuly detected as DualLayer.

Also my question at last:
How can I put my source on CVS? I didn't do such thing before. My CVS client can download Demolisher sources but I have no idea how to add my sources there correctly. I am planning to add some reversed code from few cdvdman modules there.

Krevnik

03-13-2005, 01:12 PM

I have no experience in hooking module's functions on IOP but it seems to be pretty simple:

That patches import tables, I need to patch export tables, not to mention the fact that I also have to patch RPCs and CMDs.

Also my question at last:
How can I put my source on CVS? I didn't do such thing before. My CVS client can download Demolisher sources but I have no idea how to add my sources there correctly. I am planning to add some reversed code from few cdvdman modules there.

If you read the CONTRIBUTING file in the CVS checkout, you have to submit them through me. The CVS pserver is specifically designed to prevent access beyond read, to prevent security flaws in CVS from granting access to the rest of the OS. You can submit them to me at krevnikATcomcastDOTnet. (NOTE: Bypassing search crawlers that spammers use, so you have to enter it manually, sorry)

I will modify them for consistency, add any credit headers that needs to be added that might have been forgotten/etc to make sure the file is credited to the proper people, and then add them into CVS.

romz

03-13-2005, 05:14 PM

That patches import tables, I need to patch export tables, not to mention the fact that I also have to patch RPCs and CMDs.
Probably there are to ways to perform a patch - patch "cdvdman" export table or patch every function's entry itself. I talked about second way. But if both ways acceptable I think it'll be better to implement option to select more good way for certain title. And it will even better if Demolisher can hook orignal "cdvdman" module's function or replace module with own emulator. That'lll be an ulimate compatibily, probably. As for "cdvdfsv" - I think it must be replaced because it uses hardware registers which is not acceptable during C/DVD emulation.

If you read the CONTRIBUTING file in the CVS checkout, you have to submit them through me. The CVS pserver is specifically designed to prevent access beyond read, to prevent security flaws in CVS from granting access to the rest of the OS. You can submit them to me at krevnikATcomcastDOTnet. (NOTE: Bypassing search crawlers that spammers use, so you have to enter it manually, sorry)

I will modify them for consistency, add any credit headers that needs to be added that might have been forgotten/etc to make sure the file is credited to the proper people, and then add them into CVS.
Ok. I'll contact you when some important&interesting parts "cdvdman" modules will be recoded in "C" language.

crazyc

03-13-2005, 06:26 PM

I am planning to add some reversed code from few cdvdman modules there.

Which parts are you working on? I'm planning on writing some code too, but don't want to overlap.

Krevnik

03-13-2005, 07:39 PM

Probably there are to ways to perform a patch - patch "cdvdman" export table or patch every function's entry itself. I talked about second way. But if both ways acceptable I think it'll be better to implement option to select more good way for certain title. And it will even better if Demolisher can hook orignal "cdvdman" module's function or replace module with own emulator. That'lll be an ulimate compatibily, probably. As for "cdvdfsv" - I think it must be replaced because it uses hardware registers which is not acceptable during C/DVD emulation.

This is wrong on a couple of points, don't feel offended, I had to get corrected/assisted by someone else on some of this myself. But your method is patching import tables, which is /bad/. Then you have to patch every single module that gets loaded. Also, most modules do have some sort of RPC table, but it is usually a better idea to unhook the module's RPC entirely and setup your own rather than attempt to patch it. And if it plugs any commands into the SifCmd table, you have to patch those as well. (I believe CDVDMAN does, I have to double-check)

I have already decided that a custom CDVDMAN module is the only way, the trick is how to install it. I can approach it from the standpoint of replacing an existing CDVDMAN module, or from the standpoint of sneaking it into the IOP before it gets one loaded. However, knowing how to sneak one into the IOP requires that I have a better understanding of the reboot process than I do now.

<Removed because I can't seem to read Romz' post and realize we were saying the same thing about CDVDFSV>

Crazyc: You said you would be writing some code for a custom CDVD module? If that is so, romz is more focused on reversing the existing CDVDMAN module into some C code (much like [RO]man's work). There shouldn't be any overlap at the moment, since I am starting work on the HDD code this next week.

romz

03-13-2005, 09:35 PM

This is wrong on a couple of points, don't feel offended, I had to get corrected/assisted by someone else on some of this myself. But your method is patching import tables, which is /bad/. Then you have to patch every single module that gets loaded. Also, most modules do have some sort of RPC table, but it is usually a better idea to unhook the module's RPC entirely and setup your own rather than attempt to patch it. And if it plugs any commands into the SifCmd table, you have to patch those as well. (I believe CDVDMAN does, I have to double-check)
You probably misunderstood be or I probably didn't write clear enough what I mean. I spoke about patching "cdvdman" export tables or exported functions themself. Some "cdvdman" modules export more than 120 entry and even when many of them are just dummy functions it's not easy to implement rest of them still. Especially some of them (109-120) which related to "secrman" or something like that - they use unknown s-commands (range 0x18 - 0x30). Also some modules use unknown registers sometimes - I found range 0xBF402030-0xBF40203A in one of available modules. So I highly doubt it would be easy to fully rewrite "cdvdman" for absolute compatibily but I hope most games don't mess with such little known stuff.

Btw, some rare modules may rely on count of expoted entries in "cdvdman".

romz

03-13-2005, 09:57 PM

Which parts are you working on? I'm planning on writing some code too, but don't want to overlap.
I believe it's easy to implement such usual functions like sceCdRead, sceCdSearchFile, sceCdMmode and so on, so I focused on more specific code like sceCdReadDvdDualInfo(83), sceCdLayerSearchFile(84 - actually called be sceCdSearchFile in new modules), _sceCdSC(50), sceCdSync(11). But research requires all code to be explored anyway.

crazyc

03-13-2005, 10:57 PM

You said you would be writing some code for a custom CDVD module?

I don't know what you mean, I was just going to also work on reimplementing cdvdman.

Krevnik

03-13-2005, 11:34 PM

You mean reverse engineering? If you are reimplementing, that implies writing actual code that performs its functions, rather than figuring out what it does. (Hence the two terms being different) So, if you are writing a new module, that is reimplementation, such as I am attempting to do for the CDVD/HDD module. If you are just disassembling and working out what the CDVDMAN module does, then you are reverse engineering.

So which are you doing? ;)

romz

03-14-2005, 02:23 AM

I do reverse engineering. But I am not limited to only one "cdvdman", so I going to create code for combo-version of "cdvdman" which includes features available in different modules. I believe it will be best solution for our aims.

crazyc

03-14-2005, 08:41 AM

I'm reimplementing some of those functions whose purpose we already know.

romz

03-14-2005, 03:32 PM

Here is a little very first peace of my work. It's a starting routine of "cdvdman" module. I know it's not good enough but I post it to what I am doing. Btw, there are at least two strange thing here. First, I failed to determinate size of data for toc, but I am sure it should be greater than or equal to one sector size. Second, I wonder, how can function cdrom_deinit() be called without a parameter? Did I miss something?

#define TOC_BUFFSIZE ???

struct _iop_device_ops;

typedef struct _iop_device {
const char *name;
unsigned int type;
unsigned int version;
const char *desc;
struct _iop_device_ops *ops;
} iop_device;

typedef struct _iop_device_ops {
int (*init)(iop_device *);
int (*deinit)(iop_device *);
int (*format)(iop_file *, const char *, const char *, void *, int);
int (*open)(iop_file *, const char *, int, ...);
int (*close)(iop_file *);
int (*read)(iop_file *, void *, int);
int (*write)(iop_file *, void *, int);
int (*lseek)(iop_file *, unsigned long, int);
int (*ioctl)(iop_file *, unsigned long, void *);
int (*remove)(iop_file *, const char *);
int (*mkdir)(iop_file *, const char *, int);
int (*rmdir)(iop_file *, const char *);
int (*dopen)(iop_file *, const char *);
int (*dclose)(iop_file *);
int (*dread)(iop_file *, void *);
int (*getstat)(iop_file *, const char *, void *);
int (*chstat)(iop_file *, const char *, void *, unsigned int);
int (*rename)(iop_file *, const char *, const char *);
int (*chdir)(iop_file *, const char *);
int (*sync)(iop_file *, const char *, int);
int (*mount)(iop_file *, const char *, const char *, int, void *, unsigned int);
int (*umount)(iop_file *, const char *);
int (*lseek64)(iop_file *, s64, int);
int (*devctl)(iop_file *, const char *, int, void *, unsigned int, void *, unsigned int);
int (*symlink)(iop_file *, const char *, const char *);
int (*readlink)(iop_file *, const char *, char *, unsigned int);
int (*ioctl2)(iop_file *, int, void *, unsigned int, void *, unsigned int);
} iop_device_ops;

int cdrom_init(iop_device *);
int cdrom_deinit(iop_device *);
int cdrom_open(iop_file *, const char *, int, ...);
int cdrom_close(iop_file *);
int cdrom_read(iop_file *, void *, int);
int cdrom_lseek(iop_file *, unsigned long, int);
int cdrom_ioctl(iop_file *, unsigned long, void *);

s32 cdrom_nulldev(iop_file *nuldev, ...)
{
printf("nulldev0 call\n");
return -5;
}

s64 cdrom_nulldev64(iop_file *nuldev, ...)
{
printf("nulldev0 call\n");
return -5;
}

#define IOP_DT_FS 0x10
#define IOP_DT_FSEXT 0x10000000

iop_device_ops cdvdman_cdops =
{
cdrom_init,
cdrom_deinit,
cdrom_nulldev,
cdrom_open,
cdrom_close,
cdrom_read,
cdrom_nulldev,
cdrom_lseek,
cdrom_ioctl,
cdrom_nulldev,
cdrom_nulldev,
cdrom_nulldev,
cdrom_dopen,
cdrom_dclose,
cdrom_dread,
cdrom_getstat,
cdrom_nulldev,
cdrom_nulldev,
cdrom_nulldev,
cdrom_nulldev,
cdrom_nulldev,
cdrom_nulldev,
cdrom_nulldev64,
cdrom_devctl,
cdrom_nulldev,
cdrom_nulldev,
cdrom_ioctl2
};

static cdvdman_cdrom[] = {'c', 'd', 'r', 'o', 'm', 0};

iop_device cdvdman_cddev = {cdvdman_cdrom, IOP_DT_FS | IOP_DT_FSEXT, 1, "CD-ROM ", &cdvdman_cdops};

char *cdvdman_ptoc; /* sbss:0000 */
char *cdvdman_unk001; /* sbss:0008 */

char toc_buffer[TOC_BUFFSIZE];

int start(int argc, char **argv)
{
if (RegisterLibraryEntries(&cdvdman_lib)) return 1;

DelDrv(cdvdman_cdrom);
if (AddDrv(&cdvdman_cddev))
{
cdrom_deinit(); /* ??? - No parameter here */
return 1;
}

cdvdman_ptoc = toc_buffer;
cdvdman_unk001 = toc_buffer + 0xF6DC;

cdvdman_init();

SetRebootTimeLibraryHandlingMode(&cdvdman_lib, 2);

return 0;
}

romz

03-14-2005, 04:07 PM

And this is another small piece of my work. Why did I post it here too? That's because it probably can be interesting not just for developers of Demolisher. This code is used to get info about DVD9 media. I continue my research so code is still incomplete but I present you some useful information about dual-layer disc support. Actual work performed by function DvdDual_infochk() which is not ready yet. It detects is media dvd9 but it even does more. I believe it reports type of dvd9 media also -PTP or OTP. I think it stores values "1" or "2" into location pointed by "on_dual" argument depend on dvd9 type (and of course zero for dvd5 media). Further more, "cdvdman" can emulate dvd9 disc when one of variables set to zero value. But keep in mind that image must be splitted in two discs and have special information in sector 14. Also there is a code that opens tray and write "change disc" prompt into stdout. I have no idea how and when that code will be triggered at this moment. However it's clear that new "cdvdman" modules have some dvd9 emulation support.

typedef
struct {
unsigned char trycount;
unsigned char spindlctrl;
unsigned char datapattern;
unsigned char pad;
} sceCdRMode;

char *cdvdman_masterd = "PlayStation Master Disc";

/* Exported entry #83 */
/* Params:
on_dual - pointer to variable where dvd media type will be stored
layer1_start - pointer to variable where lsn of layer1 will be stored*/
int sceCdReadDvdDualInfo(int *on_dual, u_int *layer1_start)
{
*on_dual = 0;
*layer1_start = 0;
if (word_CEC4)
{
if (!DvdDual_infochk()) return 0;
*on_dual = cdvdman_dldvd;
*layer1_start = cdvdman_layer1;
return 1;
}
else
{
int i;

if (!cdvdman_isdvd()) return 1;
if ((cdvdman_mmode != 2) && (cdvdman_mmode != 0xFF)) return 0;

read_mode.mode = 0;
read_mode.spindlctrl = 0;
read_mode.datapattern = 0;

sceCdRead0(0xE, 1, cdvdman_ptoc, &read_mode, 0, 0);
if ((!sceCdSync(3)) && (cdvdman_cderror));
{
if (cdvdman_verbose > 0) Kprintf("CDVD: ReadDvdDualInfo Read Error %02x, %d\n", cdvdman_cderror, 0);
return 0;
}

for (i = 0; i < 0x14; i ++)
{
if (cdvdman_ptoc[0x68 + i] != cdvdman_masterd[i]) break;
}
if (i != 0x14)
{
if (DvdDual_infochk())
{
*on_dual = cdvdman_dldvd;
*layer1_start = cdvdman_layer1;
return 1;
} else return 0;
}
else
{
if (cdvdman_ptoc[0x83] != 2) return 1;
if (!(cdvdman_ptoc[0x84] & 0x2)) return 1;
cdvdman_curdvd = cdvdman_ptoc[0x85];
cdvdman_elayer = 1 + cdvdman_ptoc[0x86] + (cdvdman_ptoc[0x87] << 8) + (cdvdman_ptoc[0x88] << 16) + (cdvdman_ptoc[0x89] << 24);
cdvdman_dldvd = 0;
cdvdman_layer1 = cdvdman_elayer;
cdvdman_dlemu = 1;
*on_dual = 1;
*layer1_start = cdvdman_layer1;
if (cdvdman_verbose > 0) Kprintf("sceCdReadDvdDualInfo():Cur_Disk %d layer1_start %d\n", cdvdman_elayer, cdvdman_curdvd);
return 1;
}

}
}

PS: Did you just hear about "a software hack" for DVD+DL copies from Team Toxic? Probably solution is related to code above.

Krevnik

03-14-2005, 04:15 PM

Well, Sjeep and Team Toxic haven't been really private about the fact that they got DVD9s working in their software that is yet to be released. That knowledge that needed to be gained in order to do it would also allow them to get DVD+/-R DL discs working as well, I would assume.

Romz, this work is pretty good, and I would like to start incorporating it into the HDDemolisher CVS tree. Since you haven't sent any of it directly for contribution, do you mind if I add it to the tree? Incomplete modules are not particularly a problem in the tree, as you can tell from my MODLOAD.c which is by far not complete.

romz

03-14-2005, 04:53 PM

It's ok to add that code to CVS tree but it must be changed (slightly) in the future. Things like "if (word_CEC4)" is a bad idea since actual address of this variable can be different in another version of module. I left it "as is" because I can't locate other references to this variable right now so I didn't give it a more suitable name.

zabolyx

03-14-2005, 06:00 PM

I was thinking today and think that a partition setup would not be very compatable.. using a setup like what Daemon Tools uses and mount images from the same Partition would not only speed up HDD access (loading the modules on my HDD takes a while now that I have about 40 games) but would make actual frame accesses available. If a table could be built to emulate the LBA of the game disc then even the game could be shrunk with no problem.

crazyc

03-14-2005, 06:34 PM

Here is shot a scmd support. I still need to test it but it should be very close to how scmds are handled in older cdvdman versions.

#include <thsemap.h>

static volatile char *scmd = 0xbf402016;
static volatile char *sdata_in = 0xbf402017;
static volatile char *sdata_out = 0xbf402018;
static int scmd_sema; /* cdvdman seems to serialize scmds and ncmds separately */

void scmd_init()
{
iop_sema_t s;

s.initial = 0;
s.max = 1;
s.option = 0;
s.attr = 0;

scmd_sema = CreateSema(&s);
}

/* later cdvdman version have an additional parameter, this follows the version from the rom */
static int set_prev_command(u8 command, void *input, u32 in_size, void *result, u32 out_size)
{
char j;
int out_count, i;
char buffer[20]; /* what is the largest result from an scommand? */
char *in = input, *out = result;

WaitSema(scmd_sema);
j = *sdata_in;
if(*sdata_in & 0x80)
{
SignalSema(scmd_sema);
return 0; /* older cdvdman versions return failure, newer ones do more processing */
}
while(!(*sdata_in & 0x40)) /* this clears any remaining output */
j = *sdata_out;

for(i = 0; i < in_size; i++)
*sdata_in = in[i];

*scmd = command;
j = *scmd;
j = *sdata_in;

while(*sdata_in & 0x80);

for(out_count = 0; !(*sdata_in & 0x40); out_count++)
buffer[i] = *sdata_out;

#ifdef DEBUG
if(out_size < out_count)
printf("set_prev_command:cmd 0x%x err result cnt %d:%d \n", command, out_size, out_count);
#endif

for(i = 0; i < out_size; i++)
out[i] = buffer[i];

SignalSema(scmd_sema);
return 1;
}

int sceCdApplySCmd(u8 cmd, void *in, u32 in_size, void *out)
{
while(!set_prev_command(cmd, in, in_size, out, 16))
DelayThread(2000);
return 1;
}

Stefy2

03-15-2005, 01:37 AM

zabolim what you are asking is simply impossible, not only in the part of hdd demolisher, but you should know that libhdd has a BIGGG bug it can't use partition >4gb there is nothing to do except correcting this bug but anyone has tried correcting it.
why i tell >4gb, try doing a partition bigger than 4gb, the partition allocated will be correct but if you check your free space it's = at gb from a 4gb bounduary to another to explain better in my bad english see there
1gb = 1gb free
2gb = 2gb free
3gb = 3gb free
4gb = 4gb free
5gb = 1gb free
6gb = 2gb free
7gb = 3gb free
8gb = 4gb free
9gb = 1gb free
10gb = 2gb free

and so on

romz

03-15-2005, 02:10 AM

/* cdvdman seems to serialize scmds and ncmds separately */
Yes, it really does. Many "cdvdman" modules create five event flags: (1 - for ncdms, 2 - for scmds, 3 - cd callback related, 4 - not sure about it, 5 - for sceCdRead)

romz

03-15-2005, 07:38 AM

Some help is required. I can't realize what's going on in the code bellow:

lbu $v1, 0($s0)
la $v0, (unk_B8DF+0xA062)
sb $v1, (cdvdman_cderror - cdvdman_cderror)($v0)
lbu $v0, (cdvdman_cderror - cdvdman_cderror)($v0)
j loc_4CD0
nop

I don't understand: is it "set" or "swap" of error code value. What will be in the "v0" register - old error code or new value, equal to content of "v1" register ?

zabolyx

03-15-2005, 08:38 AM

Stefy2

Thanks for the info.... did not know that... that answers why that was used for HD Loader then

crazyc

03-15-2005, 10:26 AM

It looks like it's setting the error code variable then loading it back into v0. Something like this,

cdvdman_cderror = error_code;
return cdvdman_cderror;

Krevnik

03-15-2005, 12:07 PM

Stefy2

Thanks for the info.... did not know that... that answers why that was used for HD Loader then

Well, the other reason is compatibility. Each partition has a header plus the raw rip from the disc. Since the entire rip is available for reading as if it was the original disc, it becomes easy to remain compatible to CDVDMAN because we aren't doing tricks behind the scene.

As for the work on CDVDMAN so far, great. I am still sorting through it all along with the work I have been doing, so it might take some time before CVS is updated again.

romz

03-15-2005, 01:37 PM

It looks like it's setting the error code variable then loading it back into v0. Something like this,

cdvdman_cderror = error_code;
return cdvdman_cderror;
I thought so, but then I thought about command pairing.

crazyc

03-15-2005, 02:37 PM

I thought so, but then I thought about command pairing.

Shouldn't be an issue there. I think the pseudo op (la) is making it less clear.

romz

03-15-2005, 05:10 PM

New code for CVS and for people who want to know about dvd9 discs. An additional code for previoud message about "sceCdReadDvdDualInfo()".

#define CDVDreg_TYPE (*(unsigned char *)0xBF40200F)

/* bss */
unsigned int cdvdman_layer1t;
unsigned char cdvdman_usetoc;
unsigned char cdvdman_dldvd;
unsigned char cdvdman_curdvd;
unsigned char cdvdman_dlemu;
unsigned int cdvdman_elayer;

int DvdDual_infochk()
{
if (QueryIntrContext()) return 1;

/* I suppose it checks if disc has been changed */
if (!some_proc(4)) /* sends scmd #5 */
{
if (cdvdman_dldvd != 0xFF) return 1;
}

cdvdman_usetoc = 1;
if (!cdvdman_readtoc(cdvdman_ptoc))
{
cdvdman_usetoc = 0;
cdvdman_dldvd = 0xFF
return 0;
}

cdvdman_usetoc = 0;
cdvdman_layer1 = cdvdman_l1start(cdvdman_ptoc);

/* Dual Layer & PTP/OTP detection */
if (toc[0xE] & 0x60)
{
cdvdman_dldvd = (cdvdman_ptoc[0xE] & 0x10) ? 2 : 1;
}
else cdvdman_dldvd = 0;

if (cdvdman_dlemu)
{
if (cdvdman_verbose) Kprintf("CDVD:DualEmuON\n");
cdvdman_layer1 = cdvdman_elayer;
cdvdman_dldvd = 0;
}

if (cdvdman_verbose) Kprintf("DvdDual_info: %02x Layer1_LSN:%d opo_or_para %d\n", cdvdman_ptoc[0xE], cdvdman_layer1, cdvdman_dldvd);

return 1;
}

u_int cdvdman_l1start(char *toc)
{
return 0xFFFD0001 + toc[0x17] + (toc[0x16] << 8 ) + (toc[0x15] << 16);
}

/* Exported entry #9 */
int sceCdGetToc(u_char *toc)
{
return cdvdman_readtoc(toc);
}

int cdvdman_isdvd()
{
unsigned int type;

type = CDVDreg_TYPE & 0xFF;
if (type == 0x14) return 1; /* SCECdPS2DVD */
if (type < 0x15)
{
if (type < 0x10) return 0;
}
else
{
if (type == 0xFD) return 0; /* SCECdCDDA */
if (type < 0xFE)
{
if (type == 0xFC) return 1; /* ???, but should be a DVD anyway */
}
else if (type == 0xFE) return 1; /* SCECdDVDV */
}

return 0;
}

Few more words about DVD9 emulation by "cdvdman" module itself:
As I mentioned before there is a code in the depth of that modules which can used for emulation of dual layer disc. Also I talked it asks to replace DVDs sometimes. Now I know more about this emulation. That code resides within function "u32 sceCdLsnDualChg(u32 lsn);" in module. It is an internal function and I am not sure it's exported in any version of "cdvdman" module (However it can be accessed via sceCdSC() function with certain parameter). This function translates standard lsn (used by application) to actual lsn for emulated dual layer disc and changes disk (asks to do it and ejects tray) when data from other layer requested. But it is more important, that this function is call in from such functions sceCdRead and sceCdSeek, so correct emulation is possible by "cdvdman" itself. However such lsn translation can be bypassed by some calling direct read functions (for example sceCdRead0() or sceCdRV() ) available for IOP applications so some streaming modules can fail.

ps2feen

03-15-2005, 09:20 PM

dont call me a newb i dont know anyhting about programming.. is all this programmin done with visual C++ 6.0 and if so how the hell is al thsi shit incorperated in hddemolisher.. does the program output elfs or somshit

Stefy2

03-16-2005, 01:27 AM

you can use visual c++ to program (c/c++ are always the same) but to compile you can't use visual c++'s compilers you should use patched gcc compilers available from ps2dev

v1p3r

03-16-2005, 08:24 AM

Would it be at all feasable to connect an IDE DVD drive to the PS2? This would bypass all the CD checks if i'm not mistaken, and most of the code could be recycled from HDDemolisher. The only major new code would be the CD/DVD driver itself, which I don't think has been done by ps2dev.

Is this a reasonable idea or am I talking crap...? :crazy:

Stefy2

03-16-2005, 09:51 AM

and what's the point? to do that you need to redirect cdrom calls and you have about the same compatibility rate as reading from hd (other that it's worse to take a dvd drive with a ide cable coming out of the console)

v1p3r

03-16-2005, 10:07 AM

I guess so, I was in bit of a rush to post that and diddn't consider that compatability would in fact be exactly the same.

KaylaKaze

03-16-2005, 11:37 AM

leidout

03-16-2005, 10:31 PM

hi there, i know you guys are all into your technical aspects of this thread.

but i was hoping when the end product is finally finished, we'd be able to insert our own custom menu & splash images (if there is a splash at all?) just like the hdl patches are capable of doing.

(: cheers and keep up the good work (even though i understand none of it).

Stefy2

03-17-2005, 01:15 AM

it's an opensource project so its surely possible.

Krevnik

03-17-2005, 02:12 AM

Yeah, nothing stopping anyone, and it will be easier since you can run a compile which puts the new screen in (save space even if you choose)... Although I am not entirely sure what the facination with it is. Anyone care to clarify for me?

Angelus3X

03-17-2005, 05:13 PM

What saving space or putting in custom screens?

Saving space is useful for Mem card exploit users and I guess people like to have cool looking pics when they load up HDL.

romz

03-17-2005, 05:41 PM

All cool looking pictures can be loaded from HDD so no need to insert them into the ELF. One or two bundled images for splash screens is more than enough. Of course it will require installation procedure but it can speed up loading from memory card.

Stefy2

03-18-2005, 01:09 AM

adavidm

03-18-2005, 02:40 AM

mmm loading from hd... it isn't a waste of time powering up the disc at black screen (5 seconds about) mounting a partition to load the splash screen and normal image (1-2 sec) unmounting it and then checking the partition list...

I agree, the basic program should have the object of getting started ASAP, not waste time with loading images.

What about a combination of the two approachs? Store the graphics seperate to the ELF, by default in the same folder as HDDemolisher, and then have a preference setting that lets you put whatever path you want in. Then the people that like pretty splash screens and menus can have them, and others (like me) can have nothing at all for maximum speed/minimum footprint.

I realise this is extra work but since the project is open source, it can be done by anyone with the requisite coding skills and desire to do so. This would leave the core dev(s) working on the guts of the program.

cheers

adavidm

crazyc

03-18-2005, 05:13 PM

Here's sceCdSearchFile. I'm not sure if it's correct or if it accounts for all possibilities, but it's a start.

#include <iso.h>
#include <cdvdman.h>
#include <sysmem.h>

static int search_file(cd_file_t *fp, const char *name, uint32 pvd_lsn)
{
char *buffer;
struct iso_directory_record *dentry;
cd_read_mode_t mode;
uint32 dir_len, dir_lsn, dentry_len, i, j;

if(!(buffer = (char *)AllocSysMemory(ALLOC_FIRST, ISO_DEFAULT_BLOCK_SIZE, NULL))) /* preallocate? */
return 0; /* out of memory */

mode.trycount = 16;
mode.spindlctrl = CdSpinNom;
mode.datapattern = CdSecS2048;
if(!sceCdRead(pvd_lsn, 1, buffer, &mode))
goto exit_err;

dentry = &(((struct iso_primary_descriptor *)buffer)->root_directory_record);
dir_len = isonum_733(dentry->size);
dir_lsn = isonum_733(dentry->extent);
if(name[0] == '\\')
name++;

for(i = 0; i < dir_len; i += ISO_DEFAULT_BLOCK_SIZE)
{
if(!sceCdRead(dir_lsn, 1, buffer, &mode))
goto exit_err;
for(j = 0; j < ISO_DEFAULT_BLOCK_SIZE; j += dentry_len)
{
dentry = &buffer[j];
if(!(dentry_len = isonum_711(dentry->length)) || ((dentry_len + j) > ISO_DEFAULT_BLOCK_SIZE))
break; /* end reached with no match */
if((isonum_711(dentry->name_len) == 1) && (dentry->name[0] <= 1))
continue; /* parent or current dir dentry */
if(!strncmp(name, dentry->name, isonum_711(dentry->name_len)))
{
if(!(dentry->flags[0] & 2)) /* match file (matching a directory shouldn't be a */
{ /* problem because of the version field) */
fp->lsn = isonum_733(dentry->extent);
fp->size = isonum_733(dentry->size);
memcpy(fp->name, dentry->name, 16);
fp->date[0] = dentry->flags[0]; /* is this right? */
fp->date[1] = dentry->date[6];
fp->date[2] = dentry->date[5];
fp->date[3] = dentry->date[4];
fp->date[4] = dentry->date[3];
fp->date[5] = dentry->date[2];
fp->date[6] = dentry->date[1];
fp->date[7] = dentry->date[0];
FreeSysMemory(buffer);
return 1;
}
dir_len = isonum_733(dentry->size);
dir_lsn = isonum_733(dentry->extent) - 1;
i = ISO_DEFAULT_BLOCK_SIZE * -1;
name = strchr(name, '\\') + 1; /* name better be valid */
break;
}
}
dir_lsn++;
}

exit_err:
FreeSysMemory(buffer);
return 0;
}

int sceCdSearchFile(cd_file_t *fp, const char *name)
{
return search_file(fp, name, 16);
}

#ifndef __IOP__ /* what should this define be? */

#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>

int cdrom;

int FreeSysMemory(void *ptr)
{
return free(ptr);
}

void *AllocSysMemory(int mode, int size, void *ptr)
{
return malloc(size);
}

int sceCdRead(u32 lsn, u32 sectors, void *buf, cd_read_mode_t *mode)
{
if(lseek(cdrom, lsn * ISO_DEFAULT_BLOCK_SIZE, SEEK_SET) == -1)
return 0;
if(read(cdrom, buf, sectors * ISO_DEFAULT_BLOCK_SIZE) == -1)
return 0;
return 1;
}

int main(int argc, char **argv)
{
cd_file_t cd_file;

if((argc < 3) || ((cdrom = open(argv[1], O_RDONLY)) == -1))
return -1;

if(sceCdSearchFile(&cd_file, argv[2]))
printf("found %s\n", cd_file.name);

return 0;
}
#endif

o0SMòÓKE0o

03-18-2005, 06:13 PM

Would this link (http://www.zophar.net/psx2.html) help in anyway ?

HypERSoniC

03-20-2005, 09:04 PM

Would this link (http://www.zophar.net/psx2.html) help in anyway ?

not really :p

omegatron

03-20-2005, 11:41 PM

why the hell would it??? :crazy:

CDB

03-21-2005, 11:48 AM

perhaps he meant looking at projects like pcsx2 for any information..

RyDaWg2k1

03-21-2005, 12:27 PM

anyway, attempting to get this thread back on track and stopping a possible fight for no reason, how is progress krevnik?

o0SMòÓKE0o

03-21-2005, 01:33 PM

not really :p

why the hell would it??? :crazy:

Why try to invent the wheel a second time ? :headbutt:

To emulate a PS2, means you should know how a PS2 works . And maybe these developers have found a good way to play PS2 DVD / CD's.

It's just a thought. Not very helpfull in coding. So this is the little help I can give :) (If it's helpfull is not for me to decide :P

But I'm really supporting the efforts for all the coders here. And wish them luck in developing this very promissing Ps2 app.

Cheers to them !! :applaud:

zabolyx

03-23-2005, 01:40 PM

OK.... quick idea.... or thought

A friend of mine got Tekken 5 the other day.... and it seems that it can load from the DVD the old PS games.... it doesn't seem to emulate the PS just force the system into the PS mode from the PS2 mode....

could a similar engine be used to load games from the HDD.... I did notice that the PS2 had to be reset after the mode was changed... I wouldn't mind that at all.... I just want to have my 300 PSOne games on a HDD for easy storage.

omegatron

03-23-2005, 02:03 PM

well toxic thinks they can do it so maybe so can hddemolsher

Angelus3X

03-23-2005, 02:12 PM

How do you know that when you play the old tekken ports that the PS2 boots into PS1 mode? Couldnt it just be resetting the IOP or loading a different ELF file for the old port?

Stefy2

03-23-2005, 02:21 PM

they are ps2 elfs they aren't psx games anyone can see the big difference in graphic, they are porto to ps2 of the arcade version so it's not in psx mode but in ps2 as for omegatron where you see toxic os team is going to do it? in the readme there is nothing about it.
- The ability to launch PS2/PSX games and DVD movies directly from ToxicOS,
turning it into a complete browser replacement.
if you looked at this you see there is written PS2 this mean that it talks loading from cd/dvd ps2/psx games and dvd video after having boot up to toxic os, like what you do with ps2os for example

omegatron

03-23-2005, 03:06 PM

ok my mistake... I understand they just mean their going to make it so that you can play what ever disk is in the tray once toxic is booted i see

endy

03-23-2005, 11:10 PM

Frankly, who cares.

There is still far too much work left on the program core before anybody starts asking 'can it do this', 'how about adding this', etc.

This topic is getting far too polluted with trash, and Krevnik appears to be missing in action :)

dlanor

03-24-2005, 06:58 AM

Frankly, who cares.

There is still far too much work left on the program core before anybody starts asking 'can it do this', 'how about adding this', etc.
Indeed! I agree completely.
I suggest that we stockpile our more exotic ideas for later, rather than posting them here now. Once basic functionality has been achieved we can revive discussion of things to add or modify.

This topic is getting far too polluted with trash,
True, but on a subject like this it's hard to avoid, especially if it starts attracting the newbies too... :(

I suggest the following guideline for this thread:
----------------------------------------------
This thread was created to discuss Krevnik's "HD Homebrew project", just like it says in the thread title. So lets stop discussing ToxicOS, PS2 emulators, and other unrelated stuff here. Even if they do contain some functionality similar to what we want, this is going to be a new design, not just a ripoff.

Don't get me wrong now. Of course it would be ok to discuss how others solved something when this project runs into some problem, so that new insights gained from the other solutions can inspire some new solution. But that need is not here yet, so save it for later.

and Krevnik appears to be missing in action :)
Let's hope it's 'Busy in action' instead ;)
I just checked, and it's been a week since his last post here. That may be more than some of us like, but no real cause for alarm.

Best regards: dlanor

Krevnik

03-24-2005, 12:40 PM

Yes, I have been rather busy lately, with not much time, I could have sworn I made a post saying that it would get rather busy for me... but I guess I didn't. Anyways, since I recently graduated from college, I am also in the middle of a job hunt, and because my degree doesn't cover C#/.NET development, I am also making sure I am up to snuff on that as well. In short, I have two large-scale projects being worked on, and a job hunt. This means that every so often I just won't have the time to post updates... One of the reasons why contributions are very welcome.

To those who have been reverse engineering: Good work so far, since I got rather busy and seem unable to extract the code from the webpage (some weird thing about IFRAME tags garbling the ability to copy/paste on my machine), could you guys e-mail me the work done so far in a text format?

I hope to get back to work on this for awhile next week, as I have been working out how the heck to handle partition creation/etc through ps2atad and libhdd, and how to do raw access of the hdd through the hdd0: device.

cmal1492

03-30-2005, 08:44 AM

krevnik - I stubled across this:
The first one will be the uncruncher stub. By default, it's
a zlib stub compiled to be loaded at 0x88000. You may want to compile other
stubs, to get different loading addresses. I provide another stub which loads
at 0x1a00000. Should you produce new stubs, just change the stub/Makefile.
The second section contains the packed data. This basically can be loaded
anywhere. That is the "base" option of the command line.
In the ps2packer 0.2 readme, not sure if that has much relevance to hdl having something at 0x88000 - unless it is the decrypt/decompression stub perhaps?

ps2feen

03-30-2005, 09:05 PM

i jsut boguht C++ for dummies.. im gona read the hole thing and hopefully know what your talking about

Krevnik

04-01-2005, 11:59 AM

HDL uses 0x88000 for the patching software, which catches the IOP reset, and depending on which version (since the team has done some work to improve HDL as part of Toxic) 'patches' the IOP modules.

And all I have to say about this last week is: Grr.

Things started good, and then /wham/ I get scheduled for *6* interviews out of the blue early in the week, and I am still wading through them all. However, there is a silver lining, I have gotten an offer, will have a little time for some HDDemolisher work between Tuesday and Friday of next week (after the interviews all die out and I give my response to the offer), have some time to go to a Con next weekend, and then head out to work in 10 days. Although what I am kinda thinking, is that a dev blog should be setup for this project, so we can have a place where I can rant about progress, and have some discussion on the various problems. Would this interest those following the development, and those who might be interested in contributing bits and pieces?

lazyj

04-01-2005, 08:14 PM

I think a blog would be great, if not just for ppl like me who are interested in the progress you guys are making :)

I think its awesome that this is open source.

Kiyo

04-01-2005, 08:18 PM

DJFire

04-01-2005, 08:46 PM

<-- This Is My First Post Here ;)

I Have Webspace You Can Borrow ;)
--1 Gig Space
--1 Terabyte transfer a month
With All The Fancy Features MySQL, PHP, Ect...

romz

04-02-2005, 12:17 PM

I need some help again. Code (below) can be found in "cdvdman" modules and I have problem to reconstruct it in "C" code.

addui $sp, -0x18
lui $v1, 0x38E3
sw $ra, 0x18+ra($sp)
lw $v0, SysClock.low($a0)
ori $v1, 0x8E39
multu $v0, $v1
la $a0, 0xC744
mfhi $a2
ja Kprintf
srl $a1, $a2, 13

What source can cause compiler to generate such code? It's definatelty something like

Kprintf("bla bla bla %d\n", arg0.low /* ??? */);

I suspect it's an optimized code for some divide operation.

TheCrowX

04-05-2005, 04:53 PM

Hi I have a question Iwas wondering if it is possible to write a driver to support NTFS format ? :chinscrat
Thanx

dlanor

04-05-2005, 08:34 PM

Theoretically you can of course develop any type of driver for any system.
But there's no point in having NTFS for PS2 since no PS2 software would use it.

More importantly, however, your question has NOTHING to do with the topic of this thread, which is dedicated to a specific HD homebrew project, intended to produce a program similar to (but not related to) HDLoader.

Best regards: dlanor

Zylyx

04-08-2005, 12:05 AM

Oh, I can see where he was going with that, and it's kinda relevant in a way. To him, I'll say:

It'd be nice if the replacement used NTFS partitions to store plain ISO images, since that's easier to maintain.. but...

If the Linux guys, who have spent YEARS analyzing NTFS, can't get a good workable fully writable NTFS solution, PS2Devs sure aren't going to make any more progress on it. FAT32 at least is stable and easily understood, but that 4GB file limit is a real issue for an HDLoader clone (see USB Extreme's workaround-- writing 1GB files).

There are two reasons they went with the existing partition scheme with HDLoader.

1) They already had working code to read/write it.
2) Compatibility with the upgraded PS2 Browser/FF11/Japanese installable HD games

Number two was a serious reason to not even contemplate breaking the existing scheme on. USB Extreme can afford to go its own way since the browser/FF11 will never ever work on the HD over USB.

As much as I'd love to see something easily accessed from the PC (read: a filesystem we can read/write without special tools) it's just not going to be viable. Besides, it breaks backwards compatibility with HDLoader-- do YOU want to sit there all weekend reinstalling your discs for a filesystem change that has no real performance/functionality improvements on the PS2 itself?

dlanor

04-08-2005, 01:18 AM

Oh, I can see where he was going with that, and it's kinda relevant in a way.
I guess you're right, if he meant this as a suggestion for use by HDDemolisher, but he never even mentioned that, so I doubt it.

----- snip -----

There are two reasons they went with the existing partition scheme with HDLoader.

1) They already had working code to read/write it.
2) Compatibility with the upgraded PS2 Browser/FF11/Japanese installable HD games
I think you forgot another equally important point.
3) Compatibility with all homebrew apps for access to non-HDL partitions.

Apart from that I agree with most of your arguments, except one. I really would be willing to reinstall all the games, without any functionality gain on the PS2, IF that could gain me improved file handling compatibility on a PC.

But such a gain is NOT possible, by any means whatever, since the non-HDL partition type used for normal files on a PS2 must remain as it is, for reasons of compatibility as specified in points 2 and 3 above. So such gain is an impossible wish, which we have to drop.

The ability to easily access files on non-HDL PS2 partitions on a PC may instead (and better) be added to PC applications such as WinHIIP etc, as that will not interfere with any existing compatibility.

Best regards: dlanor

TheCrowX

04-08-2005, 02:32 PM

Hi to all
I was wondering what is the current state of this HD Homebrew Project ?
what can it do ight now?
Thanx

strikeuk

04-09-2005, 07:51 AM

I think nobody is doing anything at the moment every since krevnik is busy in real life.

romz

04-09-2005, 08:22 AM

I think nobody is doing anything at the moment every since krevnik is busy in real life.
I don't know about others but I sent some of my achievements to Krevnik few days ago. However that was not placed into CVS yet.

FranktheBunny

04-11-2005, 11:56 PM

Fantastic work, Krevnik. I wish I had something more enlightening or insightful to say, but I'm nothing more than just another clueless end-user....

...I can't wait to see where this goes, and I wish everyone involved all the best of luck.

lazyj

04-17-2005, 12:28 AM

I hope this project is still goin along smoothely :)

TheCrowX

04-17-2005, 02:23 PM

Once upon a time a project called hddemolisher ............... :) :) :)

<G>

04-17-2005, 02:25 PM

its not so stop bumping this damn post. If they have something informative to post they will do it on their own.

Krevnik

04-17-2005, 03:33 PM

Well, yes, I have been very busy in real life... work swallowed my soul, somewhat. However, since this is contract work for Microsoft (thus, hours are strange, and time to myself is difficult... the joys of living at home for a couple months after college), time is a commodity that I don't actually have.

My current plan is to get Romz' reverse engineered code into CVS (my e-mail server is eating attachments for some bizarre reason, another bit of code to debug), and then attempt to move into a more administrative role for anyone else who wants to supply elbow-grease to help make it work. i.e. I will provide hosting, CVS services, my assistance for tasks that need done from time to time, and whatever knowledge I can provide from the various sources I have cobbled together.

Unfortunately, because I didn't expect that work/transportation would consume 12 hours of my time per day until I can find a better/closer place to live to the urban regions of Seattle, I cannot work on this full-time. I apologize to anyone whose hopes I raised during my short stint on this, however, I cannot continue at it as the head developer. This was literally something that requires at least 2-3 programmers, and considering the 3 weeks I worked on it, I was working 40 hour weeks on it, there is little chance that the meager 3-4 hours a week I have could be used to get this up and working in any reasonable timeframe. I might be able to setup some documentation on techniques that could be used for software like this for others, and as I have said, I am willing to be the server/guru guy for anyone else willing to take on the project from here.

Sorry.

taz030485

04-21-2005, 01:27 AM

I have been following this thread for a while and dissapointed that Krevnik is no longer going to have a big part in it :( and now, if you have noticed CrazyC has released a dual-layer patch, that so far works really well.

I was wondering if Krevnik's dual-layer code that has been developed could be used to make a dual-layer PS2 side dumping program :chinscrat and it wouldn't have to be confined to dual-layer, it could be a stand alone program for game dumping, that way people wouldn't have to dump from pc, unless the game required it or if their laser is busted. The program would dump dual-layer games and apply the layer break patch that is needed by CrazyC's method as the image is being dumped. And once finished it could be set to automaticly run hdloader (from specified location) and you could also apply ppf patches on the fly (again a specifed to the patch would be required).

It could also have a network server bult in much like hdl_svr for network dumps.

Just an idea, doesn't have to have all the implementation I mentioned mainly the dumping.

romz

04-21-2005, 06:14 AM

Although Krevnik can't take big part in development of Demolisher now, project is not dead. I continue to do some things. But I did not create Demolisher itself however some of my sources can be used in Demolisher after small modifications. Since there is no head developer for Demolisher project I will try to write C/DVD emulator core ("CDVDMAN" module replacement).

At this moment I have more than 55KB of pseudo source code of "CDVDMAN" module I wrote by doing reverse engineering few different "CDVDMAN" modules. That code can be used to create real C/DVD driver (however some workaround required) or can be very good basis for C/DVD emulator.

Currently some parts of "CDVDMAN" module aren't reconstructed yet: depth of "sceCdLayerSearchFile", interrupt-time disc reading (too many code there), most of ldd io-functions (like open, close, read, etc) and some other functions.

Are there anybody who is going to take part (programming) in this project besides me?

KomoMisk

04-21-2005, 09:26 AM

Isn't it possible to create a web page (or some other place) that all the code can be posted? If possible, even a comparison between the original and what's reverse engineered so that others can check it... (if the original can't be posted just the addresses from where the reversed engineered code is based on).

I would help but I really don't know much about this...

PS: What's the hardest part of this project? Reverse engineering the CDVDMAN?

romz

04-21-2005, 12:52 PM

Hardest part of this project is to make all the things work together. ;) The pseudo sources I wrote have beed sent to Krevnik some time ago. Btw, they are outdated now, since I do a lot of reverse engineering alsmost every day. Addresses from orignal modules are not useless but it's a time consuming task. I include some features that are available one modules and not available in others. I do research even on specific modules from rare software like BB Nav. However I am not planning to include very specific functions (like _sceCdWriteILinkID, _sceCdWriteRTC, etc) in my research in mean time since they are useless for this project.

The replacement for "CDVDMAN" module should include special "cdvdman" module emulator (sceCd* - functions, "cdrom" device) and incorporated both "dev9" and "atad" modules. This is the way how HD loader works, but unlike HD loader I am not going to include "CDVDSTM" module in the "CDVDMAN"'s replacement. I bet on a perfect emulation which will provide all specific functions that "CDVDSTM.IRX" uses.

Another replacement is required for module "CDVDFSV". That module accesses to C/DVD hardware registers by himself - that is not allow during C/DVD emulation. So it is another module to be reversed and reconstructed. I began search on it also just because it helps to understand logic of "CDVDMAN" module itself. I suppose writing the "CDVDFSV" replacement will be more easy than "CDVDMAN".

Source code for homebrew DEV9.IRX and ATAD.IRX is available at PS2DEV and that modules seems to be very compatible with commercial drivers from SCE. It looks like I can use it for C/DVD emulator. HD loader core (as I think) is also a replacement for DEV9 and ATAD modules so I am going to go almost the same way.

It was all about the core of program i.e. C/DVD emulator itself. But actual application needs much more code for GUI, C/DVD emulator installer and so on. Not to mention, Krevnik had plans to write Memory Card emulator as well. ;)

If somebody wants to help with programming it's not even necessary to write huge amout of code. ;) Some tutorial on reading CD or DVD disk image from HDD will be appreciated. However I can found it myself sooner or later but it will take some time anyway. Also I would like to have debugging software for "commercial environment". Homebrew tools will not survive IOP reset which is needed to load IOP module. Since the emulator is aimed to commercial games I want to perform "field tests" with real modules (from the games) loaded. I know ps2link source can be found but it's not an option here as I think. I don't want to use ethernet for tty output and "host:" access for at least two reason: dev9 driver will be part of the emulator's core and need to be debugged too, TCP/IP will consume memory and processor time on IOP. Other possible communication devices are USB and iLink. Communication via iLink is not good idea since the iLink require much more memory than USB driver and new consoles have no iLink port. So the best way (probably) is USB communication. I would like to have source code for USB communication (via PL2301 cable) but I didn't ever find source code for NapLink (except for Linux Client source code which I had to modify to make it work on my PC).

TheCrowX

04-21-2005, 05:36 PM

Good Luck romz :) :-pray:

TiduS`

04-21-2005, 07:21 PM

romz, I may be of some use to you in this project. I've been looking over the current source of demolisher and im slightly confused but i get the jist of it. Due to coding faplink (I have had quite a lot of experience working with hdl partitions and raw reading and writing from the hd itself. If you need some help with this project I may be able to spare some time for you.

On the debugging note, I think you should consider using sio for console output, and maybe host access (if possible?). See here (http://www.0xd6.org/ps2-ee-sio.html) for details.

romz

04-22-2005, 03:52 AM

Current source of Demolisher is available from CVS. Details can be found in the begining of this thread. But it is in very early stages and its most interresting part is IOP reset hijack attempt. Since Krevnik is busy in real life there was no update recently. Even my achievements aren't there.

I am afraid of damaging my console by installing that cable. Even man who installs mod-chips didn't want to install such "hardware" for me. Btw, I don't understand if that can handle "Kprintf" output. I suppose it won't. At this time I can use NapLink or Ps2Link to load programs and view "printf" output. But it will be useless if I'll load IOP image. That's where problems arise.

TiduS`

04-22-2005, 07:01 AM

As i understand it, the IOP sif_handler function is hooked into the address for Reboot2EE, and the irx returns from _start. The EE then reboots the IOP which, in turn sends an sbus interrupt to the EE to ee_sbus_handler right?

Now i noticed that the ee_load_modules command is commented out on the EE sbus handler, if you could load usbd you could be able to communicate through naplink - problem solved. Maybe the ee_load_modules was commented because it caused an exception.

If so it is probably because dprintf and SifExecModuleBuffer can not be called from an interupt handler. Instead you want to create a thread for ee_load_modules in ee_main and signal it on the ee_sbus_handler.

With regards to the serial cable, afaik you can use printf and the console output from the ps2 is sio by default, but is rerouted through ps2link or naplink if they are loaded.

romz

04-22-2005, 09:15 AM

I tryed to load "npm-2301.irx" (or something like that, I forgot exact name) along when IOPRPXXX.IMG is loaded. NapLink client didn't detect anything, so I believe that module requires some RPC support or can not be loaded in such environment.

My current vision of C/DVD emulator installation is to write virtual device driver & IOP reboot hijack program. When EE will try to reboot IOP the hijack program will change path to IOP image, so modified image will be loaded by "UNDL" instead of real one. For example it can change original path like "cdrom0:\IOP\IOPRP300.IMG" to something like this "image0:\IOPRP300.IMG". My virtual device will contain image with both "CDVDMAN" & "CDVDFSV" replaced by special emulation modules. It look pretty simple with except I can load debug-support modules into real games. Of course I can do many important tests using my program as "host game", so I can load extra debug related modules without problems.

TiduS`

04-22-2005, 09:28 AM

I think you probably need to reset the iop, call SifInitRpc(0) and THEN load npm-2301.irx if i recall correctly.

Krevnik

04-23-2005, 10:57 AM

Here is the catch, you can't load modules while in the middle of an interrupt. And while this method will allow you to catch the IOP reset request made, another method should really be looked into. It is possible to catch the request from the EE, and not even need my particular method that I experimented with. It involves patching the Sif interrupt on the EE which is triggered on sif commands and the like.

Something just occurred to me that would allow true injection /without/ creating a new image. The only catch is that you would have to read the IOP image into the EE RAM first, and then extract the modules into the IOP one by one. Or you can manually load the image yourself. Either way the exact format of the image is essential to be known in order to pull this off.

Romz, e-mail me ASAP and let's work out something for CVS access so you can make changes. It might be a little bit of work, since I use SSH for write-enabled accounts, but it should be possible to give you some control over HDDemolisher's CVS tree.

dlanor

04-23-2005, 01:17 PM

Here is the catch, you can't load modules while in the middle of an interrupt.
This reminds me of similar problems I've seen on M68K platforms, where interrupt driven code had to access OS functions not legal to call from within interrupts (due to interrupt masking, as those functions themselves needed to be interruptable). We came up with various ways to do this sort of thing, and I hope something similar can be done with MIPS as well. Basically our solutions boiled down to two main methods.

Unfortunately I'm not well enough informed on the interrupt/exception model of MIPS processors to describe this in proper MIPS terms. So I have no choice but to use the terms I'm familiar with. Hopefully you'll see what I mean anyway.

Method 1: Without returning from the current interrupt, modify interrupt mask and other special registers, after saving current state, so as to simulate a non-interrupt state while calling system functions. Afterwards, restore the interrupt state and proceed as usual. The drawback of this method is that special measures may need to be taken to avoid harmful reentrancy, either for the functions used directly, or for functions used by normal interrupts that may occur due to the patched state.

Method 2: Analyze supervisor stack frames tracing back to a point where the CPU was in user mode before the interrupt. At that point, patch the user stack to insert a normal subroutine return frame leading to the originally interrupted code, and repatch its interrupt return frame (on super stack) so as to lead to your own fix_it() function instead. Then proceed as usual. As you exit from the interrupt handler, that will lead (possibly after additional exits) to your fix_it() function, running in a perfectly normal pre-interrupt state. And when its work is done it can simply make a normal function return to the originally interrupted code.

Like I said, that was for M68K, and the details will surely be quite different with MIPS, but the same principle should be applicable for any CPU type.

Best regards: dlanor

TiduS`

04-23-2005, 02:34 PM

ok, I've attached some code which uses hermes' timer interrupt as an example instead of the sbus interrupt, but the principle is the same. It loads sioman and padman on the timer0 interupt, set to an interval of 1 second. I think It could be a little useful.

vsync

04-28-2005, 06:11 AM

I would really like to help you in this project. I can write C code and I have programmed some things for a TMS370 microcontroller some time ago (so I am familiar with interrupts and so on). However, I have no idea about MIPS and PS2 architecture.

If you think I can be useful for the project even I don't know so much about PS2 programming, please give me some references to tutorials or documents or tell me where to look next. I tried to learn PS2 programming myself before and I got stuck, so please don't send me back to Google ;).

Anyway, good luck!

romz

04-28-2005, 10:21 AM

If you think I can be useful for the project even I don't know so much about PS2 programming, please give me some references to tutorials or documents or tell me where to look next. I tried to learn PS2 programming myself before and I got stuck, so please don't send me back to Google ;).
If you want to learn more about programming for PS2, visit ps2dev.org It will be your starting point to PS2 programming. This project includes the programming for both R5900 (EE) and R3000 (IOP). I don't know much about the programming for the EE but for my opinion the programming for the IOP is kinda like a virtual device driver development for Win9x.

PS: I contacted Krevnik to get write-enabled access to project's CVS so I'll place pseudo source code for "CDVDMAN" modules here as soon as all technical problems will be solved.

vsync

04-29-2005, 05:27 AM

I already tried ps2dev.org, but I found no clear info.

Well, maybe the only way to learn is reading other's code, but I would like to have some hardware knowledge before getting into code. Are there any really helpful docs? Maybe the ones that come with the PS2 Linux distribution, or Sony's SDK. Also, I am not very good at reverse engineering things.

TiduS`

04-29-2005, 06:14 AM

I have a few 'in the works' tutorials at my (temporary) website. These should get you going with regards to the compiler set, see http://uk.geocities.com/kil_frenzy/ . The most docs in the linux or sdk distro are helpful, but most information can be found at ps2dev, either in their doxygen docs or forums.

romz

04-30-2005, 01:26 PM

I managed to load "npm-2301.irx" after IOP reset and it works. Thanks for your advice, TiduS`. Now I have "host" device and "tty" output so it's easy to reload IOP with new images for me.

TiduS`

04-30-2005, 02:17 PM

No problem my friend, if you need any more advice or if I can work with you on this project, send me a PM.

TiduS`

05-03-2005, 06:16 AM

One thing I did notice when I was testing the code myself. I was able to load all of the other modules pad,sio,cd,atad including a fresh copy of the hijacker module, however the ee crashes on padPortOpen. I'm not sure of the cause of it right now, but ill check it out. Incase you guys are wondering what my implementation of the thread idea I posted earlier was, I've included a doctored edition of the demolish.c file. check it out.

TheCrowX

05-14-2005, 03:09 AM

Hi to all I was wondering how is the project is going it s been about 10 days since we didn t hear something new u can tell us if there are improvements :)
many people wish to see this project advancements
Thanx

romz

05-14-2005, 10:18 AM

I can't report much progress with the project right now, it's just going on. But anybody who wanted to see reconstructed code for "CDVDMAN" module can download it now via HTTP from http://www.cdvdmania.com.ru/. Some code for "CDVDFSV" module can be found there also. That code is useful only for programmers and people familiar with reverse engineering.

_zaphod_

05-31-2005, 01:21 PM

I think I speak for all the scene when I say that someone more capable than I should take this code as a base, and write a simple elf.

1) dvd9Installer.elf:this will install dvd9 games into hdloader, and make the patch on the fly. In theory this should be a simple task. SInce hddemolisher can locate the layer break, it should have enough info to apply the patch. Yes it will probably take 45 mins to install a dual layer game.

And while we are at it, we can also make

2) hdfixinstaller.elf:this will install a game and apply the selected patches to the game upon installatiion. CUrrently there are soem string replacements, and a stack relocation patch. also (or instead) make hdpatcher.elf which works on installed games.
3) errorinstaller.elf : this will install games that give fatal error when you try to install in hdloader. There are a few of them, you know. Quite a useful app for our LEGITIMATE OWNERS. (warezers don't need it)
4) backupinstaller.elf (or should we call it warezinstaller.elf?): this is a cogswap loader that will install the swapped game after properly refreshing it's TOC without reloading protection info. I know the eqilvalent was doable on the ps1, it shoudl be possible on the ps2.

romz

05-31-2005, 05:48 PM

Here is a little piece of my work. It's a content of a header file I created for "CDVDMAN " module emulator. There are more than fifty functions I am going to emulate in order to make an "CDVDMAN" emulator fully compatible with modules from the games. During my last research I found couple of little-known functions "supported" by HD loader - they are entries CDVDMAN's entries 79, 80 and 82. It's not like they are available in regular IOP image, I wonder why HD loader do not ignore them like all other such functions.

/* CDVDMAN emulator

File: cdvdapi.h
Description: CDVDMAN API header
*/

#ifndef _CDVDAPI_H_
#define _CDVDAPI_H_

#define SCECdPSCD 0x10
#define SCECdPS2DVD 0x14

/* Structures */
typedef
struct _sceCdlLOCCD {
unsigned char minute;
unsigned char second;
unsigned char sector;
unsigned char track;
} sceCdlLOCCD;

typedef
struct _sceCdlFILE {
unsigned int lsn;
unsigned int size;
char name[16];
unsigned char date[8];
unsigned int flag;
} sceCdlFILE;

typedef
struct _sceCdRMode {
unsigned char trycount;
unsigned char spindlctrl;
unsigned char datapattern;
unsigned char pad;
} sceCdRMode;

typedef
struct _sceCdCLOCK {
unsigned char stat;
unsigned char second;
unsigned char minute;
unsigned char hour;
unsigned char pad;
unsigned char day;
unsigned char month;
unsigned char year;
} sceCdCLOCK;

/* Prototypes */
int emuCdInit(int init_mode);
int emuCdStandby();
int emuCdRead(u_int lsn, u_int sectors, void *buf, sceCdRMode *mode);
int emuCdSeek(u_int lsn);
int emuCdGetError();
int emuCdGetToc(u_char *toc);
int emuCdSearchFile(sceCdlFILE *fp,const char *name);
int emuCdSync(int mode);
int emuCdGetDiskType();
int emuCdDiskReady(int mode);
int emuCdTrayReq(int param, u_int *traychk);
int emuCdStop();
int emuCdPosToInt(sceCdlLOCCD *p);
sceCdlLOCCD *emuCdIntToPos(int i, sceCdlLOCCD *p);
int emuCdCheckCmd();
int _emuCdRI(char *arg1, char *arg2);
int emuCdReadClock(sceCdCLOCK *rtc);
int emuCdStatus();
int emuCdApplySCmd(int cmd, void *wbuf, int wbsize, void *rbuf);
int *emuCdCallback(void (*func)(int));
int emuCdPause();
int emuCdBreak();
int _emuCdMV(u_int *mv, u_int *p); /* This one is not for export in standart versions */
u_int emuCdGetReadPos();
int emuCdNop();
void *_emuGetFsvRbuf();
int _emuCdstm0Cb(void (*p)(int));
int _emuCdstm1Cb(void (*p)(int));
int _emuCdSC(int code, int *param);
int _emuCdRC(sceCdCLOCK *rtc);
int emuCdApplyNCmd(int ncmd, void *nbuf, int nbsize);
int emuCdStInit(u_int bufmax, u_int bankmax, u_int iop_bufaddr);
int emuCdStRead(u_int size, u_int *buf, u_int mode, u_int *err);
int emuCdStSeek(u_int lsn);
int emuCdStStart(u_int lsn, sceCdRMode *mode);
int emuCdStStat();
int emuCdStStop();
int emuCdRead0(u_int lsn, u_int sectors, void *buf, sceCdRMode *mode, int arg5, void *cb);
int _emuCdRV(u_int lsn, u_int sectors, void *buf, sceCdRMode *mode, int arg5, void *cb);
int _emuCdRM(int *m, u_int *p);
int emuCdReadChain(u_int *read_tag, sceCdRMode *mode);
int emuCdStPause();
int emuCdStResume();
int emuCdSpinCtrlIOP(int speed); /* This one is not for export in standart versions */
int emuCdPowerOff(int *stat);
int emuCdMmode(int media);
int emuCdStSeekF(u_int lsn);
int emuCdPOffCallback(void (*func)(void *), void *addr);
int _emuCdSetTimeout(int param, int timeout);
int emuCdReadDvdDualInfo(int *on_dual, u_int *layer1_start);
int emuCdLayerSearchFile(sceCdlFILE *fp,const char *name, int layer);
int emuCdApplySCmd2(int scmd, void *wbuf, int wbsize, void *rbuf);
int _emuCdRE(u_int lsn, u_int sectors, void *buf, sceCdRMode *mode);

#endif

crazyc

06-02-2005, 04:34 PM

During my last research I found couple of little-known functions "supported" by HD loader - they are entries CDVDMAN's entries 79, 80 and 82.

79 seems to be an hdloader id function. It returns 0xdead in the address pointed to by a0. 80 and 82 just return 1.

romz

06-03-2005, 04:40 AM

79 seems to be an hdloader id function. It returns 0xdead in the address pointed to by a0. 80 and 82 just return 1.
Entry point 79 is not just a hdloader id function, however it can be used for such purposes. That function is supposed to be sceCdReadDiskID - it's kinda like a "call 35", which reads some key from disc. HD loader just returns value "1" for 80 and 82 but they are supposed to read some unique data like console ID or iLink ID by sending s-command. I can't realize the meaning of that functions in HD loader. Is it DNAS related stuff?

crazyc

06-03-2005, 02:41 PM

Is it DNAS related stuff?

Seems probable. They are the only functions in DNAS cdvdman and not in regular.

romz

07-10-2005, 11:55 AM

Sometime ago reverse engineering of both "dev9" and "atad" modules was completed. It wasn't a hard since they are close enough to open source modules "ps2dev" and "ps2atad". At this moment I write my own versions of "dev9.irx" and "atad.irx" in order to create open source replacements for commercial modules. The purpose of this process is to make bug-free code for use in C/DVD emulator.
So if somebody wants to help there are two objectives now:
1) I don't really know how to enable lba48 - I checked "ps2atad" source and noticed they write different values to some registers. According the drafts from www.t13.org it looks like lba48 support however I'm not sure it's enough. I'd like someone to check the "atad" pseudo-code at http://cdvdmania.com.ru/downloads.html and explain how to enable lba48 support here.
2) I don't know how to handle SCE's "security" interface. Homebrew "ps2atad" uses standard ATA security for these specific functions. I think there is no need to implement ATA security support in Demolisher, however Demolisher should handle original SCE's hard drives so at least it must have ability to unlock password-protected native hard drives. The real question here is how to emulate SCE's security on standard non-protected drives.