cb2util v1.1 - compile your own CodeBreaker "cheats" files and more

Quote:

---

Originally Posted by misfire

@dlanor

That's weird, indeed. Mind sending me your cheats file so I can take a look?

---

Sure, just PM me about what email address to use.

Quote:

---

There's actually a byte that determines the type of a cheat description. AFAIK, its value is 4 if the description has no codes and 0 otherwise. Maybe there're more values cb2util doesn't understand yet...

---

Even that one value could do nicely for explaining some of those 'comment lines'.

But in some places it really seemed as if someone was trying to use a kind of 'subfolder' within the cheat list for a game.

Quote:

---

Nevertheless, I'll be tagging v1.3 with the improvements today.

---

Good, because v1.3 certainly is a great improvement over v1.2, regardless of any oddities I found in my cheat lists.

Come to think of it, some of those oddities may have resulted from CB bugs in its database maintenance. I am 100% sure that such bugs exist, as I have on some occasions suffered trashing of a database when I simply tried to use CB to delete some cheats, though on other occasions I was able to do so by handling them in a different order (thus apparently circumventing some bug).

Best regards: dlanor

@dlanor

I had a look at the cheats file you sent to me. I slightly modified cb2util to output the description type values and my assumption was correct. The value is 4 if the description has no codes and 0 otherwise (there're two or three games that use 1 instead of 0, don't know why, and it doesn't really matter).

So Pelican really screwed some codes up... :facepalm:

Quote:

---

Originally Posted by misfire

@dlanor

I had a look at the cheats file you sent to me. I slightly modified cb2util to output the description type values and my assumption was correct. The value is 4 if the description has no codes and 0 otherwise (there're two or three games that use 1 instead of 0, don't know why, and it doesn't really matter).

So Pelican really screwed some codes up... :facepalm:

---

I thought as much, but I think the cheat sets for some games may have used some descriptions without codes just to insert comments into a cheat collection, so that a cheat with complex codes might be preceded by such comments describing how to use it.

I assume that the '-c' compile command accepts such cheat descriptions followed by no codes (but another cheat description instead), so that such cheats remain valid when recompiled.

For now I guess cb2util v1.3 is to be regarded as the 'final' version. Right ?

Best regards: dlanor

@misfire:
After cleaning out the original Pelican errors from my cheats file, working on a decrypted text file produced by cb2util, I decided to repeat the same kind of test I made before to verify the validity of the recompile process.

So I started with a "cheats_fixed.txt" mentioned above, which is a first level decryption by cb2util v1.3, which I have then cleansed of the errors discussed earlier.

I then recompiled this to a file I named "cheats_fixed.cht", and then did the two types of extraction same as before, so as to get both a "cheats_fixed.asis.txt" and a "cheats_fixed.decr.txt". And then I used WinHEX to compare these two files with each other, byte for byte, which proved that they were fully identical, just like they should be when starting with fully decrypted and valid cheats as in "cheats_fixed.txt".

But for such a case of perfect recompilation and extraction, the resulting new txt files should also be identical to the original "cheats_fixed.txt" file, so I repeated a similar WinHEX file comparison again, this time comparing the 1st-generation "cheats_fixed.txt" with the 2nd-generation "cheats_fixed.asis.txt". And they did not match at all... !!! :(


It turns out to be your 'comment' convention that messed things up again, and this time in such a way that I think you will have to modify your implementation of comments for recompilation.

The original cheat collections from Pelican contain tons of cheat descriptions where the character '#' is used as an abbreviation for the word "number" which means that something important nearly always follows after that character. So some part of the cheat name/description that may be necessary for the user to understand what the code is for, or how to use it, is therefore destroyed when your recompiling code removes all text starting with the # and on to the end of that line.

Here are some examples of vital information destroyed by this comment implementation when recompiling cheats.

Phantom Brave:
"Max Remove (# Of Turns Available In Battle)" => "Max Remove ("

Dragonball Z Budokai:
"#16 Codes" => line removed
"#17 Codes" => line removed
etc...

Dragonball Z-Budokai 3:
"Android #16 Codes" => "Android"
"Android #17 Codes" => "Android"
etc...

Fantastic 4:
"Selected Panels From 'Ultimate Fantastic Four' #1" => "Selected Panels From 'Ultimate Fantastic Four'"
"Selected Panels From 'Ultimate Fantastic Four' #2" => "Selected Panels From 'Ultimate Fantastic Four'"
etc...

Indigo Prophecy:
"Artwork #1" => "Artwork"
"Artwork #2" => "Artwork"
etc...

These are just some very few of the occurrencies of '#' in cheat descriptions of the original cheats collection of CodeBreaker. There are tons and tons more of this stuff, and this makes it unacceptable for cb2util to treat '#' as a comment header in the way that it currently does, so that it and all text following it on a line is removed from a recompiled cheats file.

This is very different from the use of "//" that we discussed earlier, in two significant ways:

1: "//" is not used at all in the original cheat collections.

2: "//" is not commonly used as a standard abbreviation in the english language, whereas the '#' character has traditionally been used as an abbreviation for "number" since a very long time back. (Long before the first computers were invented.)

I hope you will see this as I do, and fix the problem with a new v1.4 release where the recompiler treats '#' as a normal text character. Surely it must be enough to have the "//" sequence for inserting pure source comments that will not be included in the compiled result.

Best regards: dlanor

Quote:

---

Originally Posted by daisensou

@kevstah2004

Sounds good!

Can it be of any size? Because, Codebreaker normally creates a 512MB partition... so you can create a smaller (128MB) and it'll still work without problems?

OMG, HDD support for Codebreaker without wasting (much) space T_T



So the settings don't get saved? Or they get saved but reset when you change the partition name back and forth?

(Sorry for going offtopic!)

---

The settings file is saved to a new to config file called hddconfig instead, I haven't messed with changing the partition size yet. When I said copy all the files from the mc PCB folder, I should of said you don't need to copy the config file or icon files, I'm still trying to figure out the HDD icon format as it's different to the mc format.

Quote:

---

Originally Posted by dlanor

I thought as much, but I think the cheat sets for some games may have used some descriptions without codes just to insert comments into a cheat collection, so that a cheat with complex codes might be preceded by such comments describing how to use it.

I assume that the '-c' compile command accepts such cheat descriptions followed by no codes (but another cheat description instead), so that such cheats remain valid when recompiled.

For now I guess cb2util v1.3 is to be regarded as the 'final' version. Right ?

Best regards: dlanor

---

Yes, cb2util compiles "comments" just like the CB expects them. And yes, v1.3 is the official new release.

Quote:

---

Originally Posted by dlanor

@misfire:
After cleaning out the original Pelican errors from my cheats file, working on a decrypted text file produced by cb2util, I decided to repeat the same kind of test I made before to verify the validity of the recompile process.

So I started with a "cheats_fixed.txt" mentioned above, which is a first level decryption by cb2util v1.3, which I have then cleansed of the errors discussed earlier.

I then recompiled this to a file I named "cheats_fixed.cht", and then did the two types of extraction same as before, so as to get both a "cheats_fixed.asis.txt" and a "cheats_fixed.decr.txt". And then I used WinHEX to compare these two files with each other, byte for byte, which proved that they were fully identical, just like they should be when starting with fully decrypted and valid cheats as in "cheats_fixed.txt".

But for such a case of perfect recompilation and extraction, the resulting new txt files should also be identical to the original "cheats_fixed.txt" file, so I repeated a similar WinHEX file comparison again, this time comparing the 1st-generation "cheats_fixed.txt" with the 2nd-generation "cheats_fixed.asis.txt". And they did not match at all... !!! :(


It turns out to be your 'comment' convention that messed things up again, and this time in such a way that I think you will have to modify your implementation of comments for recompilation.

The original cheat collections from Pelican contain tons of cheat descriptions where the character '#' is used as an abbreviation for the word "number" which means that something important nearly always follows after that character. So some part of the cheat name/description that may be necessary for the user to understand what the code is for, or how to use it, is therefore destroyed when your recompiling code removes all text starting with the # and on to the end of that line.

Here are some examples of vital information destroyed by this comment implementation when recompiling cheats.

Phantom Brave:
"Max Remove (# Of Turns Available In Battle)" => "Max Remove ("

Dragonball Z Budokai:
"#16 Codes" => line removed
"#17 Codes" => line removed
etc...

Dragonball Z-Budokai 3:
"Android #16 Codes" => "Android"
"Android #17 Codes" => "Android"
etc...

Fantastic 4:
"Selected Panels From 'Ultimate Fantastic Four' #1" => "Selected Panels From 'Ultimate Fantastic Four'"
"Selected Panels From 'Ultimate Fantastic Four' #2" => "Selected Panels From 'Ultimate Fantastic Four'"
etc...

Indigo Prophecy:
"Artwork #1" => "Artwork"
"Artwork #2" => "Artwork"
etc...

These are just some very few of the occurrencies of '#' in cheat descriptions of the original cheats collection of CodeBreaker. There are tons and tons more of this stuff, and this makes it unacceptable for cb2util to treat '#' as a comment header in the way that it currently does, so that it and all text following it on a line is removed from a recompiled cheats file.

This is very different from the use of "//" that we discussed earlier, in two significant ways:

1: "//" is not used at all in the original cheat collections.

2: "//" is not commonly used as a standard abbreviation in the english language, whereas the '#' character has traditionally been used as an abbreviation for "number" since a very long time back. (Long before the first computers were invented.)

I hope you will see this as I do, and fix the problem with a new v1.4 release where the recompiler treats '#' as a normal text character. Surely it must be enough to have the "//" sequence for inserting pure source comments that will not be included in the compiled result.

Best regards: dlanor

---

Good point. In fact I had already noticed this behavior when adding such a big cheats file to my test suite for v1.1. I had to change libcheats to not treat '#' as a comment to make the tests pass.

So yes, I think it's a sensible thing to just ignore '#' in the long term.

@dlanor

I attached a custom build of v1.3 that doesn't treat '#' as comment.

That's all I had to change: #1436637 - Pastie

Now you're able to continue testing. :)

Quote:

---

Originally Posted by misfire

@dlanor

I attached a custom build of v1.3 that doesn't treat '#' as comment.

That's all I had to change: #1436637 - Pastie

Now you're able to continue testing. :)

---

Tests completed with full success !!! :cheers:

Both decrypted texts (as-is and decrypted) from the cheat collection I recompiled (after fixing its errors) are now fully identical to each other as well as to the source file of that recompilation.

This means that it is finally possible to go 'full circle' with this decryption and recompilation method without losing any significant part of any cheat, whether in codes or descriptions.

And though you named this download file '.dirty.zip', it is really the cleanest one yet, looking at its end results. :lol:

So at this point I have no further complaints or requests, except that this should be made into a proper release. :)

Btw:
One result that I've neglected reporting on earlier, but which really deserves mention, is the huge savings of MC space that this program allows for a user who has the cheats file stored there (like we have to do with every console not having internal HDD).

The cheat file I started with had a size of 898 KB, but decryption removed so much needless complexity that the efficiency of the packing increased considerably, so that the final recompiled cheat file is only 579 KB. That's a savings of 319 KB, == over 35% of the original size. (Enough for appx 5 gamesaves of "Final Fantasy X")

Btw 2:
I almost forgot to mention that I have of course verified that the recompiled cheats also work properly when loaded into CodeBreaker on the console.

Best regards: dlanor

Quote:

---

Originally Posted by dlanor

Tests completed with full success !!! :cheers:
[...]
And though you named this download file '.dirty.zip', it is really the cleanest one yet, looking at its end results. :lol:

---

Cool. :)

Actually, the version is auto-generated from the output of git describe. All I have to do is run "make release" to compile sources and create a zip/tar archive of the result. Very convenient.

"-dirty" just means that I had uncommitted changes (the comment thing) in my working tree when I built the release.

FYI: I've just released cb2util v1.4 which mainly updates the documentation and tests, and includes the parser improvement requested by dlanor above. (I know it's old hat, but there's never been an official cb2util release incorporating that fix.)

The full changelog:
https://github.com/mlafeldt/cb2util/...14-jul-16-2012

Download of Windows and Linux builds:
https://github.com/mlafeldt/cb2util/downloads

The new project site:
cb2util by mlafeldt

In case you're wondering why I bother updating cb2util at all, have a look at "GoodCBC" - LET'S DO IT! (Codebreaker Day1 set decrypted and recompiled into RAW cbcs