Sega Saturn RAM backup parser

If you got the Croc save from the USB DataLink site, it is the USA version. The creator of the DataLink lives in the USA.

 

Ok... I was thinking about the container suggestion for the SS RAM Backup parser. After some some brainstorm, I had some light of how this file can be used by the program, and I created some pics to illustrate my ideas.



When you click in the Export Container button after select a save to export, a window can pop up like in the image below:

​

The "Save content" section shows how the information is displayed in the SS RAM Backup parser, and the "Save information" section is the information the user can insert manually. The "Save content" section have a Edit button, and if this button is pressed, a new window appears above, and when the user can edit Name, Comment, Lng.code and Date of the save (not Bytes and Blocks) before to make the Export action. After clicking in Export, you must select a place to store the file and name it.



When you click in the Import button, a window can pop up asking to open a file. After selecting the file, another window appears like in the image below:

​

It's the same like the Export button, but you can't edit the "Save information" section. You can edit the "Save content" section like in the Export button, changing the content before to do the Import action.



I don't know if something like this is possible to be done. Anyway, there is my suggestion...

 

It's great to see an active feedback like this. Thank you.

The benefit of the container is that it can contain multiple saves in a single file. Maybe with "Export" button i should add "Add to existing container" button? And also i think i should i support multiple saves export at once, but interface there might be tricky.

Things you proposed are doable. I think container should also include product code, game cover if availabe (i will try to automate these), and maybe PC date (not the saturn one).

 

Well, that's could be a super cool idea, but, as you said, put an interface to endure this mighty be tricky.

My thoughts: The purpose of this idea is to organize backups, where each backup save can be stored individually and separately in a individual file, and put some information about the save in the very same file to easy recognition the content (once the info of the save can't transmit enough information by itself, and skip the task of create a .txt file to recognition purpose). This can help to share the files online, or store multiple backups in a proper folder.

Put all the saves in a single file is work for the save file from the emulator / memory dump...

It's very appreciable to think in more things to enrich the idea, but I think put things like game cover will increase the size of the container too much (I believe a single pic can be bigger than the own save). The Saturn date is to show when was the last in-game update of the file (a PC date will show when the container was created). About adding a product code, I don't know if it an important content, once the idea I put in the picture I created have spaces allowing the user to put the name and region of the game manually (there is a game with two releases in the same region where the save files are incompatible?).

My thoughts: I love things that are simple but complete.

 

Thank you for your Saves Parser utility !
Thanks to its open-source-ness, I could port it to Saturn, and it was great help when manipulating save dump files in uncommon format.

In the case you can help me : do you know how to extract saves from Action Replay format ?
Sample data can be found here : http://ppcenter.webou.net/divers/20110129_darius_ARP.7z
(There's no typo in file name. I dumped it exactly 5 years ago ^^ )

Your utility currently supports saves in "Memory Card Plus" (boot cartridge that simulates official Memory Cartridge) format, but Action Replay format (can only be accessed from Action Replay firmware) is more common among legacy cartridge firmware.
Among other things, adding support for Action Replay save format would allow Pseudo Saturn users to import their saves to internal backup RAM before flashing their Action Replay carts.

I you don't know how saves are handled in Action Replay format, just let me know, and I will try to reverse-engineer it by myself.

 

I don't know the exact format, but i will try to reverse it. Things I already found are:

Bootcode and cheats are in the first half of the flash(128), saves - in the second.
Saves are compressed. Compression is supposedly RLE (good news, it's easily hackable), but there is a chance AR support other types. Compression is different from one used in MemCardPlus.
Saves are preempted with 0x80 0x00 0x00 0x00 sequence. I don't know if it's a copy of the header used by sega library (4th byte should be counter in that case), or just a constant. Met no exceptions to the second case in the file so far.
RLE key is 8 bit, and it's global for a cartridge (it's 0xA1 in this cart, key is stored at 0x20005 and maybe at 0x2000A). RLE sequence A1 XX YY transfers to XX values of YY. RLE sequence of A1 00 transfers to a single A1 byte.

I will try to add the support for this format into my utility soon. If i will be able to reverse the format, i will post infirmation in this topic.

 

In advance, thank you for reverse engineering it
Memory Cart Plus memory usage is limited (16KB + 8KB for both data and RAM code, vs up to 2x1MB for Action Replay), so that's maybe (just a guess) why compression algorithm is different.

 

Done. The support is still buggy (it works only if you choose 64-byte custer), and untested, but at least it works. If you're interested, look at today's commit(s) in repo.
I will explain how it is compressed a bit later.

 

Thank you !
This is a nice present to USB Data Link user since this allows to manage their saves stored in Action Replay cartridge

 

Please do not try to port currently commited code, it still have some unfixed bugs.

Compression is simple RLE. At 0x20000 there is a magic "RLE01" (it also appears in bootcode), then there is a RLE key (1-byte at 0x20005), then compressed data end pointer (4-byte at 0x20006, relative to 0x20000), then goes compressed backup ram image itself. AR compresses whole backup image, not each file (like MemCartPlus does).
Decompression is simple:
- if you meet 0xA1 0x00 sequence, you thansform it into single oxA1 byte (0xA1 is RLE key)
- if you meet 0xA1 0xXX 0xYY (0xXX is non-zero), then you transform it into 0xXX appearances of 0xYY
- in other cases you just copy each byte as-is

And there is a strange thing. After the compressed image, there are some uncompressed saves. I haven't yet checked if they overlap with compressed ones after decompression, but i will do so soon. Current code in git repo precesses them incorrectly (as if they are compressed).

 

No problem, I didn't started (lack of free time ...), and need to add some user interface related stuff before porting code from your project.

Interesting information
I thought that each save was compressed individually.

If RLE compressed data size is stored somewhere, then theses uncompressed saves are just the remains of a buffer that Action Replay firmware forgot to clear before using it ? This is just a guess, and probably wrong.

 

Okay, now repo contains a fixed code (still untested though). But actually i cheated a bit. I decompress a full image, and then repack cluster size to the one provided by user. I'm not sure you will be able to use this code inside saturn due to the memory constraints.

It may be right, because this part contains the same saves the compressed image does. But compressed image decompresses precisely into 4 MBit, so they overlap. I ignore uncompressed junk for now, but i might add and option to import this junk instead of the compressed image.

 

Thank you ! I will port it to Saturn soon.
When I ported dode to parse official memory cart data, I had to put some tweaks in order to fit Saturn memory requirements (allocation tables for all saves was declared in memory, and this was too much for Saturn's RAM), so I will try to do something (I don't already know what) in order to parse Action Replay dumps on Saturn.

I don't know if previous deleted saves are kept in this junk or not, but if it not the case (=only compressed saves are stored in junk area), then it is not worth the hassle to import theses.

 

I finally could start to port Action Replay parse code :]
I'm surprised that Action Replay save mode is exactly the same as the one from Saturn BIOS ... it seems that Datel was very well informed about Saturn's BIOS internals ^^


And, I have few questions about ss-save-parser :

Q1. Why are you expanding Action Replay image size to 512KB ? Typical size of Action Replay flash ROM is 256KB.
Edit : A1. In order to handle decompressed data as 512 KB official memory cart dump file.

Not Action Replay related :
Q2. Are you sure maximum size for cluster table is 2048 ?

Q3. Why are you detecting cluster size 16 bytes per 16 bytes ?
Detecting it 64 bytes per 64 bytes would be safer and would avoid some errors with too small (16 or 32 bytes) cluster sizes.
(If save with cluster size not multiple of 64 bytes exist, then detecting 16 bytes per 16 bytes is required, but I didn't saw cluster not multiple of 64 bytes until now.)

Q4. Are you sure that cluster allocation table is stored in contiguous clusters ?
If not, ParseSat function needs to be improved in order to read cluster table while filling it.

Also, I have a request about the UI in ss-save-parser : can you display somewhere (example : bottom of the window) which dump file name was opened ? Because it is easy to forget in which instance a given file was opened when comparing several dump files.

Edit : "But compressed image decompresses precisely into 4 MBit" :
After some testing here, I finally realized I mistook between MBit and MB ... so now I understand about Q1
Since decompressed data is smaller than I though, everything is now simpler than expected, because decompressing everything to second half of Low work RAM should do the job here too.

Edit : It seems to work here, thank you very much for Action Replay format addition


I changed my utility in order to handle non contiguous cluster allocation table and discard cluster size not multiple of 64 bytes. Please let me know if you want the sources in order to update ss-save-parser sources.
Please note that it is not tested for real yet, and that language is slightly different, so some hand work may be needed when merging changes.

 

Good job! It seems i should return to active parser development soon.

Maybe they simply used some of the BIOS subroutines? ^_^

Theoretically it is not. Maximum CS1 memory map is 128MBit (16 MByte), but i have not tested BIOS support for power memory more than 32 MBit (8 MByte). Even for the last one, there is a 64K blocks number, and if cluster size is 64 bytes, then save can theoretically take up to 60000+ clusters, making clusters table bigger than 100KBytes. If my code does have a 2048 limitation is definitely a bug. Can you please point where it is?

By the way, are there any games out there with more than 461 blocks save, so that it won't fit into internal SRAM? I'm sure i heard something about a game like this before... If they are, i should definitely test them.

I don't think 192 is a valid size too, but i don't actually know which ones are supported by BIOS. So i left the minimum granularity as the size of a single "BackUpRamFormat" token, just in case, to get maximum versatility. 16 and 32 are impossible ones (they won't be able to hold 36 bytes required for header and first allocation table instance).

Hey, you've got me there! It could be fragmented all rigth, i should fix this.

Okay.

Great! I'm glad to be of help.

I think i can fix it myself. Thank you for your support.

 

I didn't looked at what Action Replay firmware was doing, but that's highly probable. BUP library is relatively small, so reverse-engineering it isn't a big work.

Well, this is a really worst case scenario ^^;

config.h -> "short SAT[2048];"

Dezaemon 2 is using 100+ KB sized "monster saves". I use this game as a sadistic test pattern for my cartridge (and of course it doesn't works ).
If you can dump your memory cart and want to test a real world worst case scenario, I would advice you to use its Save Game Manager application in order to fill your memory cart with test save data.
There is no real surprise with this however : memory cart will be formatted with 256 bytes block length, and each saves would take around 500 blocks each.

I understand your way of computing block length, but this gave me problems with SaturnSave_1.dat save data here : its fourth token is corrupted.

Hehe, glad it helped
In the practice it doesn't causes problem, because header and allocation table generally fit in few blocks.
(I actually couldn't find save fragmented up to that point, so maybe BIOS can't handle fragmented allocation table.)

OK ! Yes, it's not very difficult to change. I think that porting from my source would be harder than directly fixing it

 

One more Action Replay dump :]
http://ppcenter.webou.net/divers/20160305_FlashSave000.7z
It seems that RLE key is different than in previous dump ? And this one incorrectly decompresses in 167346 bytes (163 KB).
Please let me know if you have an idea about why it doesn't decompresses in 512KB

To people owning Action Replay with saves and USB Data Link : don't hesitate to post your Action Replay flash ROM dumps here


Edit : compression uses two different RLE keys ? (0xDE and 0xE3)
Don't know why, but that seems to be okay like this. Will try next week.