Save Game Extractor: Backup Saves Via Audio

[slinga]

I just released the first beta of my latest project, Save Game Extractor (SGEX). SGEX transmits your saves via audio and let's you reassemble them on your PC.

Save Game Extractor (SGEX)
Transmit Sega Saturn save game files as audio to backup on a PC. Based on the amazing minimodem. Currently transfer speeds are ~70 bits/second. Build with Jo Engine or download an ISO from releases.

Demo Video
Demo Video

Screenshots

 

[cafe-alpha]

Someone just emailed me saves dumped with SGEX, and this was a pleasure for me to add them to Pseudo Saturn Kai' Save Data Manager 🙂 Thank you for the amazing project !

 

[slinga]

Glad people are using SGEX. Just curious, how large were the saves? I'll add bzip2 like you mentioned in the other thread.

 

[cafe-alpha]

Total is around 133KB :

  5,056 SEGARALLY_0.RAW
29,440 TEITOKU3_01.RAW
40,356 TEITOKU3_02.RAW
40,356 TEITOKU3_03.RAW
  1,888 T_FORCEV_00.RAW
19,964 WAD_MAPS_00.RAW

And I was told to wait for 15 hours to receive another save ^^ So adding compression may help a lot.

 

[slinga]

Lol holy crap. Yeah compression is a must, I feel super bad. ~133KB is ~4.3 hours at 70 bits a second.

I looked into bzip2 and it looks like it requires a ton of memory. The official docs says 2.2-3.7 MB of RAM for the compressor. I'm not sure how Rockin'B did it.

 

[cafe-alpha]

With Tamagotchi Park save, I did a "raw" dump of first 1MB from CS-1 space, and that 1MB ... fitted into internal memory (32KB) after compression. Because Tamagotchi Park save data is full of 00h, that's a champion result, but in typical case it's not negligible.
BTW, the saves I received yesterday fit into 67KB (input size = 133KB) with "fastest" zip compression preset of 7-Zip utility. So this would nearly double the transfer with SGEX.

I looked into bzip2 and it looks like it requires a ton of memory. The official docs says 2.2-3.7 MB of RAM for the compressor. I'm not sure how Rockin'B did it.

Save data maximum size supported by Saturn BIOS is 256KB, so with this upper limit it may be possible to estimate bzip2 requirements to a reasonable amount of RAM.
Personally, as I'm not super keen on using algorithms requiring dynamic memory allocation on Saturn, I'm using LZF ( Marc Lehmann's "LibLZF" ) in Pseudo Saturn Kai. Result is not as good as bzip2, but it doesn't uses malloc/free and doesn't chews too much CPU time too.

 

[slinga]

I went with miniz which is based on zlib. I tagged a new release: Release Compression Support · slinga-homebrew/Save-Game-Extractor.

Not much testing but I was able to transfer 3 compressed saves from real hardware just now. Requires using the updated Python script as well.

Thank you for the tip regarding the maximum save size, I will adjust my maximum. I think I was setting half of that as the max.

 

[cafe-alpha]

It's good you could find an appropriate compression algorithm. Let's hope that it will help to contribute to dump even more save data 🙂

Regarding the 256KB limit, it is written in a SEGA technical doc (I don't remember which one) and AFAIK there's no game reaching such upper limit, but Dezaemon 2 may generate saves larger than 200KB.

 

[Solitario]

I'm not able to run the sgex.py escript it give me this error

 

[Waterfuell]

I'm not able to run the sgex.py escript it give me this error

View attachment 5332

Looks like the script is made for python2 and you are using python3.

@slinga must confirm.

 

[Solitario]

This homebrew was just what I was looking for, thanks bro

 

[slinga]

Glad it worked. It's looks like you are using version 0.95, 0.96 added compression.

I looked at getting Python3 support to work and just got frustrated and stopped. Need to look into again.

 

[slinga]

I updated the Python script to require Python3 and to fail gracefully on Python2.

 

[rorirub]

Unrelated to this, but that savegame manager you released for the Satiator, can you add support for downloading/uploading entire parts of memory, perhaps with some pre-set values?
I'd like to use it to download the entire save cart in one go, and then restore it from another file. This could be also used for dumping bios, rom carts, etc. It would save me the trouble of desoldering the chips for reading in a ROM programmer.

also I have something like 100 saves in the cart and it would take ages to copy them all one by one, it would be faster if I could archive the full cart into a single file, then restore it to another cart from a single file. And some emulators use save data in single file format too.

 

[slinga]

Sure that shouldn't be too bad. I just need to think of how to I want to do it in the UI.

 

[rorirub]

I'd recommend two menu options, "download from memory" and "upload to memory", where you can then select from a list of memory locations corresponding to some more important presets; such as memory cart, entire cart port, boot rom, WORK RAM LOW, WORK RAM HIGH (to load game binaries - not sure if this is possible since it would overwrite the current code), just from the top of my head. This would get around to needing to handle number/text inputs, which in turn would get around to needing a UI for inputting characters with the controller. The filename for saving could be automatic based on the preset, with a number suffix if it already exists on the SD card. For uploading however, you'd need a file browser where you can press a key to upload the file to a selected location.

Bonus if you can also dump the CDB firmware - that and file transfers were the two major things I wanted from the satiator.

 

[slinga]

Bonus if you can also dump the CDB firmware - that and file transfers were the two major things I wanted from the satiator.

That would require something on the Satiator firmware side. AFAIK there is not a way to dump the CDB firmware from the SH-2. I think it's worth telling Abrasive to add that feature.

Dumping WORK RAM LOW, WORK RAM HIGH would just give you Save Game Copiers memory. The other regions make sense.

 

[fafling]

Regarding the 256KB limit, it is written in a SEGA technical doc (I don't remember which one) and AFAIK there's no game reaching such upper limit, but Dezaemon 2 may generate saves larger than 200KB.

That's in Sega Saturn Software Development Standards p. 51.
But that's just the upper limit if the bios backup library is used. So it's possible to produce bigger saves than 256 KB with a custom backup function.

 

[rorirub]

That would require something on the Satiator firmware side. AFAIK there is not a way to dump the CDB firmware from the SH-2. I think it's worth telling Abrasive to add that feature.

Dumping WORK RAM LOW, WORK RAM HIGH would just give you Save Game Copiers memory. The other regions make sense.

I've been asking him for ages about CDB dumping, and all he said is that it's all open source, someone can write it...

For WORK RAM, it's obviously for uploading only since downloading makes no sense. I'm not sure if uploading is the correct terminology there, what I meant is for the ability to load game binaries without needing to convert them to ISO. I think you'd also need a way to set the SH2 stack for doing so.

 

[slinga]

I've been asking him for ages about CDB dumping, and all he said is that it's all open source, someone can write it...

I threw a ticket in on Satiator-Menu. It's not clear to me how to dump the CDB via the Satiator API. Again I imagine Professor Abrasive will have to add new functionality in the firmware.

For WORK RAM, it's obviously for uploading only since downloading makes no sense. I'm not sure if uploading is the correct terminology there, what I meant is for the ability to load game binaries without needing to convert them to ISO. I think you'd also need a way to set the SH2 stack for doing so.

I apologize, I understand what you are asking for now. Writing a binary loader seems like feature creep for a save game copier, but the memory downloader is reasonable.