TrekkiesUnite118
Established Member
Can you test your patched image in an emulator and verify that works?
- Thread starter TrekkiesUnite118
- Start date
nanash1
Established Member
I'm still looking into the Grandia FMV codec, but the progress is much slower now for two reasons. Time and motivation. When I started the project I was on vacation and had a lot of free time, so in the beginning I basically worked full time on it. Now I'm working again and I'm not that motivated to spend most of my free time working on the FMV codec. I'll also admit that reverse engineering the code is what's the fun part for me. That is pretty much done though and all that's left is writing the code for the encoder and that's is more work than fun for me. The same is true btw for the Cotton 2 project.
But next week is the last week I'm working this year and thanks to Covid there's not that much going on, so I'm hopeful that I'll have more time and be more motivated. I don't want to over promise, but I'm optimistic that I'll have a functioning beta of the encoder done by the end of the year.
But next week is the last week I'm working this year and thanks to Covid there's not that much going on, so I'm hopeful that I'll have more time and be more motivated. I don't want to over promise, but I'm optimistic that I'll have a functioning beta of the encoder done by the end of the year.
nanash1
Established Member
My encoder is now at the point where it can output frames, meaning the core functionality is done. The next step is testing. But before I can begin I'll need a muxer, since I need to insert my test data into the games ROM to see if it's read correctly. @TrekkiesUnite118 did you already write a muxer? I can only find a demuxer on Github.
I also thought a bit about the best way to insert subtitles into the videos. As I said before, re-encoding lossy sources is generally a bad idea. Especially when they're already heavily compressed. But since Grandias codec doesn't use any advanced features like motion prediction, each 16x16 macroblock is independent as far as the lossy part of the compression is concerned. That means it's possible to only re-encode the macroblocks that are overlayed by subtitles and leave all other blocks untouched. The only potential problem with this method, that I can see at the moment, is the compression level. The compression level is defined per frame and can't be changed without completely re-encoding a frame. So if the compression level is high enough to make subtitles unreadable, it wouldn't be possible to change it just for the subtitle blocks.
I also thought a bit about the best way to insert subtitles into the videos. As I said before, re-encoding lossy sources is generally a bad idea. Especially when they're already heavily compressed. But since Grandias codec doesn't use any advanced features like motion prediction, each 16x16 macroblock is independent as far as the lossy part of the compression is concerned. That means it's possible to only re-encode the macroblocks that are overlayed by subtitles and leave all other blocks untouched. The only potential problem with this method, that I can see at the moment, is the compression level. The compression level is defined per frame and can't be changed without completely re-encoding a frame. So if the compression level is high enough to make subtitles unreadable, it wouldn't be possible to change it just for the subtitle blocks.
TrekkiesUnite118
Established Member
Right now there's only the demuxer, though I think a muxer could probably be made by just doing the reverse, there was a value in the frame table I wasn't sure what it was used for:
I know the 0x000C denotes the end of a run of frames, but value that comes immediately after I'm not sure what it's for.
EDIT:
Looking into it more a muxer shouldn't be too hard to make. Basically you need to update the table that's at 0x280 in the .MOV file with the new sizes for your new frames. Each run of frames is the amount of frames in 1 second. When you get to the last run, the number of frames is simply how ever many frames are left. So if the frame rate is 10fps, you'll have the size value of 10 frames in each run, and the last run will be 10 or less frames.
Muxing the audio works like this:
- Starting at 0x4000, put the first 0x18000 bytes of ADX data
- Then put the first run of frames (if your frame rate is 10fps, this will be the first 10 frames.)
- Then put the next 0x6000 bytes of ADX data.
- Then put the next run of frames (again if 10fps this will be 10 frames.)
- Repeat 3 and 4 until you run out of frames.
- If the FMV is full screen, the frame rate is 10fps.
- If the FMV is letterboxed, the frame rate is 12fps.
Last edited:
TrekkiesUnite118
Established Member
Ok, I have a first sloppy go at a Muxer:
github.com
You'll need to use the updated version of the Demuxer as well. This will now write each Demuxed Frame as it's own file in a Frames directory. The Muxer will then use those frames to Mux the file, using the QuickTime header of the original file. I've tested it with both 12fps and 10fps videos and I'm getting out files that match the originals, so hopefully this works.
GrandiaTranslation/GrandiaTranslationTools/src/com/grandia/file/VideoMuxer.java at master · TrekkiesUnite118/GrandiaTranslation
Collection of Tools and Info for Translating Grandia for the Sega Saturn. - TrekkiesUnite118/GrandiaTranslation
github.com
You'll need to use the updated version of the Demuxer as well. This will now write each Demuxed Frame as it's own file in a Frames directory. The Muxer will then use those frames to Mux the file, using the QuickTime header of the original file. I've tested it with both 12fps and 10fps videos and I'm getting out files that match the originals, so hopefully this works.
nanash1
Established Member
I was able to insert my first encoded frame into the ROM. For now I just used a hex editor to insert it manually. Not unexpectedly the game crashed at first, but now I managed to make Yabause at least display some decoded data before crashing. So here are the results.
The first image is the source, the second is the Yabause screen cap. The video I'm currently using for the test is a Youtube rip of the intro of the "enhanced" edition. Obviously there are still some major problems, but it's a start.
The first image is the source, the second is the Yabause screen cap. The video I'm currently using for the test is a Youtube rip of the intro of the "enhanced" edition. Obviously there are still some major problems, but it's a start.
If you use something without a gradient, does it still butcher the edges? Like for example pure white letters on black background without the gray elipse?
nanash1
Established Member
I haven't tried that, but since even the grey gradient gets these line interferences, I doubt it would make a difference.
I suspect it has something to do with the DCT. At the moment I basically just converted the fixed point iDCT matrices from the game to floating point and inverted them. Then I calculate the matrix product and round the results for the DCT transform. Maybe that approach is to simplistic and I've to account for the rounding errors that are caused by the fixed point math the game uses for the iDCT transform.
TrekkiesUnite118
Established Member
Also keep in mind that Yabause doesn't really handle Grandia's FMVs all that well at times.
Also I think I've figured out what that value in the Frame Table I was unsure about is. It looks like it's the number of sectors each run of Frames takes up.
Also I think I've figured out what that value in the Frame Table I was unsure about is. It looks like it's the number of sectors each run of Frames takes up.
nanash1
Established Member
I found the problem with the DCT. Turns out I interpreted hex values of the iDCT matrix as unsigned when they were actually signed. This also clears up my confusion about the DCT coefficients that were offset by +1. They aren't actually offset. The game uses a normal DCT transformation after all. Now I just wonder how this bug didn't show up before in any of my tests...
Anyway, here is what my frame looks like after fixing the DCT. Seems like there are only some minor glitches left to fix. I suspect the color errors at the edges are caused by the chroma subsampling. I think they're also visible in the original movie.
Anyway, here is what my frame looks like after fixing the DCT. Seems like there are only some minor glitches left to fix. I suspect the color errors at the edges are caused by the chroma subsampling. I think they're also visible in the original movie.
Last edited:
TrekkiesUnite118
Established Member
Yeah, that's been translated for a while now. @nanash1 was able to figure out the compression used for those graphics.
nanash1
Established Member
I'm pretty sure the lossless part of the compression and the DCT are working fine now, but there is still a problem with the color space transformation. I found the cause of the problem, but I'm not sure how to deal with it. Maybe someone here has a good idea for a solution.
This is what my current test image looks like. It's a different frame, because it's better suited to explain what happens.
We can see a lot off-color pixels around the edges of the letters. I will explain how this happens for the vertical yellow strip left of the letter "E". The original (y,cb,cr) values for these pixels are (121,0,0). These values are than encoded with the quantization and DCT. Because this process is lossy our decoded values have errors. The stronger the compression the higher the errors. In this case I used the compression level 10 and the decoded values are (120,-8,-1). On first glance these errors seem reasonable and not particularly high. Since these are signed 8 bit integers an error of 8 equals about to 3%. The game then uses this equation to calculate the RGB values:
g = (y+124)+cb+cr = 235
b = (y+124) - 2*cb = 260
r = (y+124) - 2*cr = 246
Now here comes the problem. While doing this calculation the game doesn't bother to check for under- or overflows. Thus the blue value overflows to 4 and we get a strong yellow tint. This is a simple problem that is not so simple to solve, especially when we keep in mind that the game throws out every second data point of the cb and cr channel during subsampling. I can't really predict the errors that the DCT introduces without actually decoding the video. This means in order to correct overflow situations I would have to effectively do two passes of the encoding. Another solution would be to try to introduce some head and foot room into the calculations by offsetting the luma channel. This would impact the image details in bright and dark spots though and certainly wouldn't fix overflows caused by the subsampling.
So does anyone here know a better solution than two encoding passes?
This is what my current test image looks like. It's a different frame, because it's better suited to explain what happens.
We can see a lot off-color pixels around the edges of the letters. I will explain how this happens for the vertical yellow strip left of the letter "E". The original (y,cb,cr) values for these pixels are (121,0,0). These values are than encoded with the quantization and DCT. Because this process is lossy our decoded values have errors. The stronger the compression the higher the errors. In this case I used the compression level 10 and the decoded values are (120,-8,-1). On first glance these errors seem reasonable and not particularly high. Since these are signed 8 bit integers an error of 8 equals about to 3%. The game then uses this equation to calculate the RGB values:
g = (y+124)+cb+cr = 235
b = (y+124) - 2*cb = 260
r = (y+124) - 2*cr = 246
Now here comes the problem. While doing this calculation the game doesn't bother to check for under- or overflows. Thus the blue value overflows to 4 and we get a strong yellow tint. This is a simple problem that is not so simple to solve, especially when we keep in mind that the game throws out every second data point of the cb and cr channel during subsampling. I can't really predict the errors that the DCT introduces without actually decoding the video. This means in order to correct overflow situations I would have to effectively do two passes of the encoding. Another solution would be to try to introduce some head and foot room into the calculations by offsetting the luma channel. This would impact the image details in bright and dark spots though and certainly wouldn't fix overflows caused by the subsampling.
So does anyone here know a better solution than two encoding passes?
Last edited:
TrekkiesUnite118
Established Member
So if I'm following this correctly, the issue is that when you get close to extremes the errors have a chance to go past 255 which causes the errors correct? So in the case of the values you posted 260 is instead being interpreted as a 5 which creates the yellow correct?
Could you possibly just avoid the issue by altering the colors of the videos you're encoding? Like make it so white isn't actually white and blacks aren't actually black to create enough buffer to avoid the overflows?
Could you possibly just avoid the issue by altering the colors of the videos you're encoding? Like make it so white isn't actually white and blacks aren't actually black to create enough buffer to avoid the overflows?
Hi, first of all, the work you all have been doing is simple amazing! Thank you!
I'm not a pro, but if I understood correctly, for this particular problem, multiple passes don't seem so bad at all. It will give correct results, without degradation. And the computational cost isn't a great problem, because encoding properly will be a one time job.
BTW, it's amazing how fast you got this codec working!
I just registered to say thanks for your time and efforts Trekkies and now nanash! Even though there's now an HD Collection, I'm holding out for the FMV subtitles so I can play the definitive edition of Grandia on my Saturn. Thanks again!!
TrekkiesUnite118
Established Member
1) Please post this in the Lunar thread.
2) If you can provide a save file we can more easily test it.
3) ODEs are known to have issues right now because they don't accurately emulate how a real Saturn with a CD Drive behaves. To deal with this we made a new patch that cut down the FMV quality and stripped out some files. Are you using the newer patch or the older patch?
nanash1
Established Member
Yes, that's the problem. I think altering the colors is the right approach. The article on Wikipedia about the YCbCr colorspace talks about scaling the values to gain head and toe room, so I tried that and it seems to be working for the most part. I also noticed that some areas have greater encoding errors than I would expect, so I need to look a bit further into the different DCT algorithms of the game to see if can get a better accuracy. I believe now that the 2 pass encoding is actually not really possible in the way I intended it for technical reasons. I also took a closer look at the existing Intro FMV and you can actually see color errors at hard edges all over the place. So I believe it's unavoidable to have some errors with hard edges. This is hardly ideal for text, but maybe the issue can be minimized with the right font choice.
TrekkiesUnite118
Established Member
Is what's on your github up to date? If so I can check it out and try playing with it myself if you can give me some info on how to use it.
nanash1
Established Member
The repository is up to date, but at the moment it's basically a collection of functions without an easy convert command. Imo it would be easier to program that convert function with options than to explain how to use the current state.
