SegaXtreme Cross-Platform FMV Competition

[matteusbeus]

Mappers are fine. That said the larger ROM hack I don't think will work for 32X. That only works on a Genesis with no add-ons because it's using the Memory region reserved for the Sega CD and 32X.

I'm patiently awaiting a certain persons entry to see if he's cracked it! That's if he submits 😀

Oh and regarding the mapper the SGDK mapper supports up to 3GB+. There are some limitations at present though firstly PCM can only go in the first 32MEG (4MB) of ROM as there isn't bank switching support for it. Secondly the SGDK currently has a bit of an issue with compression where if an image crosses a 512KB bank the compression screws up. This is partially why my 12.5 FPS ROMS aren't being submitted here. Covering all that in a video shortly but my main objective is to cover this competition first 😉

 

[Rasky]

Hello, I'm submitting a first round of Nintendo 64 entries.

I've been working for a couple of years on a H264 (MPEG-4 AVC, Baseline Profile) decoder for N64. It is really pushing the limits of the console if you think that N64 was released in 1996, while H264 was finalized on paper in 2003 but became commonplace around 2010 or so. The decoder is fully written in software, it runs on an unmodified, unexpanded console, and uses both the main 93Mhz 64-bit CPU and the 62Mhz RSP coprocessor with SIMD vector instructions. The RSP code is ~5000 lines of assembly code that has been optimized over the years to squeeze every cycle. My goal has always been low bitrate movies, because N64 cartridges are constrained to a maximum of 64Mb on cartridges, so most home-brew applications will try to squeeze videos in very little bitrate.

Typical performance I can achieve is 320x240, 24FPS @ 400 Kbit/s, or 20FPS @ 600 Kbps. Those bitrates sound low but H264 can do wonders (though not miracles). In general, N64 is strongly limited by memory bandwidth, so reducing framerate and resolution helps a lot. For instance, I decided to encode 4:3 videos at 288x208 which allows me to push bitrate up; on my CRT, I think the balance is better, given that H264 can really use those bits.

The player isn't really the best fit for this competition, where there is no bitrate cap. Anyway, this is what I managed to achieve with the samples of this competition. For audio, I went with uncompressed audio because there was room available and I wanted to give every drop of available resources to H264. I have a MPEG1 Layer II (aka "MP2") decoder that kinds of work but it doesn't sound too well (precision issues with fixed-point IMDCT), so I avoided using it here.

LIVE​

Video: 288x208, 24FPS, 550Kbps (22 Mb)
Audio: 44100Hz, Stereo, uncompressed
No big problems here. The bitrate is a bit lower than others because I felt like it really needed at least 24 FPS. It's not super beautiful but not bad.

ANIM​

Video: 320x176, 12FPS, 1000Kbps (11 Mb)
Audio: 44100Hz, Stereo, uncompressed
Given this is hand-drawn animation, the actual framerate is very low, so I lowered the framerate and bumped bitrate. To me, this looks very good (also the damning moving star scene). It's also at the x264 theoretical encoding perfection bitrate-wise (at this resolution/framerate), because it's using the highest quality setting, unconstrained.

CGI​

Video: 320x176, 20FPS, 700Kbps (14 Mb)
Audio: 44100Hz, Stereo, uncompressed
This was the hardest for some reason. In the end I went with 20 FPS to bump the bitrate to a level where I feel it looks good.

BADAPPLE
Video: 320x240, 24FPS, 400Kbps (11 Mb)
Audio: 44100Hz, Stereo, uncompressed
Looks perfect to me. My player has some bugs at 30FPS which I have not investigated, otherwise it would probably work.

You can watch these ROMs using the Ares emulator, which is the only one accurate enough for this H264 player. Ares has some emulation timing inaccuracies, so even if Ares itself runs full speed on your computer (60 FPS in the bottom right corner), the videos might be sluggish at times. On a real console (using a development cartridge like EverDrive64) it looks very smooth. I will try to post recordings.

ROMS DOWNLOAD:

Dropbox

www.dropbox.com

 

[ROS]

...

 

[Rasky]

Second submission for Nintendo 64.

Krom programmed a lossless version of Bad Apple. Frames of the video have been extracted as a sequence of pictures, converted to 8-bit greyscale, and individually compressed using a lossless LZ derivative. At runtime, Nintendo 64 decompresses the frames using the CPU. In this case, a higher quality video source was used, where this technique really pays off in terms of picture quality (using the same techniques over a DCT-compressed video does still work, but the output still shows the typical video artifacts). If you play this ROM, you can notice that it seems drawn in real time rather than being a video.

Added in the "lossless" folder on Dropbox.

Dropbox

www.dropbox.com

 

[TrekkiesUnite118]

DEADLINE UPDATE:
After discussing it with the participants it's been decided to extend the deadline by 1 week. This should give people enough time to put some finishing touches on submissions.

The new Deadline is February 7th, 2022.

 

[Chilly Willy]

Huh, I was so caught up in my CD32X player that I entirely forgot I already made a video player for the N64. There's just onnnnnnnnnne little problem - it requires an N64 NeoMyth cart. I modified my Simple Media Player to add RoQ video to the list of various music formats it plays. You launch SMP, navigate to the video folder on the SD card, select the RoQ encoded video, and it plays. Not sure how many people here have the N64 NeoMyth cart. Maybe I could make another build that handles the N64 Everdrive.

With the added time, I'll encode some RoQ versions of the contest samples and see how they turn out. I've got a BUNCH of videos on my SD card, like Bad Apple (the B&W and color 3D versions), several movie trailers like Revenge of the Sith, Cowboy Bebop Ep 1, some music videos, Kung Fury, several episodes of Hellsing Ultimate Abridged, some Dragonball Z Abridged, some Naruto Abridged Spoof episodes, Troops... you get the idea. Been sitting on that for YEARS. Hmm - last build, 2014. GAHH!!

 

[TrekkiesUnite118]

Does it require additional hardware in the NeoMyth cart or something that prevents it from running on other carts?

 

[Chilly Willy]

It uses the SD card port to access the SD card. No other hardware is needed. So I should be able to make a version that works on the ED64. I just hadn't gotten around to it before moving onto another console. I imagine more people have the ED64, so I really ought to make a version of SMP that targets it. It's a nice music player if nothing else. Supports mp3, m4a, midi, ogg-vorbis, and flac. When you start it playing a file, when the file ends, it will continue with the next file in the folder so as to facilitate playing albums. This also works with roq video files - when the video ends, it will continue with the next in the folder.

It targeted the NeoMyth because at the time, I made SMP for the NeoFlash competition (it won its category that year). It will play files embedded in the rom with the app, or off the SD card if there is one. Hmm - I should be able to make rom-only files as long as the video isn't too long. That would allow people with just emulators to run it (I run my N64 stuff in mess when I want to emulate it instead of real hardware - haven't tried ARES yet). I'll have to give that a try as well. The video player supports any framerate and resolutions from 256x240 up to 640x480. Need to do more testing with that - I mainly have tried 320xWhatever so far.

 

[matteusbeus]

I plan to do a feature video on my channel regarding the competition. Links to video clips of your examples would be very much welcome as I don't have hardware mods to enable me to capture Saturn, N64, etc

 

[viciious]

Hi everyone,

here are my submissions for the Sega 32X. All samples were encoded using the RoQ codec and use the Sega Mapper (aka SSF Mapper) to break the 4MB limit for ROMs. All entries run fine on my Mega EverDrive Pro on real hardware. RetroArch seems to run most of the samples fine except for the CGI one for whatever reason. The samples may also work on the MegaSD with the latest firmware, which has a bunch of fixes for the SSF on the 32X.

ANIM​

Video: 256x192, 12FPS
Audio: 22.05KHz Mono

Dropbox

www.dropbox.com

BAD APPLE​

Video: 192x144, 20FPS
Audio: 22.05KHz Mono

Dropbox - File Deleted - Simplify your life

www.dropbox.com

CGI​

Video: 192x144, 12FPS
Audio: 22.05KHz Mono

Dropbox

www.dropbox.com

​

LIVE​

Video: 192x144, 20FPS
Audio: 22.05KHz Mono

Dropbox - File Deleted - Simplify your life

www.dropbox.com

Dropbox - File Deleted - Simplify your life

www.dropbox.com

 

[viciious]

DamageX, I'd love to learn about the codec and settings that you used for your samples!

 

[DamageX]

My encodes use the audio compression that I posted about in another thread. To add a bit more detail on how that works, I take my 63 AC coefficients after the DCT and knock them down to 10 bits. Then I divide them further until the largest number fits in the range -31 to +31. The divider gets stored in the bottom 4 bits of the 16-bit word holding the DC coefficient (so the DC becomes 12 bits). Then the ACs get Huffman encoded. Every 1024 chunks (65536 samples) gets a new Huffman table.

The video codec is a newly made one. It doesn't use any color space conversions or entropy coding, instead storing everything in aligned 16-bit words for speed. A frame is divided into 16x16-pixel macroblocks, each containing 16x 4x4 microblocks in a back-and-forth sequence (ie. top row is left-to-right, next row is right-to-left, etc.). A macroblock begins with a word which tells which microblocks will update; if the corresponding bit is clear then it remains the same as last frame. For microblocks that do update, they can be encoded as 1 color, 2 colors, 4 colors, "raw", or duplicated from another location within the macroblock. The codec maintains a 'palette' of two colors which can be reused for 1/2/4-color blocks so that new colors don't always need to be specified.

The encoder calculates an error factor for each block encoding and adds it to the data size, weighted by a quality factor, and then chooses the encoding with least error+size. When I split the videos into several pieces of equal length, some parts easily compressed to less than 4MB while others needed a lower quality setting to fit (eg. Trek #1)

 

[viciious]

My encodes use the audio compression that I posted about in another thread. To add a bit more detail on how that works, I take my 63 AC coefficients after the DCT and knock them down to 10 bits. Then I divide them further until the largest number fits in the range -31 to +31. The divider gets stored in the bottom 4 bits of the 16-bit word holding the DC coefficient (so the DC becomes 12 bits). Then the ACs get Huffman encoded. Every 1024 chunks (65536 samples) gets a new Huffman table.

The video codec is a newly made one. It doesn't use any color space conversions or entropy coding, instead storing everything in aligned 16-bit words for speed. A frame is divided into 16x16-pixel macroblocks, each containing 16x 4x4 microblocks in a back-and-forth sequence (ie. top row is left-to-right, next row is right-to-left, etc.). A macroblock begins with a word which tells which microblocks will update; if the corresponding bit is clear then it remains the same as last frame. For microblocks that do update, they can be encoded as 1 color, 2 colors, 4 colors, "raw", or duplicated from another location within the macroblock. The codec maintains a 'palette' of two colors which can be reused for 1/2/4-color blocks so that new colors don't always need to be specified.

The encoder calculates an error factor for each block encoding and adds it to the data size, weighted by a quality factor, and then chooses the encoding with least error+size. When I split the videos into several pieces of equal length, some parts easily compressed to less than 4MB while others needed a lower quality setting to fit (eg. Trek #1)

So you wrote two custom codecs, very nice!

Adding bank switching support should be relatively easy if the frame header also contains the size information, should it would be possible to tell whether the chunk crosses over into the next page.

Also, if you want to optimize it a bit further, delta-encode your blocks from the frame before the previous frame. That way you could read color information from the framebuffer and avoid repainting the "undamaged" blocks.

 

[DamageX]

Adding bank switching support

Is there a 32X emu that supports this? When I tried going over 4MB before Fusion just says "not a 32X ROM"

 

[viciious]

Is there a 32X emu that supports this? When I tried going over 4MB before Fusion just says "not a 32X ROM"

Retroarch (the picodrive core) seems to work alright. Fusion has a 6MB limit for ROMs.

 

[gameblabla]

These are really bad but i did it anyway for the lulz.
This is what happens when you spend little effort on it. 😛

I sort of made the YUV to RGB code a bit faster but not much other than that.

These are for the N64 and it uses the RoQ format.
It's basically a port of DreamRoQ with libdragon.

Animation

512x224
12 FPS
22050hz Mono



CGI

256x160
10 FPS
22050hz Mono



Live action

256x224
10 FPS
22050hz Mono



Bad Apple

640x224
15 FPS
22050Hz mono

 

[trapexit]

I'll have something posted shortly. Just found a bug in my encoding workflow that I'm trying to work around.

 

[trapexit]

File folder on MEGA

mega.nz

Ran into some issue with the menu (a stock 3DO SDK app) where adding 4 videos would cause it to break. So I split it to 2 images.

• Anim: video = 320x180 Cinepak @ 25FPS, audio = 3DO SDX stereo @ 22Khz
• CGI: video = 320x180 Cinepak @ 30FPS, 3DO SDX stereo @ 22Khz
• Bad Apple: 320x240 video = Cinepak @ 30FPS, audio = 3DO SDX stereo @ 22Khz
• Live: video = 320x240 Cinepak @ 30FPS, audio = 3DO SDX mono @ 22Khz (used mono to squeeze a bit more out of the video. hard to tell if it really made any difference.)

There is some audio desync. This is a bug in the 3DO muxer and maybe player. Has been there since the beginning. 3DO's "Video" releases were synced fine so someone fixed it but not yet clear what was changed. There is also some tearing. That's also pretty common with 3DO video decoder. I've not yet dug into why it happens. If it's a lack of vsync, Cinepak artifacting, or some sort of data rate management.

I'd have liked to use the EZFlix codec but I've not yet figured out if the encoder we have works with Opera hardware as it comes from the M2 SDK and generates files far too large. As there were no releases to the SDK after P2.5/TK1.5 it's not clear where/when those tools were provided to devs.

 

[Rasky]

My second N64 submission.

The H264 player was designed for low bitrate movies, because of N64 cartridge constraints. So when I heard of this competition and its rules (where there is no bitrate cap), I decided to attempt something different: a MPEG1 player. The general idea was that maybe H264 had the wrong balance, and MPEG1 could allow for higher bitrate thanks to a much simpler decoder algorithm, and maybe we could end up with higher general quality.

So in the last weeks I ported pl_mpeg to CPU+RSP+RDP. This time it contains about ~1600 lines of assembly RSP code for the actual MPEG algorithms (prediction + AC dequantization + IDCT), plus another ~300 lines for YUV interleaving (the actual YUV conversion is performed by RDP). The RSP code uses four special opcodes that were designed by SGI specifically for MPEG1 decoding (VRNDN/VRNDP, VMULQ/VMACQ), which basically help with the AC dequantization / oddification. I had to partially reverse their workings because the documentation is scarce. To the best of my understanding, these are the first ROMs in N64 history to use these opcodes. Notice that curiously these opcodes are so tailored to MPEG1 that I didn't find any use of them in my H264 player.

It took a while to stabilize the player (and there's a bug left, visible in the animation sample), so I didn't have much time to optimize it properly, nor to learn the best settings for the encoders. In general, the player is now able to play at 320x240 @ 24 FPS @ 1500/2000 Kbit/s, though sometimes it slows down a bit on complex scenes. The current results are worse than the H264 samples, I think. In the future, I will see if optimizing the player more and increasing the bitrate, I can beat it.

Source code for the codec is available here:

libdragon/src/video at video · rasky/libdragon

Open source library for N64 development. Contribute to rasky/libdragon development by creating an account on GitHub.

github.com

not ready for prime time, but I plan to integrate it in libdragon once it will be finalized.

I encoded the samples at 320x240 or 320x176, 24FPS, 1500 Kbps. Audio is uncompressed (16 bit, stereo, 44Khz). The samples can be found in the "mpeg1" folder. They can be played with the Ares emulator. I didn't send the BadApple one because the bug visibile in the anim sample is basically everywhere.

Dropbox

www.dropbox.com

 

[jekuthiel]

NES team here. (Something Nerdy Studios)

I'm just trying to get our first post in before midnight. We have all 4 ROMS created but we're struggling to record the last video and upload everything.

Bad Apple:


(proof that it works on real NES hardware)


(pixel perfect)

Next few posts will contain links to the other 3 videos.

-jekuthiel