SegaXtreme Cross-Platform FMV Competition

[jekuthiel]

We had negotiated the privilege of submitting our videos in 2 forms for this competition: pixel perfect and live recording showing them playing back on a real NES.

Here is the link to both versions for all 4 videos:

http://www.hilbertspace.net/stuff/fmv_competition/

I uploaded them all to my personal web space so I wouldn't be using SNS company property to house copyright infringing material. (I don't think this actually matters, but it made me feel a little more comfortable.)

All 4 videos are either native 30FPS or chopped to 30FPS, and then played back at 30.05FPS due to the fact that the NES's video output rate is 60.1Hz. Similarly, all of the audio has been converted first to 44.1KHz 8-bit mono PCM and then further downgraded to 44.1KHz 7-bit mono PCM, played back at 44.2KHz. The Bad Apple and TNG clips are both full screen -- 256x240. The anime and FFXIII intro clips are both 256x184 in order to match the aspect ratio of the originals. I figured no one wanted to deal with anamorphic video for this.

-jekuthiel

 

[TrekkiesUnite118]

I think at this point we're just waiting on ChillyWilly to submit a CD32X entry. I figure I'll give him a few more hours to at least post something.

 

[Chilly Willy]

I think at this point we're just waiting on ChillyWilly to submit a CD32X entry. I figure I'll give him a few more hours to at least post something.

Not going to make this year's compo. I'll work toward next year's. 🙂

I've still got work to do on the player, and I've also been rebuilding my N64 toolchain to do some work there.

 

[Rasky]

@Chilly Willy libdragon has built-in supports for SD cards on EverDriver64 and 64drive now. You might want to look into that

libdragon: debug.h File Reference

 

[Chilly Willy]

@Chilly Willy libdragon has built-in supports for SD cards on EverDriver64 and 64drive now. You might want to look into that

libdragon: debug.h File Reference

I noticed that... it's for the newer models. I've got the older v1 ED64, which isn't supported. I'll probably add that to it - start with the current ED SD code, and add in v1 support. I also plan to add Neo Myth to it. The ED64 will be better for video since it's faster. The NeoMyth is 100% cpu driven, and gets about the same speed on the N64 as a 2X CDROM. When I play RoQ files on the N64 with the NeoMyth, I need to keep it to similar specs for 2X CDROM: 320x176 (for 16:9) at 12 or 15 FPS, 1950kbps on the RoQ video (I made my own custom version of ffmpeg that implements encoding at a set bitrate), and stereo RoQ DPCM (22050 Hz). I encode my RoQ videos intended for the CD32X at 975 kbps as it's a 1X CD.

I like how much use the RSP is getting these days, especially the RSP audio mixer. Very nice, and barely uses any time. I'll have to work the new audio code into my media player. I should also move more of the RoQ video frame decoding to the RSP.

 

[TrekkiesUnite118]

Not going to make this year's compo. I'll work toward next year's. 🙂

I've still got work to do on the player, and I've also been rebuilding my N64 toolchain to do some work there.

Then in that case I'll deem the time for submissions as closed. I've sent out the the scoring sheets to the judges so I'll keep this thread updated as results come in.

 

[Ardiloso]

NES team here. (Something Nerdy Studios)

I'm just trying to get our first post in before midnight. We have all 4 ROMS created but we're struggling to record the last video and upload everything.

Bad Apple:


(proof that it works on real NES hardware)


(pixel perfect)

Next few posts will contain links to the other 3 videos.

-jekuthiel

My brain is incapable of processing this. I'm floored. Amazing.

 

[nando]

We had negotiated the privilege of submitting our videos in 2 forms for this competition: pixel perfect and live recording showing them playing back on a real NES.

Here is the link to both versions for all 4 videos:

http://www.hilbertspace.net/stuff/fmv_competition/

I uploaded them all to my personal web space so I wouldn't be using SNS company property to house copyright infringing material. (I don't think this actually matters, but it made me feel a little more comfortable.)

All 4 videos are either native 30FPS or chopped to 30FPS, and then played back at 30.05FPS due to the fact that the NES's video output rate is 60.1Hz. Similarly, all of the audio has been converted first to 44.1KHz 8-bit mono PCM and then further downgraded to 44.1KHz 7-bit mono PCM, played back at 44.2KHz. The Bad Apple and TNG clips are both full screen -- 256x240. The anime and FFXIII intro clips are both 256x184 in order to match the aspect ratio of the originals. I figured no one wanted to deal with anamorphic video for this.

-jekuthiel

Seriously, well done! That Bad Apple demo reminds me of Red Zone on the Genesis, only it has full PCM audio and runs at a higher frame rate. The other demos are even more impressive - obviously the color palate of the NES is limited, but it's super smooth and I'd have to think on a CRT through composite, it would look pretty good. Also, at first I thought you were playing the audio through an external device.

 

[Rasky]

I noticed that... it's for the newer models. I've got the older v1 ED64, which isn't supported. I'll probably add that to it - start with the current ED SD code, and add in v1 support. I also plan to add Neo Myth to it.

That would be so great. I've never seen a Neo Myth myself; if it has a USB channel, it would be useful to also add debugf() support to it and in general hook it up to the debugging library.

I like how much use the RSP is getting these days, especially the RSP audio mixer. Very nice, and barely uses any time. I'll have to work the new audio code into my media player.

The main issue is that the mixer library is mostly thought for a pull/async design (callback when new data must be produced), while a video player is more easily done with a push design (decode a frame of the file, it contains audio, push the audio into the mixer). This also affects the way audio/video syncing must be done. I have a PR open in draft to add a mixer_throttle API to help with audio/video sync, but I would like to first see if there's way to better support the push approach for video players. I'm happy to discuss API ideas around this.

I should also move more of the RoQ video frame decoding to the RSP.

Yeah and maybe even RDP which can be a fast parallel bitterer engine to do motion compensation (I haven't use it that way in MPEG1 but I should).

In general if you want to discuss about video codecs on N64, you can find me on Discord (Rasky#6670) and/or in the N64Brew server. I'm looking forward to seeing your progresses!

 

[Chilly Willy]

That would be so great. I've never seen a Neo Myth myself; if it has a USB channel, it would be useful to also add debugf() support to it and in general hook it up to the debugging library.



The main issue is that the mixer library is mostly thought for a pull/async design (callback when new data must be produced), while a video player is more easily done with a push design (decode a frame of the file, it contains audio, push the audio into the mixer). This also affects the way audio/video syncing must be done. I have a PR open in draft to add a mixer_throttle API to help with audio/video sync, but I would like to first see if there's way to better support the push approach for video players. I'm happy to discuss API ideas around this.


Yeah and maybe even RDP which can be a fast parallel bitterer engine to do motion compensation (I haven't use it that way in MPEG1 but I should).

In general if you want to discuss about video codecs on N64, you can find me on Discord (Rasky#6670) and/or in the N64Brew server. I'm looking forward to seeing your progresses!

The N64 Neo Myth has a USB port, but I don't think I've ever used it. I'd really have to go back to my old notes to see about support for that.

I designed my N64 RoQ player around the callback async audio pull system. It made timing easier. I use the audio for timing. This helps since no console ever plays audio at the exact right rate. So I play back the audio at the console rate and use the samples-per-frame for my video timing. This keeps the audio and video in sync, but could occasionally result in a slight stutter in the video. It's tough to see in reality, and much better than trying to sync the audio to the video instead. The audio will be a fraction of a percent off, but you can't hear it unless you've got the most Golden Ears ever known to man. The video will stay in sync since it's looking at the time codes from the audio callback, which pulls a frame's worth of audio from the buffer, then updates the time code for the video. The video frame will also be a fraction of a percent off, but that also tends to be unnoticeable. So, both are off a fraction of a percent, but it means a hell of a lot less work to keep the sync perfect. 🙂

I originally looked at using the RCP block DMA for moving the video data around, but it has some alignment issues that kept it for working on all block sizes, so I wound up disabling it. If I could use RDP operations instead, that would probably be faster/better. We need more RDP hardware support in libdragon... like textured triangles. 😀

 

[ROS]

...

 

[Chilly Willy]

CHEATER

I have no idea what you mean... really! 😀

Remember, these are really low-power devices. You want to program smarter, not harder. If one way works well enough and reduces the programming tremendously, you do it! This method really should have been available on PC player programs as well, years back. I remember messing around with mplayer, trying to use their various methods to sync the audio and video on an older P3 system that wasn't nearly as effective as what I was doing on the N64. They were simply trying to brute force methods to sync both the audio and video to "perfect" timing, and many older systems just couldn't hack it, even with a few hundred MHz/a couple GHz. Modern players now count on video cards to use near infinite power (compared to the problem) to brute force the issue away. This is one reason I like programming on old consoles.

 

[TrekkiesUnite118]

Looks like @ROS deleted his Bad Apple entry. I've uploaded here for judges to download:

BADAPPLE SS(U)

MediaFire is a simple to use free service that lets you put all your photos, documents, music, and video in a single place so you can access them anywhere and share them everywhere.

www.mediafire.com

 

[TrekkiesUnite118]

I've started putting together a playlist on Youtube of the different entries minus the Live Action samples since that gets the video blocked. As I get more captures recorded and uploaded I'll add them to the playlist:

 

[TrekkiesUnite118]

Sorry this took so long, but I finally have most of the judge results in to give the top 3 for the best overall quality categories. There were a few ties so I couldn't just call it, so Emerald Nova stepped in to sub for FilmCow. There still is one tie for third place, but hopefully we can get that sorted out.

For Best Overall Quality in Animation:

In First is Rasky with his H.264 Entry for the Nintendo 64.
Second place is TrekkiesUnite118 with the Cinepak entry for Sega Saturn
Third place is a tie between the Saturn Truemotion by Trekkies and the N64 RoQ entry by gameblabla

For Best Overall Quality in CGI:

In First is again Rasky with his H.264 Entry for the Nintendo 64.
In Second place is again Trekkies with the Cinepak entry for Sega Saturn.
In third place is Rasky with his MPEG-1 entry for the Nintendo 64.

For Best Overall in Live Action:

In First place is TrekkiesUnite118 with his Saturn Cinepak entry.
In Second is Rasky with his H.264 entry for the Nintendo 64.
In Third place is Trekkies with his Truemotion entry for Sega Saturn.

For Best Overall Quality for Bad Apple:

In First place is TrekkiesUnite118 with his Saturn Cinepak entry.
In Second Place is TrekkiesUnite118 with his Saturn Truemotion entry.
In third place is Rasky/Krom with their lossless Nintendo 64 entry.

There are still some spots where EmeraldNova has blanks that he may fill in later (this is why NES Bad Apple isn't 3rd even though it's currently 1 point higher), but I think the top three for these categories should be decided now. Overall the general consensus was as follows:

For the pre-5th gen consoles the NES and Genesis entries most judges found to be very impressive for how they balanced ROM size, as well as how they took advantage of the way the hardware inherently works. The NES entries in particular most judges were very impressed with.

For 5th Gen consoles, the N64 entries really blew the socks off everyone. No one expected the N64 to be pulling off what we saw in this competition. The next most impressive was Saturn Cinepak as most judges didn't expect it to hold up as well as it did.

Honorable mentions were both the 3DO and 32X entries. Judges were not expecting to have as many 32X entries as we had, nor to see them all try very different approaches. For 3DO judges were impressed by how high the framerates were on them, even if they may not have had the best overall quality.

I'll try to get the other categories sorted out later.

 

[Chilly Willy]

The NES video was VERY fluid as it runs at 60Hz. The biggest problem is the fixed color palette. It's very hit or miss, and when it's a miss, the video is a horrible, pulsating mess of pixels! But still very fluid. 🙂

I was certainly blown away by the N64 videos. It has some damn fine video quality - I imagine if the DD had caught on, there might have been some fine videos on games on the DD. As to the Saturn, the cinepak encoder blew me away. Cinepak quality can vary drastically, from nightmarishly bad, to excellent. Coaxing excellent from the encoder is a good trick that's hard to do.

 

[Matilija]

I'm genuinely impressed with your cinepak results Trekkies, I swear some of those look better than official VCD's that I have.....I didn't think that was possible.