FPS-demo + source code

[XL2]

Hi all!

One of the hardest thing in making homebrew 3D games is the collision detection. So I did a quick FPS-style demo (with source code, done in a few hours only so don't expect Half-Life on the Saturn) with working collision detection (still buggy) and a tool to generate the bounding boxes and slopes (buggy as well, but works nicely for axis-aligned polygons). Not directly related to my game, Sonic Z-Treme, but I will continue to update the tool I created for the bounding boxes and hopefully it will help someone with his project. I'm using Jo Engine for this, but with some minor modifications it could work with SGL. I didn't use Rockin'B's tool / code, but liked the idea of using the MDL/H files directly to output the collision data, so I took inspiration from his work in a way.

There are currently several limitations and the tool isn't complete (I'm in the middle of adding features such as writing to binary), but I guess maybe that could be interresting for some of you here since I included the source code. It works better for a game like Sonic X-Treme/Z-Treme, corridor shooters like Doom/Quake or games like Resident Evil. It wouldn't work too well for a game like Mario 64 since it won't play nice with things like hills and rocks with multiple faces. It works better with axis-aligned quads (or slope on one axis). You could also just use a complex heigth mask for hills and rocks, but right now for memory constraints I use small arrays (more on that later). So in other words, it could be modified for more complex geometry, but my hands are full right now so I don't think I will do it.

Hopefully it will help someone start his/her own 3D game project on the Saturn.

Video of the short demo (I say again, I quickly coded it, so it's nothing impressive!) :



For the tool, first I want to apologize, the code is full of commented code that I didn't fully remove since I might use some in the future, and it's not user friendly. It shouldn't be that hard to use, but it requires a few things.

First, make sure you link it up with the SGL library (SGL.h) (Edit : I removed the need to make the tool point to SGL.h, but didn't test it properly so if you have issues make sure you tell me) as I'm using that to read the polygons. Then include your level's .h file (just convert it with the Jo Engine map editor, without the textures) (EDIT : The tool now accepts .OBJ files directly. I haven't done much testing, but the maps in the FPS demo are using the resulting files and seems to be OK).

If you use a level's h file, call the JO display command (it won't display anything, I just use that to initialize the polygon list - it avoids having to modify the .h file and remove one step for you).
If you are using OBJ file, just put the correct filename in the main function.

The tool needs to be compiled. It outputs a file, I just named it TESTPOLY2BB.txt, with examples of the struct I use. Look at the FPS demo, in MESHES/LVL_BB.h and Collision.c to see how it works. I use Code::Blocks to compile, but it should work with anything.

The tool works by reading the polygons, creating bounding boxes and then merging the boxes with neighbours to reduce the amount the Saturn needs to go throught (it could be reduced further than now, on my to do list). It also reads the normals to determine if it's a slope, then calculates the slope (still buggy, I will have to improve it). It also adds "depth" to the polygon to avoid going throught it if you go too fast.
I will have to improve my slope generation code as it's hit and miss but I didn't put much time in it yet. It also doesn't generate heigth masks yet, but I've included examples in the FPS demo on how to use masks, so it shouldn't be too hard.

For the FPS demo and code, just put it in your Jo Engine Src folder next to the demos, from there you should be able to open the code and modify it and compile it. And once more : it's quickly coded, so some things are probably wrong/stupid, if I update it I will share with you guys.

Again, sorry for the not-so-user-friendly-hackjob of a code!

EDIT : Updated the collision tool (I will rename it soon enough) to convert .OBJ files to binary file format. The FPS demo now accepts the binary files (.ZTM) to load the maps. Press L/R + START to switch maps. Note that the slopes in map 2 are wrong and the collision tool is still having issues with them (I haven't tried to fix this yet). I also disabled the UseNearClip as it doesn't play nicely with FPS unlike the Sonic game. I will try to find a fix for that. I also added different textures in map 2, including cheap texture animation (temporary solution, it works, but it's not great, I will add something inside the map tool to generate it or change the function to be more flexible).

EDIT (2017-08-10) : Did a quick update to make everything smoother. I fixed the look up/down (now it's smooth and looks OK) and also fixed the camera movement when you are "under" a tile (such as when you climb stairs on Map 2) to make it smoother. Also added gouraud shading on the whole levels.

EDIT (2017-08-14) : Many bugs fixed. Obj to binary now works without known issues. I also added basic view frustum culling in the FPS demo, auto-crouch (buggy, you can try it in the "air duct" on map 2, near the top), a NBG1 HUD (ugly and done in 5 minutes - if you want back the NBG2 and RBG0, just uncomment the lines in Display.c). I tried it on real hardware and had to reduce the draw distance to get decent framerate (around 60, but lots of slowdowns on map 2). Removing the "flame" background and RBG0 helps to hide the short draw distance since everything just fades to black. There seem to be an issue since putting the game at 30 FPS has the same results as 60 FPS and no advanges. Also, if you try map 3, it crashes on real hardware and in Yabause, but works in SSF. I still haven't implemented proper BSP for the collision code, so that might be part of the issue, but I know the current memory allocation while loading is bad as there are many gaps between the meshes.

EDIT (2017-08-16) : Upgraded to Jo Engine 9.0 (make sure you upgrade also). I also fixed the memory allocation, so you should be able to load much bigger files now (try map 3 to see).

EDIT (2017-08-25) : You can now write textures directly in the binary file. The map program will read TGA textures (uncompressed only) and write them. Also added auto-subdivision of the map for culling, you just need to specify how many "cuts" you want (there is a bug that will sometimes "lose" polygons, so just try different values for "cuts" (x, y, z axis) if one setting doesn't work well). I also changed the FPS demo's collision code to reduce CPU usage.

 

[mrkotfw]

Excellent work.

I'll take a closer look. I especially like the level you made!

I'm particularly bad at making levels in Blender. This one is very good!

 

[XL2]

Excellent work.

I'll take a closer look. I especially like the level you made!

I'm particularly bad at making levels in Blender. This one is very good!

Thanks, but honnestly, it's nothing impressive, just a quick level to show how my BB generator works and how to use slopes.
I added a weapon, but I recommand not using 3D models for weapons as it just creates clipping issues. I had to set the level to SORT_MAX and weapon to SORT_MIN, but it's still not good enough. SORT_CEN for a FPS level is better overall. The alternative is to set everything to Single_Plane (I highly recommand doing that), but this particular level doesn't suit Single_Plane since the wall are just one layer thick and you can see accross the map.

 

[Red Ring Rico]

This is great work. I know I haven't been active here as of late (work and other things getting in the way =/), but I do come here at least a few times a week to check in. Are you going to go further with this or is it just to be a starting point for others to use?

 

[XL2]

This is great work. I know I haven't been active here as of late (work and other things getting in the way =/), but I do come here at least a few times a week to check in. Are you going to go further with this or is it just to be a starting point for others to use?

It's really for others as I'm busy with my own game, Sonic Z-Treme. Since I am writing tools, I just thought why not share it? And if I share it, I need some kind of demo to show how it works, so I just quickly put together this FPS demo. I will add VDP2 exemples (3D plane), proper map loading from ROM/CD, maybe a VDP2 weapon and maybe ennemies (no AI) and damage system.
I doubt someone will truly make a game out of this, so I'll try to make it a check-in-the-box hoping it will help the homebrew community grow a little. We'll see!

 

[mrkotfw]

Thanks, but honnestly, it's nothing impressive, just a quick level to show how my BB generator works and how to use slopes.
I added a weapon, but I recommand not using 3D models for weapons as it just creates clipping issues. I had to set the level to SORT_MAX and weapon to SORT_MIN, but it's still not good enough. SORT_CEN for a FPS level is better overall. The alternative is to set everything to Single_Plane (I highly recommand doing that), but this particular level doesn't suit Single_Plane since the wall are just one layer thick and you can see accross the map.

I would love to port this to libyaul. There are a lot of issues with the graphics pipeline that I wrote (no clipping), but it should work...

How exactly does SGL's sorting work? It's rather confusing to me to have to hint to SGL using SORT_MIN, SORT_CEN, SORT_MAX.

Shouldn't it simply sort by average of (z1+z2+z3+z4)/4, from back to front?

 

[XL2]

I would love to port this to libyaul. There are a lot of issues with the graphics pipeline that I wrote (no clipping), but it should work...

How exactly does SGL's sorting work? It's rather confusing to me to have to hint to SGL using SORT_MIN, SORT_CEN, SORT_MAX.

Shouldn't it simply sort by average of (z1+z2+z3+z4)/4, from back to front?

That would be SORT_CEN. The problem is that the Saturn doesn't consider Y/X 3D coordinates for that sorting, only the onscreen values (so the X/Y values for what to actually draw).
The z-sort is quite problematic for that.
SORT_MAX will just take the max Z value as reference point, while MIN takes the minimum value. There is a forth sort type, SORT_BFR, which puts the same sorting prority to the polygon as the polygon wrote before.
You can see how bad it is with everything set to SORT_CEN in a previous version of Sonic Z-Treme. Notice how the ground keeps hiding Sonic's feet :

 

[mrkotfw]

That makes a lot more sense.

So is changing from SORT_CEN to other sorting hints determined manually? If you see the case above, would you play with the sort hint until it looks okay?

I think the PlayStation doesn't have this issue. It has that whole OT thing going, but from what I know, it sorts by SORT_CEN, and if there is contention with the same Z-value, it behaves like SORT_BFR.

 

[XL2]

That makes a lot more sense.

So is changing from SORT_CEN to other sorting hints determined manually? If you see the case above, would you play with the sort hint until it looks okay?

I think the PlayStation doesn't have this issue. It has that whole OT thing going, but from what I know, it sorts by SORT_CEN, and if there is contention with the same Z-value, it behaves like SORT_BFR.

The PSX also has this issue, since both consoles deal with it in software. Now, maybe the official SDK are much better on PSX and give better results, but I wouldn't know. The SORT_BFR just takes the same sorting value as the polygon drawn before in the drawing list, no onscreen. The 2 polygons could be 600 pixels apart, it would still take the same value. Sega themselves wrote that it's rarely used.

You can set everything manualy, but the SGL 3DEditor allowed you to do it per polygon without having to guess in c.
From my tests, you should use the following data for a level :
Single_Plane, SORT_MAX ... UseNearClip (so that it doesn't try to draw offscreen polygons). I never played with the Windows_IN/Out options, but I guess it should also be considered in case you want to add splitscreen to the game.
For a caracter, Single_Plane (unless you want to use a dual_plane quad and use it as a sprite, like for hairs, scarf, etc.), SORT_CEN/MIN, UseNearClip (I don't know why anyone would turn it off).

Also, the Saturn's VDP1 struggles with textured polygons (PS1 had the same issue from what I read), so the lower resolution your textures are, the better it is. It clips like crazy with higher quality textures as soon as you hit a certain amount (like 200 maybe for 64x64). The solution is to have lower quality textures - and low poly count - or use mipmapping and only show higher quality textures when the quad is close to the screen. Since the screen resolution is so low, I doubt that using 64x64 textures is worth it anyway. The problem is that emulators have no issue with it! So you will test in emulators, everything is fine, then on real hardware it falls apart. It took me a while to figure out what the problem is.

Here is a video of an earlier version of Z-Treme where you see the (heavy) clipping in action :


There are lots of other options in SGL and I haven't tried several of them yet, so maybe I will find something better.

 

[mrkotfw]

I'll look into the SGL documentation.

For the early video of Sonic Z-Treme, if the Saturn can't keep up with drawing all those polygons by the end of V-BLANK-IN, it would by default swap the frame buffers anyway, and continue onto V-BLANK-OUT.

Did you switch to frame manual erase/change? That would have the frame buffers swap when the VDP1 is actually done (SCU has an interrupt along with a polling register) with drawing. So if it wasn't on 1-cycle mode, I would expect the frame rate to plummet, no?

 

[XL2]

I'll look into the SGL documentation.

For the early video of Sonic Z-Treme, if the Saturn can't keep up with drawing all those polygons by the end of V-BLANK-IN, it would by default swap the frame buffers anyway, and continue onto V-BLANK-OUT.

Did you switch to frame manual erase/change? That would have the frame buffers swap when the VDP1 is actually done (SCU has an interrupt along with a polling register) with drawing. So if it wasn't on 1-cycle mode, I would expect the frame rate to plummet, no?

Honestly, I never touched that and don't know much about it.
So with frame buffer swap, would it be able to draw the whole screen at the cost of framerate?
I never really played with 30 FPS, always left it to 60, but since it's interlaced anyway it might be worth it.

 

[XL2]

I updated the FPS demo to show use of gouraud shading and NBG2 scrolling (both depending on player movement and auto-scrolling). The demo seems to crash on SSF for a few seconds, then resumes (only happens after around 5-6 minutes).
It doesn't happen on Yabause, and since it doesn't completely crash the game, it's hard to know what is causing this.
Can anyone confirm me they have this issue (with the latest version)?
Thanks!

 

[mrkotfw]

You'll be able to draw as much as you want. Effectively, if you tell the VDP1 to start drawing immediately based off the command tables you sent to VRAM, it'll do it until it's done. Once it's done, at the start of the next frame, it'll swap buffers. This is beneficial because you can do work as you're rendering. This also means that you'd have to sync VDP2 with VDP1. That would involve buffering all VDP2 registers and writing back to the registers when the VDP1 has caught up. But I believe SGL takes care of this for you.

I believe you can achieve this with manipulating the synchCount variable in SGL.

I haven't messed with interlacing as much in Yaul. It brings in more complexity, and buys you very little.



I haven't ran in SSF, but there's a good chance that you're hitting a CPU exception. Maybe look in the Yabause logs, or enable more thorough debugging in Yabause to find out. It usually logs unaligned reads/writes.

 

[XL2]

You'll be able to draw as much as you want. Effectively, if you tell the VDP1 to start drawing immediately based off the command tables you sent to VRAM, it'll do it until it's done. Once it's done, at the start of the next frame, it'll swap buffers. This is beneficial because you can do work as you're rendering. This also means that you'd have to sync VDP2 with VDP1. That would involve buffering all VDP2 registers and writing back to the registers when the VDP1 has caught up. But I believe SGL takes care of this for you.

I believe you can achieve this with manipulating the synchCount variable in SGL.

I haven't messed with interlacing as much in Yaul. It brings in more complexity, and buys you very little.



I haven't ran in SSF, but there's a good chance that you're hitting a CPU exception. Maybe look in the Yabause logs, or enable more thorough debugging in Yabause to find out. It usually logs unaligned reads/writes.

Thanks, I will try to take a look at that!
For the FPS demo, I will just try on another PC, it must be something stupid causing problems (like usual!)

 

[XL2]

Hi all!
I wonder if a good soul would be willing to help me with a current issue related to memory management.
With so many of you having worked on emulators, I guess I'm at the right place to ask about RAM.
I'm still struggling with c pointers and memory allocation.

So here it is :
I wrote a couple of functions for the tool to read .Obj, write to a custom binary file format (using either the .Obj file or a file header), and read it on the Saturn.
Right now, it's barebone, but the binary file format goes like this :
-Total number of meshes
(then, for each mesh)
-Number of points
-Number of polygons
-Then I write/read all the points, then the polygons, and then the attributes (only texture no. ATM), and loop until I reached the end (number of meshes).

Now, it works fine in Yabause, but in SSF it doesn't work, everything just gets mixed up together (like if you put all your polygons and use the same points). Even if it did work on a real Saturn, I would still want to make it work in SSF as well since most people use emulators (but my guess is that it won't work well on a real Saturn). My current model only has 1 mesh.

Here is the function (the parameter being sent is (void*)0x00200000 (low work RAM adress)

void * LoadBinaryMAP(void * startAddress)
{

char * stream;
void * currentAddress;
int length;
mesh_tot = 0;
stream = jo_fs_read_file("MAPT.ZTM", &length);

register int idx = 0;
register int i=0;
register unsigned int ii=0;
register unsigned int iii=0;
register unsigned int nbpoint;
register unsigned int nbpolygon;
register POINT *pointArray;
register POLYGON *polygonArray;
register ATTR *attributeArray;

currentAddress = startAddress;
mesh_tot = jo_swap_endian_ushort(*((unsigned short *)(stream)));
idx+=2;

for (i=0; i<mesh_tot; i++)
{
nbpoint = jo_swap_endian_uint(*((unsigned int *)(stream+idx)));
idx+=4;
nbpolygon = jo_swap_endian_uint(*((unsigned int *)(stream + idx)));
idx+=4;
pointArray = (POINT*)currentAddress;
for (JO_ZERO(ii); ii<nbpoint; ii++)
{
for (JO_ZERO(iii); iii<3; iii++)
{
pointArray[ii][iii] = jo_swap_endian_int(*((int *)(stream + idx)));
idx+=4;
}
}
polygonArray = (POLYGON*) (pointArray + sizeof (POINT)*nbpoint);
for (JO_ZERO(ii); ii<nbpolygon; ii++)
{
for (JO_ZERO(iii); iii<3; iii++)
{
polygonArray[ii].norm[iii]= jo_swap_endian_int(*((int *)(stream + idx)));
idx+=4;
}
for (JO_ZERO(iii); iii<4; iii++)
{
polygonArray[ii].Vertices[iii]=jo_swap_endian_ushort(*((unsigned short*)(stream + idx)));
idx+=2;
}
}
attributeArray = (ATTR*) (polygonArray + sizeof(POLYGON)*nbpolygon);
for (JO_ZERO(ii); ii<nbpolygon; ii++)
{
attributeArray[ii].texno = jo_swap_endian_ushort(*((unsigned short*)(stream + idx)));
attributeArray[ii].sort = SORT_MAX;
attributeArray[ii].flag = Dual_Plane;
attributeArray[ii].colno = No_Palet;
attributeArray[ii].gstb = No_Gouraud;
attributeArray[ii].dir = MESHoff, sprNoflip, UseLight;
idx+=2;
}

//i, not 1, but I can't write it else the forum thinks I want italic
jo3dMesh[1] = (jo_3d_mesh*) (attributeArray + sizeof(ATTR)*nbpolygon);
jo3dMesh[1]->data.pntbl = pointArray;
jo3dMesh[1]->data.pltbl = polygonArray;
jo3dMesh[1]->data.nbPoint = nbpoint;
jo3dMesh[1]->data.nbPolygon = nbpolygon;
jo3dMesh[1]->data.attbl = attributeArray;

currentAddress = (void*) (jo3dMesh + sizeof(jo3dMesh[1]));
}
jo_free(stream);
return currentAddress;
}

The jo3dmesh is just an array of meshes for the drawing list. I declare it global like this : static jo_3d_mesh *jo3dMesh[10];
(I would like to get rid of the [10] so that I can scale it depending on the level, but I don't know how I could make it dynamic).
I know the stream isn't ideal, but right now it's the only way in Jo Engine AFAIK, the alternative would be to use SBL functions and read from pointers (I didn't have time yet to look SBL up)
I know that using the PointArray/PolygonArray isn't good, but I couldn't get it to work otherwise (again, I'm still learning all this)
So, the questions :
1) How should I rewrite my function to send everything to low-work RAM, tightly packed?
2) Is there anything here that would explain why it works in Yabause, but not in SSF? I guess it must be related to initialisation.

Of course, once I fix this, I will update the tool here and the FPS demo so that others can also enjoy loading levels from the disk!

 

[mrkotfw]

I'll look it over when I have more time, but here are some ideas:

1. There may be an unaligned read/write somewhere. Make sure that half-word reads/writes are on a 2-byte boundary, and same for words (check that they're on a 4-byte boundary).

2. Install CPU exception handlers. Check out https://github.com/ijacquez/libyaul/blob/master/libyaul/common/exception_handler.c

3. It could be the memory allocator. Statically allocate a buffer to rule out that it isn't the Jo allocator.


I believe SSF actually crashes if a CPU exception is thrown, so again, that might be it.

 

[XL2]

Thanks for your reply.
Sorry if it sounds dumb, but how do I use the CPU exception handlers?
I put it in my main loop?

I've attached a small test program just to see if the loading works.
I've also attached my latest version of the collision tool that reads obj and writes to binary files (custom format, I named it ZTM for now).

As SSF doesn't crash, I'm not sure if it will throw any exceptions.
When I look in Yabause memory for the LWRAM, there seems to be some "garbage" data way under the main chunk, so I guess I made mistakes with memory allocation.
Or maybe I just didn't initialize the memory correctly and need another function to put everything to zero before trying to populate the arrays?
Thanks again for your help!

EDIT : I changed SSF compatibility setting from "Full" to highest and now it works. I guess I should just try on a real Saturn now to see if it really works as it is my understanding that "Full" is more accurate than the highest setting.

EDIT 2 : Ok, so it doesn't even load on real hardware, it just crashes right away.

 

[cafe-alpha]

I didn't looked at the code in detail, but I think the problem is here :

register int idx = 0;
[...]
mesh_tot = jo_swap_endian_ushort(*((unsigned short *)(stream)));
idx+=2;

for (i=0; i<mesh_tot; i++)
{
nbpoint = jo_swap_endian_uint(*((unsigned int *)(stream+idx)));
idx+=4;

mesh_tot data width is 2 bytes, hence nbpoint (4 bytes large) in your for loop will be read from non 4 bytes aligned address, and consequently crash ?
A quick and dirty workaround would be either :
1. To use only 4 bytes data in your structure, so that alignment problem shouldn't happen.
2. Modify jo_swap_endian_* functions in order they accept pointer (rather than value) as input parameter, and do data only byte data access there.

 

[XL2]

Guys, you are awesome!!
You were both right.
I thought "data alignment issue" meant something like me trying to fit a int in a short, or vice versa, but I did some reading and now I understand better that it's all about bus width (32 bits) and memory addresses.
I did your fix cafe-alpha, I changed the mesh_tot to a int instead of short and idx+=4, not it works in SSF.
I kept the texno and vertices as short.
After I'm done with work today I will try to update the FPS demo and the collision tool (I guess I should change the name to "Map tool" or something). I still need to do more testing, but with binary files, Obj converter and collision data generation, the tool is starting to be more interesting.
I still need to improve the memory allocation as it seems I waste a lot of space as the memory is too spread (I use from 2000000 to 2120C0, which is way too much for such a small map).
Thanks again!

 

[mrkotfw]

I know this is out of the scope of your test, but have you looked into "proper" clipping for the test? I think this is due to SGL, but when you have polygons closest to the projection plane, the tend to disappear.