• Rise from your gwave!

Building your own toolchains

Discussion in 'Saturn Dev' started by antime, Dec 25, 2007.

  1. antime

    antime Extra Hard Mid Boss

    If you can't or don't want to use a prebuilt toolchain there is the option to build one yourself. Since the Saturn is a "bare-metal" environment this is a much simpler process than what is described in most tutorials, and there is no need for any http://www.kegel.com/crosstool/]extra tools[/url]. The instructions here work on Linux and OS X. They should work on Cygwin as well, but I haven't tried it myself. Getting the build working under MinGW probably requires some extra work, but on Windows you're probably better off using the GNUSH kit anyway.

    To build the toolchain you obviously need the developer tools for your OS. On OS X this means installing Xcode, and on Linux you need at least GCC, GNU make and the autotools (autoconf/automake). The binutils and GCC documentation lists the requirements in more detail.

    Before you start building, there are two things you must decide: the toolchain object format and install location. The object formats available are COFF and ELF. COFF is an older, partly proprietary standard that is being deprecated in favour of the more open, capable ELF format. Sega's binary libraries are in COFF format, so if you intend to use them you may want to configure your toolchain for COFF, but otherwise ELF is recommended. Note that this choice affects the object format your toolchain produces - it can still understand both formats and you can convert object files using the objcopy tool.

    If you picked COFF, your TARGET will be sh-coff and if you picked ELF it is sh-elf. (The canonical name or "configuration triplet" is sh-unknown-coff or sh-unknown-elf, but typing the full name gets annoying very quickly and the shorter form works just as well.)

    The install location can be basically anywhere on your system. If you pick a system folder (eg. /usr/local) you will need to run the installation steps with root privileges, but you can just as well install the toolchain in your own home directory. The only thing you must do is to add INSTALLDIR/bin to your path. The installed tools will find their other bits automatically. Note that INSTALLDIR should not contain any spaces.

    The first thing to build are the binutils, so download the newest version of the sources to a new folder somewhere on your harddrive. The path to this folder should not contain any spaces either, and will be referred to as the "build root folder". Open a terminal, go to the build root folder and extract the binutils source archive. Then create a new folder in the build root folder called "build-binutils" and change to the build-binutils folder:
    Code:
    cd [i]build root folder[/i]
    
    tar jxf binutils-x.yy.tar.bz2
    
    mkdir build-binutils
    
    cd build-binutils
    All the toolchain components use a system where the build is done outside the source folder. This has the advantage that if something goes wrong you can start over just by deleting your build-folder and creating a new one. Next configure the build environment:
    Code:
    ../binutils-x.yy/configure --prefix=[i]INSTALLDIR[/i] --target=[i]TARGET[/i] 2>&1 | tee configure-log
    The "2>&1 | tee configure-log" at the end means that all output is captured to a file called "configure-log" which can be useful if you run into problems. There are a number of other configure-time tweaks which are documented either in the manuals or the source code, but it's best to leave them alone unless you're absolutely certain what they do. One useful option is "--disable-nls" which means all the messages printed by the tools will be in English rather than translated to whatever language your operating system is set to use. This is good because you don't have to worry about poor translations and other people will understand the error messages as well. It doesn't affect the produced code in any way. Once the configuration is finished, you can compile and install the binutils:
    Code:
    make all 2>&1 | tee make-all-log
    
    make install 2>&1 | tee make-install-log
    To verify that the build completed succesfully, run "sh-(elf/coff)-as --version" and the assembler should print out its version string. If you get an error that the command can't be found, verify that you've added INSTALLDIR/bin to your path directive.

    To build the C compiler you need the C library, but to build the C library you need a C compiler. To solve this chicken-and-egg problem you will first build a bootstrap compiler that can be used to build the C library. First download the GCC sources to the build root folder (you only need the gcc-core and gcc-g++ packages), then change to the build root folder, unarchive the GCC sources and create a folder called "bootstrap-gcc":
    Code:
    cd [i]build root folder[/i]
    
    tar jxf gcc-core-x.y.tar.bz2
    
    tar jxf gcc-g++-x.y.tar.bz2
    
    mkdir bootstrap-gcc
    
    cd bootstrap-gcc
    Next configure, compile and install the bootstrap compiler:
    Code:
    ../gcc-x.y/configure --prefix=[i]INSTALLDIR[/i] --target=[i]TARGET[/i] --without-headers --with-newlib --disable-shared --disable-threads 2>&1 | tee configure-log
    
    make all-gcc 2>&1 | tee make-all-gcc-log
    
    make install-gcc 2>&1 | tee make-install-gcc-log
    Now you're ready to build the C library. Download the newlib sources to the build root folder, unarchive them and create a "build-newlib" folder:
    Code:
    cd [i]build root folder[/i]
    
    tar zxf newlib-x.yy.tar.gz
    
    mkdir build-newlib
    
    cd build-newlib
    Configuring and building newlib is pretty straightforward:
    Code:
    ../newlib-x.yy/configure --prefix=[i]INSTALLDIR[/i] --target=[i]TARGET[/i] 2>&1 | tee configure-log
    
    make all 2>&1 | tee make-all-log
    
    make install 2>&1 | tee make-install-log
    As the final step we build the full C and C++ compilers. Go to the build root folder, create a "build-gcc" folder, configure and build. The options that are disabled either aren't supported on SuperH or require special operating system or library support. Note that the "--disable-nls" option works with GCC as well.
    Code:
    cd [i]build root folder[/i]
    
    mkdir build-gcc
    
    cd build-gcc
    
    ../gcc-x.y/configure --prefix=[i]INSTALLDIR[/i] --target=[i]TARGET[/i] --enable-languages="c,c++" --disable-shared --disable-threads --disable-multilib --disable-mudflap --with-newlib 2>&1 | tee configure-log
    
    make all 2>&1 | tee make-all-log
    
    make install 2>&1 | tee make-install-log
    
    
    That's it! The same instructions can be used to build a 68000 toolchain, just set the target to m68k-elf or m68k-coff. After you're done, you can freely delete the source and build folders, and when new versions of the tools are released you can install them on top of the older versions.
     
  2. vbt

    vbt Staff Member

    Nice tutorial Antime. By that ay would it be possible to make a toolchain that produces better/faster binaries ?
     
  3. Kuta

    Kuta Staff Member

    That looks like a good description. Why not write a document and add it to the Saturn Dev section?
     
  4. antime

    antime Extra Hard Mid Boss


    If you're familiar with compiler technologies and the GCC codebase you can of course add or tweak the optimizers, but there's no magical configuration switches that will make the compiler produce better code.
     
  5. Omni

    Omni New Member

    Thanks for the guide. These worked very well under Cygwin...even it took several hours to compile everything. :) I don't think the new compiler suffers from the same problems as the GNUSH one, under Vista.

    However, I am unable to make a compiler to target sh-coff when making a compiler from the very latest sources (GCC 4.2.2, Binutils 2.18). Configure for binutils works fine, but when doing make all, I received a message that says "support for this target has been dropped." I couldn't find anything about that on Google, but I would have guessed there would be a note about it somewhere.

    I knew GNUSH dropped support, but I didn't think that had anything to do with GCC itself. I'm relatively new to all of this stuff, but I suppose sh-elf-gcc will work just as well.
     
  6. antime

    antime Extra Hard Mid Boss

    I/O under Cygwin is really slow, that is one of the reason why MinGW-based toolchains are popular on Windows.


    Support for the sh-coff configuration seems to have been dropped when gas dropped all assemblers that didn't use the BFD library. If you do run into problems caused by using the sh-elf configuration I'd be interesting in hearing about it. I've always used that myself, but I've never tried using Sega's libraries.
     
  7. Omni

    Omni New Member

    One thing I have noticed...


    I can compile nearly all of the SGL samples correctly. However, the demos BIPLANE and FLYING give me great troubles.


    I get an error about "undefined reference to __sdivsi3_i4i", which I believe has something to do with the math library (is this libm.a? I believe that's the offending library) and the SGL square root function. For this reason neither BIPLANE nor FLYING will compile.


    Is this a general problem with those demos, or specifically with my compiler? SaturnOrbit's included KPIT GCC doesn't work under Vista, so I can't test it with another compiler.


    If so, of course, I have no clue how to fix this problem.
     
  8. antime

    antime Extra Hard Mid Boss

    Looking at the GCC sources it seems to be an optimized integer division routine. According to this KPIT support answer, the definition is provided in the library gcc-Os-4-200.
     
  9. Omni

    Omni New Member

    I remember seeing a note about that library on google, and I have that library.

    However, doing as KPIT Support says and setting the linker libraries in the Saturn makefile as

    -lm -lc -lgcc -lgcc-Os-4-200 [followed by the SGL libraries]

    doesn't seem to help. Still gives the same error. Would this have anything to do with the fact that I must pass "-L$(SGLLIB) -Xlinker --format=coff-sh" to allow linking with the SGL libraries? I don't know if that command has any effect on the _other_ libraries you link with.

    While we're at it, I have a small question about these library directories. I have two groups of libraries, one in

    (gcc root)/lib/gcc/sh-elf/4.2.2

    and one in

    (gcc root)/sh-elf/lib

    Since the compiler itself is sh-elf, I have the idiot question: why are there two sets? The libraries are spread between them (for example, libgcc.a and libgcc-Os-4-200.a are in the former while libm.a and libc.a are in the latter).
     
  10. antime

    antime Extra Hard Mid Boss

    I don't think so. Make sure the libraries are listed after the object files on the command line.


    As far as I can tell, /sh-elf/lib holds stuff related to the standard libraries, while /lib/gcc/ holds some support libraries private to the compiler.
     
  11. antime

    antime Extra Hard Mid Boss

    As an addendum/attempt at explanation on what is going on, if a platform doesn't support some particular operation GCC emits calls to specific helper functions that implements the operation instead. In this particular case GCC doesn't like the SuperH single-step division and instead calls the function __sdivsi3_i4i to perform signed 32-bit by 32-bit integer division. Due to some real brain-deadness on part of the GCC SH developers this SuperH-specific helper function is used by default, but is not included in the normal libgcc and so you have to link against libgcc-Os-4-200 as well.


    A neat thing about these helper functions is that they are implemented in a way that lets you replace the default implementation just by providing your own function with the same name - this will not cause a linker error. This provides us with an easy way to replace the GCC-provided division routines with one that uses the SH7604's on-board division unit. The specifications for this routine is that the parameters are passed in r4 and r5, the result is returned in r0 and r1 can be destroyed. The math library in SBL does in fact do this, but I think the function name used there is not valid anymore.


    (Note that in the SH port of GCC there exists another way to replace the division routine, namely the "-mdivsi3_libfunc" command-line switch. For example, if you wanted to use the same division routine as in earlier GCC releases, you would use
    Code:
    -mdivsi3_libfunc=__sdivsi3
    This function is located in libgcc and you no longer need the other library, but is potentially slower/bigger. You can of course also use this switch to use your own routine instead.)
     
  12. Omni

    Omni New Member

    I tried the -mdivsi3_libfunc=__sdivsi3 for my compiler flags, but I still can't get it to work. But the explanation about GCC platform helper functions was very helpful, thank you.

    I wonder why including the gcc-Os-4-200 didn't work? I mean, that part makes perfect sense to me...and I _believe_ I modified the makefiles correctly. But I'll work on something else for a while.

    EDIT: WAIT! It DID work!

    Adding -lm -lc -lgcc -lgcc-Os-4-200 didn't work, but I just tried adding -lm -lc -lgcc-Os-4-200 did, when I took out -lgcc. The error about undefined reference to __sdiv****etc went away. It now complains about a missing reference to a palette variable for one of the textures, but that's completely different. I am very glad to fix that. Thanks a lot!

    EDIT2: It took both changes. I had to add -lgcc-Os-4-200 to the libraries and ALSO add -mdivsi3_libfunc=__sdivsi3 to the compiler flags. Without the -m flag, linking the library alone doesn't seem to help.

    EDIT3: It works with both -lgcc and -lgcc-Os-4-200 linked; so -lgcc linked as well is not the problem. I just needed the -m flag too.

    EDIT4: [EDIT5: This error reported below was my own fault, please ignore it. I failed to correctly link the COFF SGL libraries themselves.] The error was as follows, from SSF:

    -----------

    MasterSH2 code = 2000

    00000000: (2000) MOV.B R0, @R0

    SlaveSH2 code = 6010

    060208C2: (6010) MOV.B @R1, R0

    68000 code = 60FC

    00000402: (60FC) BRA $00000400

    SH2 Unknown Code: 0200

    -----------
     
  13. Omni

    Omni New Member

    Some last notes: the -m flag seems to work, whether I link gcc, gcc-Os-4-200, or neither of them; it works with neither linked. I don't know why the -m flag didn't work at the start of my earlier post, but now it seems to always work. Maybe I made a typo in the makefile.

    However, I am certain I copied the error message from SSF verbatim. I have not yet tried it on my real Saturn, though; maybe it would still work.

    Oh...oh dear. I just figured out the mistake. I had messed with the makefiles for the FLYING SGL demo earlier and changed them to use those LIB-ELF libraries made with objcopy in Saturnorbit; I figured maybe the problem was with the use of COFF libraries and ELF libraries at the same time. Changing them back to the COFF libraries and using --format=coff-sh works fine.

    So the FLYING demo works, with the -m***** flag. But only with the -m flag; I cannot get it to work otherwise, no matter what libraries I link. Is something wrong with my gcc-Os-4-200?

    Sorry for my incompetence; in summary, your suggestion to use the earlier GCC routine worked fine. But I couldn't get the demo to compile any other way; even with -lgcc-Os-4-200, it still complains about an undefined reference to __sdivsi3_i4i.
     
  14. antime

    antime Extra Hard Mid Boss

    Can't really say I have any ideas on what's going on with your builds. Did you make sure to clean out any old object files when you changed the build flags?


    By the way, you don't need to explicitly link against libgcc. It is always implicitly linked, unless you use either the "-nodefaultlibs" or "-nostdlib" compiler switches.
     
  15. Omni

    Omni New Member

    Yes, I clean the object files every time. Do you also need to explicitly link libm.a? I assumed that was a math library earlier, but I could be wrong.


    I'm not sure what's wrong either, but I am satisfied enough that I was able to get the demo to compile with the -msdivsi3_libfunc flag. It's not a huge deal for me; I'm not too worried about why the gcc-Os-4-200 library doesn't seem to work when I link it. I don't think my build process has a huge problem (but that's open to debate; I newly created these makefiles based on the ones from SaturnOrbit a few weeks ago and I _thought_ I had everything working properly). As long as I know that my compiler doesn't have a deep-seated problem that will cause me heartache later, I'm satisfied.
     
  16. ExCyber

    ExCyber Staff Member

    I'm seeing this too, with GCC 4.3.2. The '-mdivsi3_libfunc=__sdivsi3" flag fixes it with no need to explicitly link anything from libgcc. My testcase is:


    Code:
    int main (void) {
    
            signed long a = *(signed long *)0xfffffff0;
    
            signed long b = *(signed long *)0x00000000;
    
            return a/b;
    
    }
    The pointers are essentially arbitrary, and used in lieu of constants to stop any optimization passes from precalculating the result.


    I looked through an strace a bit (as if gcc wasn't complicated enough on its own, cygwin injects unbelievable amounts of bullshit), and gcc (rather ld wrapped by gcc, I suppose) is opening libgcc-Os-4-200.a, yet it doesn't resolve the function. What's interesting is that it also fails for -m3 and -m4, but not -m1 or even -m2a (the latter seems to be undocumented, but I assume it's for the newer SH-2A microarchitecture) ...


    edit: building with the following gcc configuration seems to make the problem go away, for reasons that I don't really understand (it was a bit of a shot in the dark):


    Code:
    ../gcc-4.3.2/configure --prefix=$PREFIX --target=$TARGET --enable-languages="c" --disable-shared --disable-threads --disable-mudflap --with-multilib-list="m2" --with-newlib --with-cpu=sh2
    ($TARGET is sh-elf)
     
  17. trap15

    trap15 New Member

    I am unsure if I'm building binutils wrong or something, but whenever I try to convert any coff-sh binary to elf32-sh, objcopy segfaults on me. I first tried converting one of the SBL libraries, and that segfaulted, so I tried building a very minimal program, converting from ELF to COFF, then converting COFF back to ELF, and it segfaulted again. At first I took a guess that it was because I was on a 64bit system, so I recompiled binutils with -m32 so it built a 32bit binary, but that gave a sagfault as well.

    Basically, if someone can convert the libs for me, or something, it'd be very appreciated [​IMG]

    EDIT: Uhm, woops, didn't realize this topic was so old! D:

    EDIT2: I figured out the problem. The newest versions of binutils don't work properly. Anyone trying to do this should use version 2.18 of binutils, as it's the newest version that still works.
     
  18. antime

    antime Extra Hard Mid Boss

    It's probably worth it to file a bug report about it.
     
  19. slinga

    slinga Member

    Do you have a simple test source file? I'm trying to compile "Copperbar" but its complaining about VDP2_RAM_BASE being undefined. I grepped through the header files and I didn't see it there. I'm looking for just a simple "hello world" if you have one.
     
  20. antime

    antime Extra Hard Mid Boss

    Change it to "VDP2_VRAM", I must have changed my mind about how I wanted to name the macros but forgot to update the code.
     

Share This Page