The only sure way is to look at what's already in it. Generally there are only three types that you really need to worry about unless you have an older (pre-Pentium 2) system or a high-end workstation/server system:
1) Normal SDRAM: This is what you'll have if you have an older P2/Celeron/K6-2/K6-3 system. This normally comes in 66MHz, 100MHz, and 133MHz flavors (like processor speeds, these are really just ratings - you can try to run them at any speed your motherboard supports, it's just not guaranteed to work). Industry standards exist for 100MHz and 133MHz modules called PC100 and PC133, but 66MHz SDRAM is sometimes also called "PC66" for the sake of consistent naming. This comes on a 168-pin module. In general, faster modules can be used in slower boards without problems - they'll just run at the slower clock speed. A 512MB stick of good PC133 costs about $150.
2) DDR SDRAM: This is what you'll most likely have if you have a recent Athlon/Duron system, or a P4 system based on a non-Intel chipset. This is similar to normal SDRAM, but faster and incompatible with a normal SDRAM interface. There are speed ratings/standards for these too, called PC1600, PC2100, PC2400, and PC2700 (don't pay attention to the numbers being so much bigger than PC66/100/133, it's just a marketing ploy). This comes on a 184-pin module, and like normal SDRAM faster modules will generally work in slower boards. A 512 stick of good PC2700 costs about $170
3) RDRAM (Rambus): This is the oddball. Intel was allied with Rambus and pushing this for a while, apparently in an effort to make life difficult for other CPU and chipset makers. It has superior bandwidth compared to normal SDRAM (and possibly superior to DDR SDRAM depending on how the system is built), but the additional cost is generally thought to be unjustified for all but the most bandwidth-intensive apps. Late Pentium 3 and early Pentium 4 chipsets were built for RDRAM, though Intel offered a "memory translator" for some that let board manufacturers use SDRAM instead. Intel later abandoned the big push for RDRAM, but still uses it for its high-performance P4 chipsets. There are different speds on these too, but I'm not entirely familiar with them. A 512MB stick of "800MHz" RDRAM costs about $200-250.