An emulation can look better than original hardware. There are really a limited number of things that are reasonable to do before compatibility starts suffering though. For example, an emulator could filter individual textures/sprites or the entire screen (e.g. with interpolation) and perform antialiasing on polygon edges. However, things like adding polygons to models or extending visible distance are too game-specific to really be feasible at the emulator level.
However, if the emulator author is willing to do work on a per-game basis, radical changes in visual quality can be made by creating new sets of graphics for existing games. At least two emulators,
RetroFX and
EmuDX, are based on this concept. Nebula also allows the use of external semitransparency masks.
In any case, the generic stuff is nothing that couldn't be done with new hardware; the division between hardware and software is largely arbitrary anyway...