People used them for many things, and still do. For example, until a couple of years ago, the display at the nearby subway station what showed bus departure times was controlled by a C64 (as everyone could see when one day the computer had reset itself and came up showing the default startup screen).
At my dad's workplace they used it as a datalogger for an oscilloscope and he used our machine for scientific computing (this was around 1984-1985). People have used them to write books and run businesses. You obviously haven't seen any of the more serious Commodore mags (eg. 64'er or Compute's Gazette), there were some really heavy projects in them.
Anyway, if a machine is primarily designed to run video games, it can be called a video game system. The C64 wasn't.