It seems like many gaming groups use dice, colored pawns, disks, counters and other objects as playing pieces for their RPG sessions. In this day and age of collectable minis, downloadable standups and old school metal minis. Using something so abstract just seems silly.
I find that eBay is a great source of miniatures of all types. Most gamers don't know that one can find common minis from the plastic collectable miniatures games here for as little as $.25 a miniature.
So If you are looking for Fantasy (ie Mage Knight and Dungeons and Dragons minis), Star Wars, Superheros (ie HeroClix) Ebay is the way to go.
Oh and if you don't care for the way the mini is based (ie MageKnight and HeroClix) it doesn't take much work to pop those minis off their "Clix" base and reglue them to a 25mm disk or a 25mm magnetic base.
Precis Intermedia, PI's Disposable Heroes are minis that you print out on your inkjet or Laser Printer. The economy here comes when you print large amounts of the minis you like. If you don't want to waste ink/cardstock then they also sell the same artwork preprinted on cardstock.
Worldworks Games sells printable scenery pieces and has a collection of standups that can be printed out
There are other free sources of standups available. Google can be your friend in finding them,
Many board games have nice game pieces that can be used as miniatures for RPGs.
Hero Scape has nice booster sets that have nice minis that don't seem to cost a bunch. Check discount stores and eBay for deals on older sets. Sometimes an otherwise bad game will have great pieces that can be salvaged for your RPG fun.
I'll talk about traditional minis and getting into the hobby without spending a ton of money later.