As a general rule, DO NOT pay for apps! I have an iPod touch with 9 pages of apps on it, and every last one of those was free. They do everything I want them to. For every paid app, there are probably at least 3 others that do the exact same thing and are free. There are a few (very few!) exceptions to this, but I've never found an app that didn't have a free counterpart that I wanted badly enough to purchase it.
Just go through the Featured and the Top 25 or 50 Free apps; that's a great starting block. After that, scrolling through categories is a good way to find interesting stuff.