Shutter Speed

For the first 350 shots or so that I took with my new D80, I was very stingy with the ISO setting. Even in dark situations I would change it from 100 to 400 which didn't really help much in terms of increasing the shutter speed. I was always worried about introducing too much grain into my pictures;