When I skate my board turns without me wanting it to?

I got a new board quite recently and I've started to skate around town a bit more, recently i've noticed that when i skate with the board facing forward, it slowly turns to the left and when its facing backward it slowly turns right. at first i thought that one wheel was slightly looser than the others so tightened it, nope, that wasn't the problem, if you know whats happening please help, its really annoying and when i skate past someone i nearly skate into them!

Thanks, Joe