That's how you control the degree of polarisation. Your best bet is to compose the picture, then turn the filter whilst looking through the viewfinder until you get the effect you like most.
Some scenes, like the blue skies mentioned above, will show very strong polarisation, so you might decide to tone it down by backing off the polariser a little.
Have a read at
this page.