I agree with graham , not so much to keep dust off but to protect the front ellement from damage. I once took a cricket ball smack on the end of the little bigma (it was a bouncing four shot and I was trying to get it in the frame) smashed the filter and lens hood but their sacrifice saved the front element.
Another time, at santago leopard sanctuary an african leopard smacked the end of the lens with its foot (this was my canon 70-300mm) as i tried for shots of it pouncing, again the filter was lunched but the front element was undamaged - I had a lovely blackeye though
The only time you might want to take it off is iff you are putting more than one other filter on the front, stacking more than 2 can lead to vignetting and loss of image quality.
edit: yes definitely put one on all your lenses.