I wear subdued coloured clothes, but in my experience it's other birders that get more agitated by bright clothes on an individual than the birds!
Generally on reserves the birds are often used to people walking around + realise here people are generally benign as opposed to a shooting club's area. Sudden movements of surprise loud noise are more likely to scare the birds than some bright coat.
Given that birds have far better eyesight than us I don't think all those barbour clad birders are invisible to our feathered friends!
