I think there is a distinct difference it certain cyclists. The ones I feel most confident of when driving are those who are experienced, cycle with confidence & follow the basis rules of the road. I have no problem with these cyclists using the road rather than a cycle path.
The cyclists I have a problem with are the ones who cycle so slowly that they can't help but weave around, move without signalling and, as Pauline says, try and overtake you and ones who have bags hanging from both handlebars which they chat on their mobile while cycling

!
When I get my bike out I'm no way near as fast as some of you have mentioned but I hope I cycle with confidence and with other road users in mind.
I'm sure it's probably very unworkable, I have no idea how it could work, but if only people had to have some kind of compulsory proficiency test.