I am no expert on the spore thing, but I do know one forestry lecturer (or did, about 6 years ago) who was very strongly told to get rid of his collection of bracket fungi. It is something that needs considering even if it is very low risk. Certainly, allergy to mould on hay can be a right problem. I am not worried, as I don't intend to collect large bracket fungi, but if you are considering doing so, especially if you have breathing problems, then it might be a problem.