I would be inclined to put your ponies in there anyway if the Hornets flightpath is well out of the way, thinking shetlands, they are quite small and as the nest is up in the ceiling chances are the Hornets would fly in a fairly straight line to and from the nest so in theory they should be safe.
Check on the flightpath first and if it is as high as the nest or near enough there should be no need to worry as long as the shelter is standard size and not dog kennel size.
If i am waiting for a bus in the rain because the shelter is missing, i know who to call