Welcome to WAB, Annee!
If the nest isn't anywhere near your home and you're having problems locating it - chances are, that it may not even be local! We're fast heading towards Autumn now and I'd expect their activity to start tailing off now.
Secondly, moving it is most likely to end up with you getting stung... and not just by
one!

Even if you were to successfully move it without too many stings - they may abandon through damage, and set up home nearer you! The nest may of course be in a nature reserve. If so, you would have no right in moving it.
I take it you have your windows/doors open/on a chain due to the summer heat - that's how they're getting in? Well, you don't
have to shut them - just put mosquito netting up over them, secured at the four corners. Also, try keeping the lights down to a minimum around this time, or try lower wattage. It's most likely the light that is attracting them.
I have yet to see a hornet - send a few my way, won't you?

I can definately see your concern, especially around children. Keep us posted. Good luck
Take care,
Jason