I do sometimes just not take a camera at all, I find the lack of concern quite liberating. Casual "just going for a wander" walks can be dealt with by carrying my much loved Nikon 4500. If you're walking, walk, if your going to take photos, go loaded for bear.
I must emphasize that this is just what suits me, and not a rant to say that this is what you should do.
Similarly, when the children were still young, I did get fed up with seeing the world through a piece of glass, and cut back drastically on photos. Each one now is so much more of a memory trigger, and we don't have to watch long ordeals of many photos.