[Fauxreal]

Well, what you said just now made sense and didn't sound like it was really hurting anybody.
You work with kids, and when you aren't at work it's just not your bag to hang out with more kids.

But yes - I say just be honest. If your friend doesn't understand that, I'll come over and we can hang out without kids running around.

Do you play board games? :)