[Quiet Man Cometh]

Well that's all I can suggest. A calm conversation is likely to go over better than one that's happening in the middle of any action, and if you know what they'll say, then you need to work your arguments around that. Otherwise, I guess let him be there problem and put your stuff and such where he can't get it while you work on your own situation.