[Coda]

Your body is a good electrical conductor, while the rubber of the balloon or the plastic of the pen is a good electrical insulator. Your own body ends up providing a connection between the cloth in your hand and the ground, so the cloth is able to drain off its excess charge THROUGH you.

In isolation -- say, if you used a pair of rubber tongs to manipulate the cloth... You might be right.

EDIT: Your other questions!

Yes, the buoyancy of balloons in the air helps. The pen doesn't have a lot of surface area to collect charge on, and it has more weight.

And yes, that's the limit. It's not STRICTLY a limit, mind you; it's not some fixed measurable cap. It's just a question of how much work you want to apply to the task. Rubbing the objects together by hand within the atmosphere may simply not impart enough energy to go beyond the small effect you've seen. On the other hand, van der Graff generators can build up HUGE potentials because their parts are very well isolated from grounding.