[Quiet Man Cometh]

I've heard a couple potential origins for the Easter bunny and such, like rabbits being a symbol of fertility for a god that the Romans would have known about (can't remember which) and may have adopted into their own culture, or an interpretation of a statue of Artemis/Diane as being covered in eggs.

EDIT: Goddess was Ishtar, apparently pronounced "Easter" and was something of the usual Christian usurping of details.