No, nobody's punished. The winning side just gets to enjoy the sweet feeling of victory and knowing that they did a good job. And sometimes, in more official games, they get a trophy.

Ideally, you play in teams, with five people on each side. But it's pretty easy to modify it for any number of players.