Sterling: Banned for Life!

Donald Sterling has been banned for life from the NBA.

Question:  Presumably, Sterling assumed he was having a private conversation.  Do we have the right to punish people for private conversation, regardless of how repugnant that conversation is?

I think Sterling is a pig but I struggle with the notion that Americans cannot have private conversations without the public’s prying eye.  Help me through this obvious psuh me/pull me feeling.  I had the same conflict with Dawg the Bounty Hunter.  He assumed his conversation was private.

Absolutely people are responsible for their public words.