He didn't say it belongs to the Feds. He said removing it might be illegal.

Your mailbox doesn't belong to the Feds, it belongs to you. But if someone were to tamper with it, it would still be a federal offense. Login to read more.