I have had this happen many times. Not only do I never use the word "Yes", I just hang up. My account still gets crammed occasionally.

I'll have to call ATT about a password. I believe you can also request that any changes are authorized in writing only. Login to read more.