Why would I want to refinance, with the resultant costs, when I have a fixed rate, and can prepay at my leisure.

you would not,unless the rate is much lower. refinance costs can be close to zero if you deal with the right bank. talk to your banker ! Login to read more.