What do they specifically saying is that the SOURCE of the fund is from a loan from another bank or institution. Usually banks levy foreclosure charges if we close it via Balance transfer or another loan.
So if you haven't borrowed it, try to show the source to him and ask for waiver of charges.