Don't take back the complaint under any circumstances. This is pretty common, especially with yes. They never lose the job as they are trained like this by the company itself. Banks generally hire them through 3rd party to save themselves from legal trouble.
Many of them, though work for 3rd party, have company id card and work under ASM in loan branches.