Evelyn de Rothschild left his family bank after an allegation of sexual misconduct, it has emerged