Former English National Opera music director Mark Wigglesworth has spoken of his struggle to keep "the art above the politics" during his