Plans to develop flats above the Finborough Theatre in west London have been rejected by the council, securing continuation of the venue.