This safe haven for music lovers broke down racial barriers " and hosted luminaries including Marvin Gaye, Billie Holiday and Ella Fitzgerald