The Tony winner visited the late night TV show to promote her new holiday album Christmas: A Season of Love.