"The X Factor" judges Paula Abdul and Nicole Scherzinger, as well as host Steve Jones, will not return for a second season of the Fox singing competition, according to EW.com.