After she was pulled into the Jeffrey Epstein scandal that has swirled around her ex-husband, Prince Andrew, for more than a decade, Fergie has seen backers fall away.