Simon Cowell on Baby Drama: I Will Clear Things Up "When The Time is Right"

Bad timing for a Television Critics Association press event for Simon Cowell. The X Factor host appeared on stage in Beverly Hills Thursday to answer questions about Season 3 of his singing reality series just one day after news broke that the judge was expecting a child with his best friend's wife.

So there was little doubt that press would ask him about his alleged affair, a topic that overshadowed details of The X Factor Season 3. But did he respond to queries?

Unsurprisingly, no. Cowell managed to deflect questions surrounding his relationship with socialite Lauren Silverman, who is 10 weeks pregnant with the TV personality's first child. "I haven't read the newspapers," he joked to reporters before adding, "Unfortunately at this moment I have to keep it private."

Later, Cowell elaborated further to People, telling the outlet, "There are a lot of things I will eventually clear up when the time is right, but I really have to be sensitive because there's a lot of people's feelings involved here."

Cowell, however, had plenty to say about Season 3 of his reality series, which is undergoing changes following Season 2's middling ratings — a surprise, considering Britney Spears served as one of the judges. And it seems this fall's season is suffering the consequences, with the FOX series lowering the prize from $5 million to $1 million.

"It's like having a dinner party," Cowell said about his Season 2 panel, which included Spears, L.A. Reid, and Demi Lovato, who is staying on for Season 3. "You invite people for dinner — and sometimes its fun and sometimes it's not as fun as you thought it was going to be." Clearly, FOX is hoping dinner goes more smoothly with Season 3 judges Cowell, Lovato, Kelly Rowland, and Paulina Rubio.

Fans of the series will also see changes to the show's format come Season 3. What those changes are, however, is not yet known. "I felt that what we were doing was similar to what other people were doing, and I had this impatience to get on to Season 3 to do what I wanted to do," Cowell said. "You have to be competitive, and you have to make changes. I work hard. It makes it fun. I'd love to be number one."

Especially since the judge famously missed his mark in Season 1 when he claimed X Factor would attract 20 million viewers. (The Season 2 finale instead attracted only 9.6 million viewers.) But, amid chatter surrounding American Idol's judging panel — the network announced that Keith Urban would return to series come Season 13 — would Cowell ever return to a series that had attracted 20 million viewers during his run?

Unfortunately for Idol fans, the answer is no. "I can't imagine the scenario," he said. After all, following the news of the impending birth of his first child, we imagine he'll have plenty to do.