Boris Johnson has insisted allegations about his personal conduct are not overshadowing the Tory conference.
Journalist Charlotte Edwardes has accused the PM of touching her thigh, and that of another woman, at a lunch in 1999, which he denies.
Rumours were circulating at conference that Mary Wakefield – who is married to the PM’s chief adviser – was the second woman, but she has rejected that.
The PM said people wanted to hear about his plans to “improve their lives”.
The row erupted after Ms Edwardes’ wrote a column in the Sunday Times on the eve of the Conservative conference in Manchester, describing the alleged incident.
The PM was already facing questions over his ties to a US businesswoman, Jennifer Arcuri, during his time as London mayor – he insists he acted with “full propriety”.
The Conservatives are trying to focus this week on their key conference message – “Get Brexit Done” – and a raft of policy pledges.
In her first column for the paper, Ms Edwardes said the incident took place in 1999. She said she was seated on Mr Johnson’s right at the lunch, held at the Spectator magazine’s offices – he was editor of the magazine at the time.
“More wine is poured; more wine is drunk. Under the table I feel Johnson’s hand on my thigh. He gives it a squeeze,” she wrote.
“His hand is high up my leg and he has enough inner flesh beneath his fingers to make me sit suddenly upright.”
On Sunday evening, No 10 released a statement calling the claims “untrue”, but Ms Edwardes later tweeted: “If the prime minister doesn’t recollect the incident then clearly I have a better memory than he does.”
Asked on Monday if the incident had taken place, Mr Johnson said: “No.”
When it was put to him that the row was overshadowing everything else at the conference, he replied “not at all”.
“I think what the public want to hear is what we are doing to bring the country together and get on with improving their lives,” he added.
Ms Edwardes said another woman at the lunch later told her Mr Johnson had done the same to her.
Spectator magazine commissioning editor Ms Wakefield, who is married to the prime minister’s adviser Dominic Cummings, issued a statement to say she was “not the woman referred to in Charlotte Edwardes’s column”.
“Boris was a good boss and nothing like this ever happened to me. Nor has Charlotte, who I like and admire, ever discussed the incident with me.”
Earlier, ex-Tory minister Justine Greening said Ms Edwardes’ story was “deeply concerning”.
However, Ms Greening told BBC Radio 4’s Today programme: “I can’t comment on those accusations, but they are deeply concerning, and in a sense they go to the heart of this question about character and integrity of people in public life and what standards the electorate have a right to expect.”
Chancellor Sajid Javid, however, backed the PM.
“The prime minister has said that this is completely untrue,” he said. “I have full faith in the prime minister and I don’t doubt that and what he has said for a second.”
On Sunday, Health Secretary Matt Hancock said “these issues are incredibly important”.
He said he knew Ms Edwardes well and knew her to be “trustworthy”.
Former minister Amber Rudd – who quit the Conservative Party over its handling of Brexit earlier this month – tweeted that she agreed with Mr Hancock’s conclusion.
But Housing Secretary Robert Jenrick said, unless further evidence emerged, he would “take [the prime minister] at his word”.
“I don’t have any inside information into this,” he told BBC Politics Live.
“It’s very hard for any of us to speculate on what may or may not have happened.”