Pressure is mounting on the PM to act over his senior aide’s lockdown trip, as the cabinet is to meet later to discuss plans to ease restrictions.
Senior Church of England bishops and a scientist advising ministers on the pandemic have strongly criticised the government’s handling of the row.
Acting Lib Dem leader Sir Ed Davey said it undermined the PM’s authority and Labour has called for a inquiry.
Mr Johnson has defended his aide, who had travelled from London to Durham.
Speaking at Sunday’s Downing Street briefing, the prime minister said he believed Mr Cummings had “no alternative” but to make the journey at the end of March for childcare “when both he and his wife were about to be incapacitated by coronavirus”.
At the press conference, Mr Johnson also confirmed the phased reopening of England’s primary schools will begin on 1 June.
The prime minister is this week expected to set out details of plans to ease restrictions.
Plans to be outlined by the government will reportedly include information about the reopening of some non-essential shops in June.
However, BBC political correspondent Iain Watson said Mr Johnson is finding it difficult to shift the political focus away from his key adviser.
Speaking on BBC Breakfast, Sir Ed said that the row over Mr Cummings was “preventing the government from getting on and doing its job, and doing it better”.
He said that the prime minister should sack Mr Cummings “so the government has more credibility in what it says about public health”.
Sir Ed added added: “The instruction the prime minister gave us all to stay at home has been breached by his top adviser and that’s what you can’t get away from in this story, its pretty simple.
“I hope the prime minister will come to his senses, recapture his judgement and reinstall authority on this crisis by acting.”
Boris Johnson failed to close down Cummings story
If Boris Johnson’s decision to appear at Sunday’s press conference was an attempt to close down the story about Dominic Cummings’ behaviour during the lockdown by handling it himself, it failed completely.
It certainly was not an attempt to give the public the full information.
Instead the prime minister refused to answer the questions that remain about the specifics of his adviser’s visit or visits, to the north east of England while his team was telling the public again and again and again that they had to “stay at home”.
Some of the scientists that advise ministers were concerned the prime minister’s decision to back Dominic Cummings would undermine the message on controlling the virus.
Stephen Reicher, a professor of social psychology who has advised the government on behavioural science during the pandemic, said the prime minister’s backing of Mr Cummings made him feel “dismay”.
He said trust was vital to maintaining public health measures. “You can’t have trust if people have a sense of them and us, that there’s one rule for them and another rule for us,” he told the BBC.
And the Bishop of Leeds, the Right Reverend Nick Baines, said Mr Johnson was treating people “as mugs” and the Bishop of Bristol, the Right Reverend Vivienne Faull, accused the prime minister of having “no respect for people”.
Meanwhile an investigation has been launched into a tweet posted on the official UK Civil Service Twitter account, which asked “Can you imagine having to work with these truth twisters?” The since-deleted message appeared shortly after Mr Johnson’s daily news briefing.
The prime minister said he held “extensive” discussions on Sunday with Mr Cummings, who he said “followed the instincts of every father and every parent – and I do not mark him down for that”.
One report alleges that a witness saw Mr Cummings in Barnard Castle, more than 25 miles from Durham, where he had been self-isolating, on 12 April.
Labour leader Sir Keir Starmer said he would have sacked Mr Cummings if he were prime minister, and he said Mr Johnson’s failure to to take action was “an insult to sacrifices made by the British people”.
Sir Keir said: “This was a huge test of the prime minister and he has just failed that test.
“Millions of people across the country have made the most agonising choices – not visiting relatives, not going to funerals – they deserve better answers than they got from the prime minister.”
Ministers to publicly support Mr Cummings include the Health Secretary Matt Hancock and Cabinet Office Minister Michael Gove.
But Sir Roger Gale is among the Conservative backbench MPs to have publicly questioned Mr Cummings’ position, telling BBC Radio 5 Live: “It’s up to the prime minister to exercise judgement about who he has around him. In this case, I do think that that judgement is flawed.”
Another Tory, former minister Paul Maynard, said of Mr Cummings: “It is not as if he was unfamiliar with guidance he himself helped draw up. It seems to me to be utterly indefensible and his position wholly untenable.”
Meanwhile, in a statement posted on Twitter, Amanda Hopgood, leader of the Liberal Democrats on Durham County Council, said “a number of local residents have reported seeing Dominic Cummings on several occasions in April and May”.
She said that “given the clear public interest” she has referred the matter to Durham Police to see if there had been a breach of the coronavirus regulations.