"The logistics of these negotiations have never been simple or predictable," the White House said on Thursday night.
Vance and the US delegation had been ready to depart as soon as plans were finalised.
The talks, set for the mountaintop resort of Burgenstock, would not take place, Switzerland's foreign ministry confirmed, but gave no details.
There was no immediate response from Iran, which had earlier said it was ready to begin technical talks after Wednesday's 14-point accord extended a tenuous ceasefire by at least 60 days.
Iran's negotiators first needed to see signs of the US implementing the interim deal, and there was no confirmation its delegation would travel to Switzerland, the semi-official Tasnim news agency said before Vance's Thursday announcement.
US officials had also said they would hold a formal signing ceremony for the US-Iran agreement in Switzerland, but Iran's foreign ministry called it unnecessary after both countries' presidents signed the pact.
The war, which began on February 28 with US and Israel air attacks on Iran, has killed at least 7000 people, sent energy prices soaring and shaken global markets.
Israel, left out of the peace talks, has distanced itself from the US-Iran accord and kept up fighting against the Iranian-allied Hezbollah militant group in Lebanon, also raising questions about whether the agreement would hold.
In Washington, some of US President Donald Trump's Republican allies in Congress questioned whether he had conceded too much in order to end the conflict, unpopular with most Americans in the run-up to midterm elections in November.
Trump had sworn to end the war only with Iran's "UNCONDITIONAL SURRENDER".
But the memorandum signed with Iran instead provides relief from economic sanctions, unfreezes assets worth tens of billions of dollars and immediate US waivers for its exports of oil.
Iranian Supreme Leader Ayatollah Mojtaba Khamenei said Trump had signed the deal "out of desperation" and signalled that approaching talks over Iran's nuclear program, among Trump's stated reasons for starting the war, would not be easy.
"If the American side wants to be too demanding, we will not accept it," he said in a message.
The deal gives negotiators 60 days to agree on the status of Iran's nuclear program, unless an extension is agreed, and set up a $US300 billion ($A427 billion) reconstruction fund for Iran and other financial incentives.
When the US and Israel launched the war nearly four months ago, Trump said he aimed to destroy Iran's nuclear capabilities to ensure it could never develop such weapons.
He also sought to end Tehran's ability to strike its neighbours, prevent it backing allied anti-Israel militants in the region and make it possible for Iranians to topple their theocratic government.
None of those objectives had been met when Trump signed the agreement, in which Iran restated its decades-long assertion not to get or develop nuclear weapons, a position doubted by a succession of US presidents.
US officials say the negotiations could still yield a strong agreement on Iran's nuclear program, aiming to better one dating from 2015 between Iran, the US and other countries that Trump tore up in his first term.
But critics say Iran is in a stronger position now, having withstood a superpower attack, demonstrated its control of the Strait of Hormuz and gained valuable waivers to financial sanctions.
Iran has said it will still exert control over Hormuz in partnership with neighbour Oman, and intends to charge ships service fees that did not exist before the war, although not during the 60-day talks.
Oil prices dipped on Friday as prospects brightened for more supply after tankers began moving through the reopened Strait, which had carried nearly a fifth of global crude oil and liquefied natural gas supplies before the war.
