A separate blast caused at least 11 more casualties in Kunduz, another northern Afghan city, according to a provincial health official.
IS claimed responsibility for both attacks, according to statements on the group's Telegram channel.
The explosions happened during the Muslim holy month of Ramadan and two days after blasts tore through a high school in a predominantly Shi'ite Hazara area in western Kabul, killing at least six.
"A blast happened in 2nd district inside a Shi'ite mosque," Mohammad Asif Wazeri, the spokesman for the Taliban commander in Mazar-e-Sharif told Reuters.
Zia Zendani, the spokesman for the provincial health authority, said 11 people had been killed and 32 wounded in the blast.
The Shi'ite community, a religious minority in Afghanistan, is frequently targeted by Sunni militant groups including IS.
Hospitals in Kunduz had received 11 killed or wounded people in a separate explosion, according to Najeebullah Sahel, from Kunduz's provincial health authority.
An Interior Ministry spokesman said a roadside blast had targeted a van of military mechanics in Kunduz and said school students were among the wounded.
He added another roadside blast in the capital, Kabul, had wounded three, including a child.
Wazeri, the spokesman, also said Taliban forces had come under attack from unknown gunmen at a property in Mazar-e-Sharif, the provincial capital, on Thursday evening and that both sides were exchanging gunfire.
He did not confirm whether the attackers were linked to the blasts and said their identity was unknown.
Richard Bennett, the United Nation's Special Rapporteur for Afghanistan on human rights, condemned the blasts.
"Today more explosions rocks Afghanistan... systematic targeted attacks on crowded schools and mosques calls for immediate investigation, accountability and end to human rights violations," he said in a tweet.
A resident of Mazar-e-Sharif said she was shopping with her sister in a nearby market when she heard a large explosion and saw smoke rising from the area around the mosque.
"The glass of the shops was broken and it was very crowded and everyone started to run," the woman, who declined to be named, said.
