"At least" one perpetrator had attacked several people at the gym located across from the Duisburg city hall, using a "stabbing or cutting weapon," according to police.
The people who were injured were being treated in hospital, the police said.
No one has been arrested yet after the attack, police said on Tuesday evening.
Police in Duisburg were alerted by emergency calls about 5.40pm, they said.
Investigators searched the gym and secured it, police said shortly after 8pm.
Witnesses who were working out at the time of the attack are currently being questioned.
According to a DPA reporter at the scene, heavily armed special police forces were on site in the evening.
Eyewitnesses were questioned by the police in a nearby restaurant, he reported, adding that one person was still carrying his sports bag.
A police spokesman said earlier there is no information yet on whether one or more perpetrators are still on the run.
Police called on citizens to avoid the area around the city hall, adding that a large police contingent was present.
Public broadcaster WDR reported earlier that two perpetrators stabbed visitors in a fitness centre in the old town of Duisburg, 70 kilometres north of Cologne.
Germany's Bild newspaper reported that police suspect the perpetrator or perpetrators went on a rampage and are still on the run.
The president of the German parliament, Baerbel Bas, who is from Duisburg, expressed shock over the attack.
"Terrible. My thoughts are with the injured," Bas tweeted.
"I hope that the emergency services will soon be able to clear up the threatening situation."
with AP
9°C