The roof caved in at the reception area of the Charlotte Maxeke state hospital as construction work was under way to seal a leak, officials say.
At least five people have been treated for minor injuries, including patients.
Rain had hampered rescue work at the hospital, one of the country's biggest.
One worker told the BBC: "I heard people screaming, saying: 'There are people under the rubble'."
Another said: "There was a noise, I saw rubble and dust everywhere. I don't know how many people are under there."
Stones had been moved onto the section of the roof that later collapsed, the main opposition Democratic Alliance party said.
But a statement from the provincial department of health in Gauteng province said the cause of the collapse was still unknown.
South Africa's Health Minister Aaron Motsoaledi says the hospital is expected to reopen on Friday.