They say dozens of people are still missing since the landslide on Saturday night at the Koshe landfill. A resident said 150 people were there at the time.
A number of makeshift houses are now buried under tonnes of waste.
The area has been a dumping ground for Addis Ababa's rubbish for more than five decades.
A city spokeswoman told AP news agency that many children were among the dead.
There are fears the death toll could rise further.
Local resident Musa Suleiman Abdulah told AFP he heard "a big sound" and saw "something like a tornado... rushing to us" when the landslide occurred.
Tebeju Asres said that the family's house had been swallowed by the landslide.
"My mother and three of my sisters were there when the landslide happened. Now I don't know the fate of all of them," he told AP.
Hundreds of people attempt to make a living by scavenging at the landfill site, sifting through the rubbish for items they can sell, the BBC's Emmanuel Igunza in Addis Ababa reports.
Some people even resided at the rubbish dump permanently.
The authorities have been building Africa's first waste-to-energy plant near the landfill.
They plan to burn rubbish generated by the capital's estimated four million people and convert it into electricity.