Marti, for one, asked himself why he didn't question why Castro -- who, he thought, lived alone -- would return each day with bags of McDonald's food, or who would watch the little girl he occasionally took outside. He also recalled how Castro seemed to steer him away from the house when they talked: "Now that I think of it, he didn't want nobody back there."
Another neighbor, Israel Lugo, said saw Castro at the park Sunday with a little girl and asked who she was: "He said it was his girlfriend's daughter."
Lugo said his sister got a bad vibe from the house and asked him not to let the children play unsupervised nearby. He said he heard yelling in the house in November 2011 and called police to investigate, but they left after no one answered the door.
And Nina Samoylicz, who lives nearby, said she called police about two years ago after spotting a naked woman in the backyard of Castro's house. Samoylicz said when she called out to the woman, a man told the woman to get in the house, then ran in himself.
"She was just walking around and naked," Samoylicz said. "We thought that was weird. We thought it was funny at first, and then we thought that was weird, so we called the cops. They thought we was playing, joking, they didn't believe us."
She said she had also seen tarps covering the backyard.
But Sgt. Sammy Morris, a Cleveland police spokesman, told CNN that the department had no record of a 911 call reporting a naked woman at Castro's address.
In fact, authorities never had any indications that the women were being held in the home or that anything suspicious was going on there, Cleveland Public Safety Director Martin Flask said. Neighbors had not provided any tips, he added.
Police had visited the home twice, authorities said Tuesday, once after Castro called about a fight in the street and another time to investigate Castro on an unrelated incident involving a child who had been left on a school bus.
The 2004 incident was the first of four exhibitions of "bad judgment" that led to Castro's November firing by Cleveland's school district, according to records released Tuesday night.
"He previously had been suspended for 60 days for leaving a child on a bus; 60 days for making an illegal U-turn in rush hour traffic with a bus load of students, and last school year for using the bus to do his grocery shopping," the letter recommending his dismissal states. His firing came after he had left his bus unattended outside a school after his preschool routes had been canceled, without notifying his dispatcher or depot.
Tito DeJesus, a bandmate of Castro's, said he had been inside the bass player's home once, about two years ago, to help deliver a washer and dryer he'd sold to the suspect and saw "a normal environment." DeJesus said he isn't related to the rescued Gina DeJesus but had known the family for years.
"It didn't seem to be a place where women were being held against their will," he said. "Of course, mind you, I didn't go throughout the entire house. I was just at the beginning of the house, in the living room, but it seemed normal."
Berry was last seen after finishing her shift at a Burger King in Cleveland on April 21, 2003. It was the eve of her 17th birthday. DeJesus disappeared nearly a year later, on April 2, 2004. She was 14.
Michelle Knight vanished on August 22, 2002, and her family reported her missing the next day, Flask said. She was 21.
Little was known about Knight's case Tuesday. Her mother now lives in Naples, Florida, and was contacted by Cleveland police late Monday, a neighbor, Sheldon Gofberg, told CNN.
The three women and the child were released Tuesday from the hospital where they had been taken for evaluations, a spokeswoman said. Tomba said all four appeared to be in good condition, if in need of a good meal.
While amazing, such discoveries are more common now, said John D. Ryan, CEO of the National Center for Missing & Exploited Children.
"To us at the National Center, this is not something that we find shocking any more," he said. "The fact is, we have seen more and more long-term missing cases end up in the victim being rescued many years after their original abduction."