Widespread media reports have described Kelley as a liaison at MacDill Air Force Base in Tampa, Florida, where the U.S. Central Command is headquartered. Both Petraeus and Allen were previously stationed at the base. A Central Command spokesman said she is a volunteer with no official position.
Kelley has not responded publicly to the latest news.
On Monday, her brother, David Khawam, told CNN affiliate KYW-TV that she went to authorities because she was scared after receiving the e-mails. He described his sister as a dedicated mother and said it would be "completely uncharacteristic" for her to have an affair.
Both Allen and Petraeus appear to know Kelley's sister, Natalie Khawam. The men wrote letters in support of Khawam in a custody battle, court records show.
One of the sources familiar with Kelley said she first mentioned the alleged harassment in a casual conversation with an FBI agent she knew socially. She did not seek him out specifically for action on the matter, but was happy for the help, the source said. The source added that Kelley did not know at first that the e-mails led to Petraeus.
Earlier, amid national talk about the Petraeus scandal, Kelley, 37, and her husband released a statement saying they have been friends with Petraeus and his family for more than five years and asked for privacy.
The sprawling story continued to sprout new details Tuesday, including a revelation from a senior official close to Allen claiming that it was Allen who received an anonymous e-mail about Kelley, and tipped her off that someone was threatening her.
Other U.S. officials have said Kelley received harassing e-mails that concerned her and went to the FBI.
"There is no affair," the senior official said. "She is a bored rich socialite involved with every single senior commander at CENTCOM, because she worked as an honorary ambassador."
In another bizarre twist, a jogger running in Rock Creek Park in Washington on Sunday found Broadwell's North Carolina license, according to Lt. Bill Kellogg, spokesman for the Maryland-National Capitol Park Police. Police attempted to contact Broadwell, who lives with her husband and children in Charlotte, but were unsuccessful. Because her name has been in the news recently, they also reached out to the FBI.
Meanwhile, the FBI continues to look into the Petraeus affair amid congressional calls for an inquiry into why leaders were not notified of that matter sooner.
The Petraeus scandal also has raised questions about potential impacts on national security, including concerns that his paramour may have had access to his classified schedule and a New York Times report that she had classified documents on her laptop computer.
FBI agents were at Broadwell's Charlotte, North Carolina, home late Monday, said local FBI spokeswoman Shelley Lynch. She declined to say what the agents were doing there.
A source told CNN national security contributor Fran Townsend that Broadwell was acting as Petraeus' archivist, and that the FBI went to the house to look for any documents she might have. It was not clear whether any of the material was classified, the source said.
Also, a video has surfaced of a speech by Broadwell in which she suggested the Libya attack on September 11 was targeting a secret prison at the Benghazi consulate annex, raising unverified concerns about possible security leaks.
"I don't know if a lot of you have heard this, but the CIA annex had actually taken a couple of Libyan militia members prisoner and they think that the attack on the consulate was an effort to get these prisoners back," said Broadwell in a speech last month at the University of Denver.
A senior intelligence official told CNN on Monday, "These detention claims are categorically not true. Nobody was ever held at the annex before, during, or after the attacks."
Broadwell's source for that previously unpublished bit of information remains unclear, and there's no evidence so far that it came from Petraeus.
Administration officials have said the Benghazi assault was a terrorist attack.
Petraeus was scheduled to testify on Capitol Hill about the attack this week at closed-door hearings. Some Republicans have criticized the administration's response to the Benghazi attack and have speculated that the timing of Petraeus' departure was linked to the congressional inquiry.