Secretary of Defense Leon Panetta on Wednesday defended his request to withhold the nomination of Gen. John Allen to NATO commander pending an investigation into his contacts with Jill Kelley, whose complaints about anonymous, harassing e-mails led to the discovery of the affair between Petraeus, 60, and Broadwell, 40.

Broadwell's government security clearance has been suspended pending the outcome of investigations, two U.S. officials with direct knowledge of the move told CNN Wednesday.

Defense officials announced Tuesday that the FBI had referred information to them indicating Allen may have exchanged potentially inappropriate e-mails with Kelley, who was a volunteer at MacDill Air Force Base.

Kelley's access to MacDill without an escort has been suspended, a Defense Department official said Wednesday.

She had been given access to the base because of her position as a booster and promoter of programs to help U.S. troops, the official said.

A U.S. official familiar with the e-mails Allen sent to Kelley described them as warranting the investigation.

"If they got out, John Allen would be very embarrassed by them," said the official, who added that there was no evidence of physical contact between the two.

The official said that the e-mails under investigation are from Allen.

But a senior official close to Allen told CNN on Tuesday that the e-mails contained nothing pointing to sex or anything of a romantic nature. Allen may have said, informally, "thanks sweetheart" in an e-mail, the official close to Allen said.

"Anyone who knows him knows his style; he has a habit of replying to every single e-mail (he is sent). Kelley would e-mail his business and personal accounts," the official said.

It will be up to the Defense Department's inspector general to decide if the e-mails' content represents conduct unbecoming an officer, said a third source, a senior U.S. official.

Allen has yet to be questioned by Defense Department inspector general staff, but that could be completed in days, a U.S. official with knowledge of the investigation said.

Allen, who was once stationed at the base, has denied wrongdoing, a senior defense official said. In a statement, Col. John Baker, the chief defense counsel of the Marine Corps, said Allen "fully intends to cooperate" with the inspector general's investigators.

Broadwell's anonymous e-mail to Allen was sent after May, perhaps in June, the official said.

The e-mail, which had also been sent to a number of other officers, bore the handle "kelleypatrol -- or something similar," the official said.

He described the e-mail as "a warning that Kelley was a seductress or something along those lines" and said it was vaguely threatening, but above all weird.

"Allen did not know it was (from) Broadwell," the official said.

The official also said it was unclear when Kelley went to the FBI or whether Allen's warning to her was the trigger, but that Allen saw nothing in the e-mail's wording to warrant referral by him to the FBI.

Kelley's version differs from one offered by the senior official close to Allen, who said it was Allen who received an anonymous e-mail about Kelley, and tipped her off that someone was threatening her.

One of the sources familiar with Kelley said she first mentioned the alleged harassment in a casual conversation with an FBI agent whom she knew socially. She did not seek him out for action on the matter, but he was happy to help, the source said. The source added that Kelley did not know at first that the e-mails led to Petraeus.