(CNN) -

In ruling California's death penalty unconstitutional, a federal judge said Wednesday the system is so broken it unfairly leaves inmates with uncertain fates -- often for decades.

In his decision, Santa Ana-based Judge Cormac J. Carney vacated the 1995 death sentence of Ernest D. Jones, who petitioned the court to determine whether his death sentence was valid.

Carney wrote: "Allowing this system to continue to threaten Mr. Jones with the slight possibility of death, almost a generation after he was first sentenced, violates the Eighth Amendment's prohibition against cruel and unusual punishment. "

The decision will certainly be appealed, said CNN legal analysts Jeffrey Toobin and Mark Geragos. Toobin said no executions could proceed in California while the decision stands.

In California, there are 748 people on death row, the most of any state.

No executions have been carried out since California put a moratorium on the death penalty in 2006. Four years later, the state sought to execute a man who raped and murdered a 15-year-old girl, but it was blocked by state and federal courts over concerns about lethal injections.

In Wednesday's ruling, Carney called the administration of the state's death penalty system "dysfunctional."

"In California, the execution of a death sentence is so infrequent, and the delays preceding it so extraordinary, that the death penalty is deprived of any deterrent or retributive effect it might once have had," he wrote. "Such an outcome is antithetical to any civilized notion of just punishment."

Since 1978, more than 900 people in the state have been sentenced to death row, where inmates spend 23 hours alone in their cells. Of those people only 13 have been executed; 94 have died of other causes. Carney said about 40% have been on death row longer than 19 years.

"California's death penalty system is so plagued by inordinate and unpredictable delay that the death sentence is actually carried out against only a trivial few of those sentenced to death," Carney said, noting that it takes an average of three to five years for a death row inmate just to get an court-appointed attorney for an appeal.

The judge noted 285 of the people on death row have been there longer than Jones.

"There clearly was a crisis coming. A lot of people have recognized there is a problem with hundreds of people of death row and hardly any executions," Toobin said. He noted the situation in California is so unusual it wouldn't affect death penalty cases in other states.

California's most recent execution was on January 17, 2006, according to the website for the state's Department of Corrections & Rehabilitation.

Clarence Ray Allen, 76, spent 23 years and one month on death row before dying from a lethal injection. He was convicted of orchestrating a triple murder from prison in 1982.

Michael Laurence, executive director of the Habeas Corpus Resource Center and Jones's lead attorney, said he was glad that a federal judge reaffirmed what a state commission found in 2008.

"There is no rational explanation, much less any moral or societal justification, for which people are ultimately executed," Laurence said in a written statement. "The execution of Mr. Jones, and the others like him whose meritorious legal claims have gone unheard for decades, serves no valid state interest."

The executive director of Death Penalty Focus said the law in California needs to be changed.

"Justice requires that we end this charade once and for all," said Matt Cherry. "It's time to replace California's broken death penalty with life in prison without the possibility of parole. That's the best way to ensure that convicted killers remain behind bars until they die, without wasting tens of millions of tax dollars every year on needless appeals."

Jones' case is expected to be appealed by prosecutors to the U.S. Court of Appeals for the Ninth Circuit, said.

Nick Pacilio, a spokesman for state Attorney General Kamala Harris, said her office is reviewing the decision.