More recently, Gerald Ford's 1976 loss to Jimmy Carter certainly had an economic element as inflation and recession mingled in "stagflation." But it was other issues -- including his pardon of Richard Nixon, a devastating debate gaffe over the Soviet Union, and the perception that Ford lacked legitimacy as an unelected president -- that were his undoing.
Legitimacy was also factor after Benjamin Harrison defeated incumbent President Grover Cleveland, the popular winner in 1888, by winning the electoral vote that year. It was a campaign tainted by widespread charges of election fraud. Four years later, Cleveland regained the White House by ousting Harrison, whose personality never won over those convinced that the Republicans had "stolen" the election.
Assuming that the economy is the overwhelmingly decisive element in presidential elections relies on the "fallacy of a single cause." That is a black-and-white thinking style born of a desire for a simple narrative that tidily explains everything, when the real story is always far more complicated.