His campaign manager, David Garth, came up with a slogan that helped Koch beat fellow Democrat Mario Cuomo, who many commentators viewed as the more dynamic character, and Republican Roy Goodman.
''After eight years of charisma and four years of the clubhouse, why not try competence?" was a slogan that spoke to New Yorkers who were disappointed by Koch's predecessors, John Lindsay and Abe Beame.
Koch was a popular mayor -- winning a second term with 75% of the vote and a third with 78% -- but as the Times put it: "With New Yorkers wearying of his in-your-face shtick and seeking a balm to racial polarization, Mr. Koch was defeated for the Democratic nomination by Manhattan Borough President David N. Dinkins."
Before he was defeated by Dinkins, he criticized the Rev. Jesse Jackson, a candidate for president in 1988, and some felt he angered many black voters. Race relations in New York were also fractured at the time, especially after a 1986 incident in Howard Beach when white teenagers attacked three black men, killing one.
Koch's third term was beset by corruption scandals involving his political allies. Koch himself was never directly tied to wrongdoing, but the scandals hurt Koch's image with voters.
Only three New York mayors were ever re-elected twice -- Fiorello LaGuardia and Robert Wagner were the others -- and all three left office, as The New York Times put it in 2008, "drained, diminished and disdained."
Some new Yorkers thought Koch, who published an autobiography in 1984, had lost control of his ego.
Koch even said he lost because "voters got tired of me."
Koch, who never married, was often criticized by playwright, novelist and LGBT rights advocate Larry Kramer for not doing more to stop the spread of AIDS in New York.
"He was a closeted gay man, and he did not want in any way to be associated with this," Kramer declared to New York magazine.
Koch found discussions of his sexuality to be humorous.
"Listen, there's no question that some New Yorkers think I'm gay, and voted for me nevertheless. The vast majority don't care, and others don't think I am. And I don't give a (expletive) either way!" he told New York magazine.
There will be a funeral on Monday.