"But when and at what level, that's an entirely different matter," he said, according to RIA Novosti.
Brahimi told reporters that the plight of displaced people reflects the misery that exists throughout Syria. Since the civil war began last year, more than 540,000 refugees have spilled into nearby countries such as Jordan, Turkey, Iraq, Lebanon and Egypt.
"When we say there are  million internally displaced, what does it mean? It means that there are children and women who have nowhere to sleep, nothing to eat and are roaming the streets. We should thank their Syrian brethren who, in many cases, share their bread with them. These 4 million people are in need for bread and medicine," he said.
"You must have seen on TV boys breaking their desks in school to bring some wood for their families for heat. This is the meaning of what is happening in Syria."
With host countries strained and winter settling in on refugee camps, the crisis is expected to worsen.
The U.N. humanitarian affairs office says the number of refugees is expected to double next year to 1.1 million.
"These figures insufficiently express the suffering of the Syrian people and the injustice they suffer from," Brahimi said. "And, therefore, those who can have mercy on the Syrian people should do so. This is what we should not forget. We do not speak in a vacuum or about theoretical things; we are speaking about humans of flesh and blood, men, women, elderly, young children."
Plans have firmed for the movement of two Patriot batteries to Turkey, with German army officials saying they will begin the transportation by ship from northern Germany on January 7 or 8, Turkey's state-run TRT news agency reported. Some 170 German troops will be flown to Turkey.
The batteries are expected to be operational in February.
The transfer was ordered after NATO accused Syria of having fired Scud missiles near Turkey's border with Syria in an attempt to quash rebel gains.
Though the missiles have not hit Turkey, the development highlighted the need for a protection plan for the neighboring nation, Anders Fogh Rasmussen, the secretary-general of the alliance, said this month.
Analysts say the Syrian government maintains up to 400 of the short- and medium-range Russian-developed Scud missiles.
"The insert has an exclusively defensive character and not the purpose of establishing a no-fly zone, but the de-escalation of the border," the German army website, Bundeswehr, said.
The number of deaths in Syria rose Thursday by 197, including 103 in Damascus and its suburbs and 50 others found in a refrigerated truck on the Damascus-Daraa highway, the Local Coordination Committees of Syria said. The 50 had been "field executed by regime forces," the opposition group said.
CNN cannot independently verify opposition and government reports from Syria, as the government has restricted international journalists' access.