LAKE CITY, Fla. -

Police arrested a man Sunday on suspicion of the murder of a 36-year-old Lake City woman.

According to officials, police were able to locate Carlis Lindsey, 31, due to a stabbing that occurred earlier in the day.

When officers went to Lindsey's residence at 378 Northeast Double Run Road they found the house next door had the door kicked in and saw blinds moving.

After police ordered Lindsey to surrender, he came out of the house. When the house was checked out by police they found prescription bottles and pills scattered on the floor.

After arriving to the police station, Lindsey passed out and was sent to North Florida Regional Medical Center for treatment.

Chaquasha Aviger Police began searching for Lindsey after his girlfriend, Chaquasha Avinger (pictured right), a mother of three, was stabbed 20 times a couple of blocks from their home on Jackson Street. Her body was left in the street, where she died.

According to court records, Lindsey has an extensive criminal record, with prior arrests for aggravated assault on a pregnant woman, false imprisonment and drug possession. Just last year, Avinger filed an injunction against him for dating violence.

The news of Lindsey's arrest was what Sanshea Green, Avinger's cousin, had been praying for.

"We are so relieved that they caught him. I can't say how much rejoicing there is. We just wanted to see justice done," Green said.

Even though her cousin is gone, Green said she still feels Avinger's presence.

"We had a candlelight vigil for her yesterday" Green said. "With the wind blowing, you would think that the candles was out and the candles are still lit. She is here with us. She is actually here with us."

Green said now that the family has peace of mind, their focus is honoring Avinger's memory and taking care of her three children. She also made one final plea to women everywhere.

"If you're in a domestic violence relationship, please don't be scared to get out of it, because you never know what the outcome will be. This is to all young women that's out there. Get some help."

A VyStar account has been set up for Avinger's three children for those want to make donations. The account is in Avinger's name.