Women, however, should remember that sex is usually the primary way men feel connected and bonded.

"It is not because they don't care about you and only care about sex," she said. "For most men, those two are intertwined."

Ultimately, being open, honest and communicating with each other can not only lead to a better sex life, but can help avoid any underlying rejection or resentment.

"Because being sexual is such a place of vulnerability, it can be a place of deep sensitivity," Lipthrott said.

One thing to keep in mind is that every relationship is different. If you have tried to work on the relationship and are still having problems, both experts recommend speaking to a doctor.