He did not say you cheated on your wife but just correlating the two. Saying that since all your friends are cheating on their wives it makes it all right for you to do it. It is like if your kids come home and say they got into trouble for cheating on a test but say that everyone did it. Just because others does it does not make it alright to do.
Too bad he's not bright enough to get that.
Cheating on tests is a "fact of life" now so I guess he condones his kids doing it because that's what "adults" do.