The men may logically know that an offsite colleague is working lots of hours, but it doesn’t feel the same for one reason: they don’t see the person as sharing the same pain.
The men I interviewed often mentioned a sports analogy: in the heat of the summer, football teammates would bond during sweltering two-a-day practices, getting in shape to win their games. Never would one teammate say “Coach, I don’t have to practice with the others, because I can get in shape on my own time over here in my air-conditioned gym.” He might technically be correct, but his teammates would not look at him the same because—you guessed it—he didn’t share the same pain.