Mutatis Mutandis

Below, in my post on Scientific vs Normative Truth, I wrote:
"[O]ne might think that the comparison here is unfair. We're comparing physical and normative truths by how they vary across time and space, which are physical dimensions."

The worry being that you shouldn't compare scientific and normative truths by how they relate to other factors which are themselves part of the scientific of normative domain.

In Humberstone's paper Direction of Fit, similar issues arise. He argues that Smith's account of direction of fit is objectionable since it contrasts beliefs and desire in virtue of how they relate to other beliefs:
"[Y]ou cannot informatively characterize a fundamental disanalogy between the ways in which beliefs and desires relate to their objects by contrasting them in a respect itself specified by reference to one of those two ways. It's as if one were to suggest that there is the following deep asymmetry between men and women as regards sexuality: whereas a heterosexual man will not be sexually attracted to males, a heterosexual woman will be."
(p64)

This all seems correct, but I'm not sure I could actually specify the force of these objections. Why does the presence of such a "failure to mutate" mean that something false has been said? One can imagine an objector: "Surely it really is a difference between heterosexual men and women that the latter but not the former are attracted to men?"

As I say, I tend to think that this kind of argumentative strategy is effective. But I can't work out why. Can anyone help me? (Bonus points if you manage to express it in logical notation.)