It primarily has to do with hormones, in particular prenatal hormones, which sexually differentiate the brain. Prenatal testosterone is neurologically associated with, as examples, both preferences for things vs. people-related interests, and gynephilia, in addition to many other masculinized mental qualities