Eh, there are at least few levels from sex chromosomes to fully grown human. Hyperfocusing on XY chromosomes isn't something even scientists do.
Currently, we have something like this. The binary choice of presence or absence of SRY region (which is usually on Y chromosome, but not always) is the lowest level. SRY region starts a cascade of various gene expressions that in turn produce gonads which, when developed, introduce another level of indirection. Then, those gonads produce hormones which act specifically on various tissues in different ways. Hence, we call it "sex-biasing factors". Why am I speaking about it that way?
Because sometimes chromosome level and hormone level won't work synergistically. For example, in mouse models it is shown that two copies of X chromosome increase the body weight of female mice, but estrogen reduces it. In XO mice, the body weight is lowwer, because there is no second X chromosome.
And we didn't even start to talk about how hormones influence various organs, like the brain. I will cite, because it's bit complicated to write
>For example, structural MRI of brains has been applied to a population of humans with variations in effectiveness of the androgen receptor (AR), caused by different lengths of CAG repeats encoding polyglutamine tracts in the AR gene (Raznahan et al., 2010). In males, shorter CAG repeats (causing more effective function of the AR) were found to be associated with more masculine adolescent developmental changes in cortical thickness of the inferior parietal lobule in males, suggesting that some sex differences in the cortex are caused by greater action of androgens in males. Moreover, in females more effective variants of the AR were associated with more masculine patterns of adolescent change in the inferior frontal gyrus, suggesting that androgen action in females may account for some phenotypic variations among females. Here's a doi in case someone is interested:
https://doi.org/10.1016/j.yhbeh.2020.104691To finish my spiel, sex is not chromosomal in humans, because of the above. Also, if we really decouple sex from culture, then we will come to the conclusion that female sex gets pregnant. If someone who is "of female sex" (culturally) can not get pregnant, they are not a real female. This is what purely functional understanding of sex comes down to.