Ashwagandha is a herb that is traditionally used as a 'male supplement' and touted to increase vitality and virility of men, and second to that it has been at times recommended as a testosterone booster. A common inquiry is that, if it only benefits men, then why?
It seems that ashwagandha can influence the testicles, and by doing so promote fertility (at least in men that have abnormally low fertility and sperm quality); this has been demonstrated in a few studies and it isn't known if it increases semen quality of men who are otherwise fertile. In the men who do see benefits to fertility, however, there is an increase in testosterone which is known to benefit one's mood when it is low and then normalized.
While women do indeed have testosterone and can gain benefits through it, this particular mechanism of ashwagandha may require testicles to work and thus provide a difference in benefits of the herb based on sexual organs.
While it does have benefits for men, ashwagandha does affect women as well.
While there are no studies on whether or not ashwagandha affects female fertility (nor any convincing reason why it should) the benefits to anxiety, stress, and cholesterol have been found in studies where men and women were both enrolled. Furthermore, while the exact mechanisms of ashwagandha are not fully known, it is shown that they do not only work via the testicles but can influence the organs that both sexes share.