“Sex” and “gender” bear disparate meanings but many people accidentally conflate these words. In the technical distinction, “sex” is a neutral word referring solely to physiology including differences in brain chemistry and hormones, whereas “gender” refers to cultural constructions/assumptions about what personality traits best suit (or are innately connected to) a particular biological anatomy. The female Ants all seem to identify as female, meaning their gender is female, regardless of 1) whether or not they can reproduce and 2) whether or not they have any interest in reproduction of any sort. It is not clear whether or not they subscribe to a belief in gender essentialism and, even if they did, whether their brand of gender essentialism would include reproduction as the main, or even as a main, component of what "female" means to them.
The Queen and the other feminine Ants were all given female seiyuu (voice actors) and they speak in feminine Japanese. The Queen especially speaks in a very motherly, or perhaps, grandmotherly, fashion.
Hina (formerly known as Hirin) has a nen ability to remove curses that gives her the appearance of being pregnant. She does not comment on this visual appearance in regard to human pregnancy, but only to complain about her figure.
(As an aside, the feminine Ants do seem to roughly align with modern Western gender norms for females, but Japanese gender norms have historically differed significantly from Western ones. For example, the 11th century Heian era ideal man was a master of concocting perfume, wearing make-up, and penning love poetry, and in the present day you can often see young straight men styling each others’ hair in public, wearing numerous plushie Disney key chains dangling from their man purses, and dressed in pink and lavender — none of these practices are non-masculine in contemporary Japanese culture — whereas the culture does maintain extremely strict gender norms, copious self-segregation by gender, and rampant sexism.)
The fact that a Chimera Ant is female is not dependent on the prior human and/or animal who was/were eaten by the Queen also having been female. Many of them are an amalgamation from a number of humans/animals who were killed, which were likely a mix of males and females. The main illustration of this is Kite, who was a human male, was killed and apparently at least some part of him was eaten by the Queen (though much of his body was not eaten in order for Pitou to make her puppet), then was birthed prematurely as a female Ant, and insisted that her name was Kite from a young age. The characters specifically mention that the Ant Kite is a girl. Her appearance came from a random female villager killed by Koala, meaning it is likely that that girl's entire body was eaten by the Queen, but the resulting Ant retains the memories of Kite instead, even though only a small portion of Kite's body was eaten.