By a man-sized margin, my favourite recent news story is the one about two feminists who went to a men-only swimming session in Dulwich, south London, because, as they explained, they now self-identify as men. An elderly gentleman was initially confused. “I told him I was a man and he said, ‘Oh really?’” one woman later told reporters. “It was a very British response.” Other men at the pool were less sanguine, and complained to reception.

This protest was announced on Mumsnet, a pleasing hotbed of radical feminism these days, as part of a campaign against proposed changes to the Gender Recognition Act. Currently, anyone who wants to change gender needs to have lived in their chosen gender for two years and been diagnosed with gender dysphoria. If the changes go through, anyone will be able to declare they are a man or woman, regardless of whether they have made any actual changes to their lifestyle or body. This is known as “self-identification” and the reactions have borne out that Margaret Atwood line, “Men are afraid women will laugh at them while women are afraid men will kill them.” Men have largely ignored the issue, until it comes charging into their changing room, while a lot of women have argued that predatory men could now come into female-only spaces unchallenged.

You might have thought that the #MeToo campaign, in which women have been speaking out about the universality of sexual assault and rape, would make people more sympathetic to concerns about female safety. You would be wrong: nothing makes you look more liberal these days than shouting at women who express anxiety based on their experiences.

But then, as with experts, apparently we’ve all had enough of lived experience now. When a 19-year-old trans woman was elected a Labour woman’s officer last year, a Labour councillor explained that “lived experience as a woman” was not a pre-requisite to be a woman’s officer. Biology, too, has been deemed terribly passe. “Inclusive feminism,” Plaid Cymru’s Leanne Wood wrote when considering why self-identifying trans women should be allowed into women’s refuges, understands that “gender is a complex and deeply personal thing, and is about so much more than outdated ideas of biology.” On the day of this year’s Women’s March, trans model Munroe Bergdorf tweeted that to “center reproductive systems” at the demonstrations was “reductive and exclusionary”.

I’m trying to think of anything more patriarchal than telling women to stop fussing about vaginas at a Women’s March. A biopic about the Old Testament God starring Mike Pence? Because none of this is about making feminism inclusive; it’s about policing the way women talk about their lives. No one – male, female, trans or not – has the right to do this. Inclusivity means campaigning for the greater good of the group, not catering to each individual’s differences – can someone please tell Bergdorf that Irish women are still fighting to repeal the 8th? Female biology has been used by men to oppress women for millennia, and to tell women not to talk about it now is another form of oppression.

Women have had to fight so hard for a place at the table, for the right to define themselves, for spaces where they feel safe. Any man who sneers at them now for worrying about the shifting paradigms, offering only meaningless platitudes or accusations of bigotry, is showing his male privilege.

There is understandable concern about being on the wrong side of history. But I’ll tell you what has never put anyone on the right side of history: shouting women down. Gender is a feeling and biology is a physical fact, and the reason women-only spaces exist is not to protect some special inner feminine essence, but because there are significant physical differences between male-born bodies and female-born ones, and the latter have long been at a disadvantage.

This is something women and trans women will have to work out between themselves, because this is a woman’s rights issue. Any man, who has no idea what it’s like to be oppressed as a woman or self-identified trans woman, and who comes riding in on a white horse and tries to shut down the debate with inflammatory language, is not helping. There is no simple solution to accommodating both the rights of self-identified trans women and other women. In some cases, they’ll share spaces, in some they won’t, and sometimes a third option will have to be found.

Contrary to what some men think, the feelings of self-identified trans women are not the only ones that matter. There are a couple of four-letter words for those who insist otherwise. But “woke” ain’t one of them.