Set in an Elizabethan England haunted by the threat of plague, Maggie O’Farrell’s unexpectedly timely novel about the death of Shakespeare’s son, Hamnet, will go up against Hilary Mantel’s The Mirror and the Light for this year’s Women’s prize for fiction.

The novels, which are among the biggest books of 2020, lead a heavyweight shortlist that will compete this autumn for the £30,000 prize. Bernadine Evaristo’s Booker prize-winning Girl, Woman, Other, that follows a cast of 12 characters, mainly black women, is also among the finalists, as are Jenny Offill’s Weather, an exploration of climate anxiety; Natalie Haynes’s A Thousand Ships, a retelling of the Trojan war from the perspectives of the women involved; and Angie Cruz’s Dominicana, about a teenage girl who is married off to an older man in order to get her family out of the Dominican Republic and into the US.

The judges picked the six-strong shortlist after a “long Zoom meeting” during which emotions ran high, said chair and entrepreneur Martha Lane Fox. “It’s a big year. There were a lot of extremely impressive books on the longlist. It was difficult. We’re all still friends but I’m not going to pretend, it was tense. We weren’t fighting, but passions were high – just as you would want in such a process.”

Longlisted books that didn’t make the cut include Ann Patchett’s The Dutch House, Jacqueline Woodson’s Red at the Bone, and Edna O’Brien’s Girl.

Hamnet, an account of the short life of Shakespeare’s son, who died at the age of 11, and the effect of his death on his mother, is “extraordinary”, said Lane Fox, who also called The Mirror and the Light, the bestselling conclusion to Mantel’s trilogy about the life of Thomas Cromwell, phenomenal. Mantel, who won the Booker twice for the two previous instalments, Wolf Hall and Bring Up the Bodies, has been shortlisted three times for the Women’s prize but never won.

Hamnet opens as its title character tries to find help for his twin sister Judith, who is lying in bed with what is feared to be plague. At the thought of lumps forming beneath her skin, “a cold fear rinses down through his chest, encasing his heart”.

“It’s quite on the money in terms of the harrowing experiences we’re currently going through,” said Lane Fox.

All of the books, she said, draw from “big, chunky” experiences: “identity, gender relationships, power, war, grief, the climate crisis, slavery, immigration”.

“These writers are not shying away from the topics we’re all facing over the next decade,” she said. “We are all living in challenging, sad and complex times, so incredible stories provide hope, a moment of escape and a point of connection now more than ever … We are all so proud of these books – all readers will find solace if they pick one up.”

The Women’s prize is intended to reward “excellence, originality and accessibility in writing by women in English from throughout the world”. It is marking its 25th anniversary this year with a new digital book club, #ReadingWomen, which will tackle each previous winner, from Helen Dunmore’s A Spell of Winter, which won in 1996, to An American Marriage by Tayari Jones, which won last year. A public vote this autumn will choose the “winner of winners”.

Lane Fox was joined on the judging panel by writer and activist Scarlett Curtis, co-founder of the Black British Business Awards Melanie Eusebe, author and comedian Viv Groskop and author Paula Hawkins.

This year’s winner’s ceremony has been postponed from June until 9 September due to the coronavirus.