The author of the scariest haunted house novel ever written and the most controversial short story The New Yorker ever published was notorious in her lifetime but forgotten shortly after her death. Now, Shirley Jackson is beginning to receive the kind of acclaim she’s always deserved

Readers of the New Yorker weren’t prepared for what they were about to read when they opened the 26 June, 1948 edition of the magazine. Nestled within its pages was a short story – just over 3,000 words long – that would prompt them to write, in their droves, to demand an explanation. What did it mean? Was it true? And who did this Shirley Jackson think she was, anyway?

The story that caused all the controversy was The Lottery. Set in an unnamed village somewhere in America, it depicts a community carrying out a time-honoured ritual. Once a year, on 27 June, all the townspeople come together to draw lots from a battered old box. Whoever picks the paper marked with a black spot is that year’s sacrifice; the one person chosen to be stoned to death by their friends and neighbours in front of the whole town.

The letters began pouring in to the New Yorker’s offices within days of the story’s publication. Some readers were outraged enough to want to cancel their subscriptions immediately; others wanted to know the name of the town where the lotteries were held, so they could go and watch. Jackson would later claim that she received over 300 letters about The Lottery, only thirteen of which were complimentary.

It wasn’t the first story the then 32-year-old author had had published, but it was the one that would come to define her reputation. Even now, nearly seventy years later, it’s shocking – both in its subject matter, and in its presentation, because Jackson’s prose is so precise, so efficient, you really could almost believe it is non-fiction. Now, though, it’s not regarded as subscription-cancellingly gruesome – it’s regularly taught in US high schools.

“I don’t know how old I was the first time I read it on my own, but I definitely encountered it in high school,” remembers F. Brett Cox, one of the founders of the annual Shirley Jackson Awards for horror literature. “The teacher tried to reinforce things by holding a lottery. No rocks involved! But we each drew lots, like in the story, and one of the lots had a black spot on it. Everyone else would crumple up their pieces of paper and throw them at whoever got the black spot.”

Sensibly, Cox’s teacher rigged his lottery so that he drew the black spot. That meant the teens in his class could vent their pent-up frustrations on their teacher, rather than taking it out on one another. As a literature professor himself now, Cox has often taught the story to his own students, and even without the theatrics, there’s plenty to discuss: it’s a story about rituals, about outdated tradition, and about the darkness that can lurk, unexamined, beneath even the most picturesque surface.

That darkness, the sense of something being slightly askew somewhere, permeates Jackson’s work. The Lottery might be her best known story in the US, but on this side of the Atlantic, she’s probably better known for her 1959 novel The Haunting Of Hill House.

Even if you’ve never read it, you’ll recognise the plot: a professor sets out to investigate a notorious haunted house, with the help of a couple of psychics and the house’s young heir, and things get creepy, fast. It’s been adapted for the screen twice, first as 1963’s black and white classic The Haunting (directed by Robert Wise) and then as 1999’s not-quite-so-classic The Haunting (directed by Jan Du Bont). And it’s also the template for any number of other horrors, from Richard Matheson’s Hell House to Hammer’s 2014 chiller The Quiet Ones.

But no matter how many ghost stories may have come after it, Jackson’s novel remains the most spine-tingling. Even Stephen King, a writer who’s given plenty of people nightmares himself, finds it chilling; he called it “as nearly perfect a haunted house tale as I have ever read.”

Reading The Haunting Of Hill House is one of those experiences you never forget. It’s obvious from the novel’s opening lines that you’re in the presence of a master storyteller, one who has an utterly unshakeable grasp on her characters, her plot, and her atmosphere, and who never wastes a single word. And while, yes, it’s scary, it’s not just the constant threat of ghosts that makes it so spine-tingling. The Haunting Of Hill House is a sensitive, thoughtful, heartbreaking character study. The story unfolds from the perspective of one of the ghost hunters, Eleanor, a young woman who’s been plagued by apparently supernatural occurrences for most of her life. Hill House doesn’t just scare her, it also beguiles her, offering her a twisted version of the home she’s always wanted. It’s Eleanor’s slow unravelling that makes the novel so compelling.

Like bewildered New Yorker readers desperate for answers, it’s tempting to search for explanations of Jackson’s work in her life. Born in 1916 to well-off and socially conservative parents, Jackson was always something of a black sheep – a creative eccentric, rather than a glamorous socialite. Her mother, Geraldine, was always her harshest critic, even when Jackson had become a successful writer. And while Jackson’s marriage had a fairy-tale beginning – her husband, Stanley Edgar Hyman, had read one of Jackson’s stories in their university magazine and decided to marry her on the spot – it had plenty of problems, too. In the last years of her life, Jackson suffered from agoraphobia that left her housebound for months on end.

“She writes about wrongness and outsiders, and she herself was an outsider,” says Cox. “She spent most of her adult life in [the small town of] Bennington, Vermont, but she was born in San Francisco, and she had lived in New York. So there she is, this odd person writing odd stories. In the context of her day, she was someone from a very mainstream American family who was married to a Jewish intellectual. It’s sad, but that set her apart as well.”

While it might be possible to read genuine angst into the sufferings of her characters, though, that doesn’t really do anything to explain Jackson’s brilliance. There’s a magic about her writing that’s hard to define – her own words are always far more compelling than anything anyone can write about her.

Speaking of magic, many Jackson fans will point to her final novel, We Have Always Lived In The Castle, as her masterwork. Published in 1962, it’s strange sort of mystery story about the Blackwood family – Merricat, her sister Constance, and their uncle Julian – who live in a crumbling mansion on the outskirts of small American town. Six years earlier, the rest of the family were murdered: poisoned by arsenic in the sugar bowl. While Constance was suspected at first, she was acquitted, but the townspeople still think of her as a murderer, and her agoraphobia effectively keeps her a prisoner in her own home. Meanwhile, Merricat, the story’s narrator, imagines taking violent revenge on the village, and uses her own kind of ritual magic to try and protect what’s left of her family.

The novel is a little over 200 pages long, but every sentence is rich with symbolism, every word carefully selected to have the greatest possible impact. Though her earlier works had received mixed reviews, We Have Always Lived In The Castle was critically acclaimed. It was a commercial success, too, flying off the shelves so fast that its publisher, Viking, had to double the size of its second printing. So why isn’t it being taught in schools? Why haven’t there been multiple films made of it? Well, possibly because Jackson died in 1965, just a few short years after its publication.

“When Jackson died, she was at the peak of her fame,” explains biographer Ruth Franklin, whose gloriously comprehensive book Shirley Jackson: A Rather Haunted Life was published last year. “Her obituary ran in every major newspaper and magazine. I think it was her early death – she was only 48 – as well as her husband’s, only a few years later, that led to her decline in recognition. Many of her books went out of print.”

In the introduction to the anthology Let Me Tell You, Franklin tells a story that illustrates how frustratingly under-appreciated Jackson has been. In 1966, the Library Of Congress wrote to Hyman, then an eminent academic and critic, to ask if he’d like to donate his manuscripts and notes. Only as an aside did the letter mention that the Library would also be interested in Jackson’s papers.

From a modern perspective, it’s easy to blame Jackson’s fall into relative obscurity on sexism. She wasn’t just a woman writing at a time when women’s writing wasn’t taken seriously; she also dared to write in multiple genres – including memoir, a type of writing that’s often dismissed as less worthwhile than other genres.

“People are often surprised to discover that, as well as her dark fiction, she wrote light, humorous reflections about her family life with her husband and four children, which she published in women’s magazines,” says Jessica Harrison, Jackson’s editor at Penguin Classics. “She wrote one of the most searing stories of the twentieth century with The Lottery, and yet still had to spend much of her day doing housework. There’s a fascinating tension in her work between those two roles.”

That tension was what sparked Franklin’s interest in the writer, too. “I’ve always loved Shirley Jackson’s writing, especially The Haunting Of Hill House, one of my all-time favourite novels,” she says. “But I actually became interested in writing her biography after reading her household memoirs, Life Among The Savages and Raising Demons, which describe her adventures and misadventures in family life. I was fascinated not only that the same person could write in such different styles – Gothic suspense versus humour – but also that Jackson was an American woman in the 1950s who clearly felt pressure to be a perfect housewife and mother. Nevertheless, she had a creative genius that she couldn’t suppress.”

Jackson’s creative genius is still being uncovered. The current resurgence of interest in the author can be traced back to the mid-1990s, when Laurence Jackson Hyman found a scruffy old crate on his front porch. Inside were stacks of his mother’s notes and manuscripts – including several unpublished stories – all typed on Jackson’s signature yellow paper.

The contents of that box led Laurence and his siblings to track down several of Jackson’s other uncollected works, which were published in the collection Just An Ordinary Day in 1998. Other anthologies of Jackson’s unpublished work followed, and Penguin Classics published new editions of all of her novels and short story collections. Fast forward to 2017, and Ruth Franklin’s biography has won awards by the handful, Netflix is working on a Hill House adaptation with Mike Flanagan, and a film of We Have Always Lived In The Castle, directed by Stacie Passon and starring Taissa Farmiga, Alexandra Daddario, Sebastian Stan, and Crispin Glover, is currently in post-production.

Without that shabby old box, maybe none of it would have happened. But in an enjoyably Jacksonesque creepy twist, the box didn’t have a return address on it, so no-one knows where it came from, or who sent it.

If the mysterious archivist ever came forward, though, there’d be plenty of people lining up to shake his or her hand. There are a lot of Shirley Jackson fans out there, and they tend to be super devoted. “I get a lot of emails from authors and publishers, and I think we’ve become a kind of lightning rod for people to declare ‘hey, I really love Shirley Jackson!’” says JoAnn Cox, administrator of the Shirley Jackson Awards.

Researching this feature led to lots of similarly enthusiastic conversations. “Shirley Jackson pretended genre boundaries and genre conventions weren’t even there. She reinvented herself and her craft with every book,” says MR Carey, author of The Girl With All The Gifts.

Joanne Harris, author of The Gospel Of Loki and Runemarks, is also a fan: “Her handling of the mundane, and its closeness to horror, is very impressive, and shows us how subtle the literature of unease can be,” she says, while Jeffrey Cranor, co-creator of Welcome To Night Vale, raves about “her ability to make quotidian life unsettling. There are so few obvious devices or jump scares in her stories. They’re immersive in their ability to capture a certain recognisable Americana, and twist it ever so slightly, not just once in a shocking jerk but slowly, slowly. You don’t really ever notice it happening, until you do.”

Barry Hyman, Jackson’s youngest son, says he’s definitely noticed an increase in attention paid to his mother recently. “All my life, I have been aware of the impact my mother’s writing has had on people,” he recalls. “For more than fifty years, fans have approached me eagerly to say how important her works were for them, so I have always known that there was a devoted core group of readers who respected her talents and abilities as much as I do. What has changed in recent years is that the sheer number of her fans has grown exponentially.”

Jackson’s profile might be higher right now than it’s been at any time since her death, and the momentum is still building. Even in North Bennington, the town where Jackson seemed to feel like such an outsider, appreciation of her work is quietly growing. “About a dozen years ago, as a resident of North Bennington, I realised that not much had been done with Shirley Jackson, and that it was time for her to receive her due,” explains writer and curator Tom Fels. “The result was Shirley Jackson Day, which I’ve produced each year since in co-operation with Jackson’s four children, all schoolmates of mine from earlier days.”

Held on the Saturday closest to Lottery Day (27 June), Shirley Jackson Day doesn’t involve any ritual sacrifices – instead, it features readings of her works by her children, and by appreciative fans. “Shirley Jackson Day has proven to be popular with the local community, both those who appreciate her and some who are just learning about who she was,” says Fels. “It also attracts wider attention; sometimes people show up from afar, I have no idea how they hear of it. I think Jackson has a network of her own.”

If she does, it’s about to get bigger. Next year’s Shirley Jackson Day is probably going to need a much larger venue.