One Flew Over the Cuckoo's Nest - the film that flew away with the 'big five' top Oscars

As Danny DeVito, Christopher Lloyd and co stare dead-eyed into the camera, Jack explains with a grin that he's been institutionalised, and is led off by Louise Fletcher, playing the fearsome Nurse Ratched. It's an amusing glimpse into the making of a very special film, one of only three movies that have won the 'big five' Oscars - Best Film, Best Director, Best Actor, Best Actress and Best Screenplay.

One Flew Over The Cuckoo's Nest is 40 year old this year, and is now regarded as a timeless classic. But while Jack Nicholson's BAFTA speech might suggest all was sweetness and light on the set, Cuckoo's Nest was marred by money problems, the disapproval of the story's creator, Ken Kesey, and a nasty power struggle between director Milos Forman and its star. They got on so badly it's a wonder the film ever got made, and its subsequent success took everyone by surprise.

The film's story began in 1962 when Kirk Douglas read an unpublished galley of Ken Kesey's original novel and fell in love with it. Cuckoo's Nest followed the adventures of Randle Patrick McMurphy, a rowdy and charismatic minor felon who energises the cowed inmates of a mental institution when he's transferred there from prison.

Douglas was enchanted by McMurphy's free spirit, and played him in a 1963 Broadway adaptation. He also planned to star in a movie version, and bought the film rights for $47,000. But he quickly found that the studios were not so keen on funding a movie set in an asylum. And getting the film off the ground took so long that Douglas became too old to star in it.

By the mid-1970s his son Michael had become involved in the project, and chose Czech filmmaker Milos Forman to direct it. The casting of Randle McMurphy proved divisive. Ken Kesey, who was hired to adapt his own book, wanted Gene Hackman, but both he and Marlon Brando turned the part down. The Douglas's were interested in Burt Reynolds, but when Nicholson's name came up, it instantly made sense to Michael Douglas.

"At that point," he remembered later, "Jack had done passive characters in Easy Rider and Five Easy Pieces, but when I saw him as the flamboyant but sensitive shore patrol-man in The Last Detail, I knew he could play the part".

Ken Kesey, however, was not thrilled about the casting of Nicholson, or the direction the production was taking. He was particularly miffed that Native American inmate Chief Bromden would not be the story's narrator, as in the novel, and just two weeks into shooting he left the production and later filed a law suit against the producers.

Many years later, the writer was watching television one night when he stumbled on the film and was enjoying it - until he realised what it was.

Michael Douglas, meanwhile, had been hard at work making sure his film stood every chance of becoming something special.

It was he who sorted out independent financing when the studios refused to back the production, and he who chose the Oregon State Hospital For The Insane as Cuckoo's Nest's primary location, because of its atmospherically gloomy late 19th century building, and because its managers offered the production unrestricted access to its facilities.

So much so, in fact, that the hospital's director, Dean Brooks, made an appearance in the film, playing the chief psychiatrist, Dr John Spivey. And though he'd never acted before, and never would again, Brooks did a fine job of coping with Jack Nicholson's constant ad-libbing in the scenes they shared. Many of the extras in the film were inmates, and others helped out behind the scenes. But things didn't always run smoothly, and on one occasion a patient jumped out a third floor window that had been left open by the crew, prompting a local newspaper to run the waggish headline, 'One Flew Out of the Cuckoo's Nest'.

Apart from Nicholson, and Louise Fletcher, who took on the unattractive role of Nurse Ratched, the rest of the cast were mainly unknowns. Danny DeVito had starred as jovial mental patient Martini in a 1971 off-Broadway production of Cuckoo's Nest, and reprised his role in the film. Christopher Lloyd and Brad Dourif also took key parts that would make their names. But they and Nicholson didn't immediately hit it off when the high-flying Hollywood star arrived on set, as Michael Douglas later explained.

"Dr Dean Brooks gave us tremendous cooperation and let our actors sit in on some of the patients' therapy sessions. Suddenly, our actors get serious and don't want to break character. So Jack arrives not knowing any of this, or who these actors were.

"I'm having lunch with Jack on his first shooting day in the cafeteria with the rest of the actors and inmates, when suddenly he pushes his tray aside and storms out of the building. I follow him outside where he's pacing in the courtyard having a smoke. Jack shouts at me: 'Who are these guys? Don't they even stop being in character for lunch?' Then I told Jack that some of the actors were real inmates. He thought about what I said, then laughed…"

The production took over an entire wing of the hospital and Nicholson spent two weeks preparing himself for the shoot, sitting in on therapy and getting to know many of the inmates by name. Despite his initial reservations about the rest of the cast, Nicholson got on famously with Lloyd, Dourif and DeVito. Jack became the ensemble's leader, and staged a kind of on-set coup after falling out with Milos Forman.

According to production designer Paul Sylbert, the two men disagreed over the scene where Randle McMurphy first enters the asylum. Forman had envisaged the patients being in uproar when McMurphy appears on the ward, but Nicholson argued that his character would have no impact on the other inmates if they were already up in arms. Both men refused to back down and a tense stalemate lasted several weeks.

A compromise was eventually reached, but Nicholson and Forman communicated through their cinematographer Haskell Wexler for the remainder of the shoot. Forman, though, had the good sense not to stand in the way of what would be an extraordinary central performance.

Nicholson delivered an extraordinary, live-wire portrayal of the mercurial and charismatic McMurphy that was both funny and moving. He ad-libbed constantly, my favourite example being the moment when he started a game of on-ward poker with the immortal phrase: "Which one of you nuts has got any guts?"

Randle McMurphy fatally misinterprets his situation in the asylum. He thinks it'll be an easy way of finishing off his prison sentence, and only realises when it's too late that he's been committed and can only be released on the say-so of the cruel and manipulative Nurse Ratched.

And after he stages a rebellion and attempts to escape with the help of Chief Bromden, things turn tragic when he's subjected to electric shock therapy, and ultimately lobotomised. Contrary to popular belief, Jack Nicholson did not actually undergo shock therapy during the shoot. But Cuckoo's Nest's depiction of the treatment as a painful and barbaric ordeal turned public opinion against electric shock therapy, much to the disgust of those psychiatrists who passionately believed in its beneficial effects.

As for Louise Fletcher, she did such a good job of portraying the chilling and manipulative Nurse Ratched that she struggled thereafter to shake off the character. And though she won the Best Actress Oscar in 1976, she was never offered as big a role again.

During the gruellingly intense three-month shoot, Nicholson lived on the asylum campus and totally immersed himself in his role. But he was visited at least once by his girlfriend Anjelica Huston, who made an uncredited appearance in the film, playing a woman on the pier when McMurphy and co return from their unscheduled fishing trip.

With its challenging themes, gloomy conclusion and troubling tone that veered wildly between tragedy and broad humour, Cuckoo's Nest did not seem like the kind of film that would dominate the awards season. But at the 48th Academy Awards on March 29, 1976, the film won five Oscars, including of course Best Actor for Nicholson. After striding up to collect the award, Jack told the audience: "I guess this proves there are as many nuts in the Academy as anywhere else."

But he was no nut. He'd taken a comparatively modest salary for a cut of Cuckoo's Nest's box office and it made him a very wealthy man. By 1978 the film had earned him a cool $15million, and left Jack free to choose whatever role he pleased.

What Jack did next

In the aftermath of Jack Nicholson's Oscar triumph in Cuckoo's Nest, scripts and offers came flooding his way. He was considered for the part of Travis Bickle in Taxi Driver, but wisely turned it down, and also said no to starring roles in everything from The Goodbye Girl and Caligula to Apocalypse Now.

In 1976 he co-starred with his idol Marlon Brando in Arthur Penn's eccentric western, The Missouri Breaks. And in 1978 directed himself in the comic western, Goin' South. In fact the few years after Cuckoo's Nest were a relaxed time for Jack, but in 1979 he bit the bullet and agreed to work with Stanley Kubrick on The Shining (left). It would prove a gruelling, exhausting shoot. And in the 1980s, he'd be drawn more to lucrative mainstream productions like Terms Of Endearment, The Witches Of Eastwick and Batman, as the former rebel became the ultimate Hollywood insider.