[This article is subject to a legal complaint]

So there was this one time, back in the 1980s, when Alan Clark, the Tory MP, was having lunch with a glamorous bird at the Ivy. Nothing unusual about that. But the poor girl couldn’t hold his attention. Because Clark was distracted by an exotic creature with flowing locks on another table.

The Ivy is always full of celebs, of course, but this one was extraordinary. A vision in diamonds and furs. Not another woman, but Paul Raymond. The Paul Raymond of Raymond Revuebar – the world centre of erotic entertainment, as it said on the sign. King of Soho, nightclub impresario, property mogul. Clark couldn’t keep his eyes off him, allegedly. Is it true? Probably. But Raymond the man has long been lost to myth.

Twenty years later, this March, a small Catholic funeral took place in southwest London. Raymond had died on March 2 of a chest infection at the age of 82. It was a small affair. His son Howard was there, and his son-in-law, John James. His granddaughters Fawn and India Rose. But not their mother, John’s ex-wife and Raymond’s daughter, Debbie. She died in 1992 of a drugs overdose. Nor his first-born son, Derry. Very few other mourners were present.

The obituaries all told the same tale, of rags to riches via pornography and tragedy. Booze-ups in the penthouse, topless dancers and his famous epigram: “Sex always sells.” With a Howard Hughes-like epilogue: Multimillionaire recluse shuns the world outside and dies with untold wealth but friendless and alone. Like every legend, it contains a grain of truth. But it is not the whole story.

In the 1970s and ’80s, Raymond would glide around Soho in his Rolls-Royce with blacked-out windows and the personalised registration plate PR11. Doors would open as if he were a Hollywood celebrity. He would make a flamboyant entrance in his immaculate suit and man-bag. He was brazen, once offering the newsreader Anna Ford £75,000 to reveal all in his magazine Club International (she declined). “I like to look at nude birds and I think most normal men do,” he said.

By contrast, towards the end of his life Raymond rarely left his apartment. The great West End theatre producer Michael White recalls seeing him in Soho in his car two years ago; he went over to say hello. Raymond wound down the window and told him: “I can’t lunch. I don’t lunch any more.” And the window closed again. The heady days were long gone.

“I thought his ending was rather sad,” says the thong-loving nightclub owner Peter Stringfellow. “For a long time, he left his name up – Raymond Revuebar – and it got really tatty. Then eventually it was turned off. When he died, I thought it would have been a nice touch for someone to have turned it on. But no one did.”

Stringfellow and Raymond hadn’t been friends. They had met once in the 1990s during negotiations over a premises on Coventry Street. Raymond let Stringfellow have the place rent-free for a year, but Stringfellow had to bear heavy legal costs relating to planning permissions that he didn’t manage to obtain. At the end of the year he sued Raymond for not disclosing the planning issues prior to his entering the agreement. They settled out of court.

“It was no big deal,” says Stringfellow. “He did nothing wrong from a business point of view. I had great respect for him. He was fundamental in shaping Soho.” It’s easy to forget that when Raymond started buying up properties in Soho in the 1970s, this now-vibrant area still had its share of burnt-out shops and shuttered premises. Raymond’s investment was key in turning Soho into what it is now. “When he died I was astonished there weren’t more people standing up and waving flags for him,” says Stringfellow.

In fact, some of Paul Raymond’s closest friends and family do want to wave a flag for him. Nobody is saying he was a saint. Only that he should not be judged by his flaws alone. The fact that they have remained tight-lipped until now speaks volumes about the apprehension and awe that surrounded him, the control with which he ruled his empire and perhaps anyone in his circle.

What was this man’s private life like? Raymond had married Jean Bradley, a dancer, in 1951 and had two children with her, Howard and Deborah. Howard has inherited his father’s wariness, and rarely speaks out about him. One of his earliest memories is playing cricket with his father on the family lawn for a BBC documentary, when he was five or six. “It was hours before he could bowl a ball that hit the wicket.” The press presence seemed normal. As did the pet cheetah. “At six o’clock at night he’d put it in the back of his van and drive off to the Revuebar and put it in the show.”

Raymond could lose his temper, “blow his top like Vesuvius”, then carry on as if nothing had happened – mostly over business matters. He never hit his children, but nor was he affectionate. “He didn’t know how to express his love. He was never a huggy-huggy dad, because he didn’t have that love from his own parents. I was frightened to death of his mother – an old-fashioned Catholic, always wearing black.”

Howard was kicked out of his Catholic school when they found out who his father was. He was glad, though: “It was grim.” Boarding school at Henley was better. Even if his father did turn up at sports day in his Rolls-Royce, wearing a billowing shirt and a silk scarf as a belt. “I couldn’t dig a hole deep enough.” When he and a friend bunked off, Paul was cross – that Howard had been caught. “We’d walked past his flat at 3am and his lights were still on – he was probably having a party.” His father was “a bit of a lad” himself, you see.

Howard was 14 or 15 when his mother told him he had a half-brother, Derry. Ironically, whereas Paul wanted Howard to take over the business and Howard refused, causing a brief rift, Derry would have loved to but never had the chance. Derry McCarthy only met his father once. The received wisdom is that when Raymond was 25 he left his pregnant girlfriend, Noreen O’Horan – stage name Gay Dawn – and turned his back on the child, paying only minimum maintenance: £1 a week at first, then £1.50. This feeds nicely into the “stingy old miser” stereotype. But it is not what happened.

McCarthy, who has turned down frequent offers of money from the tabloids to spill the beans about his father, agreed to break his silence for this article. He is keen to put the record straight on his father’s behalf. “My mother was making a good living on the stage. She didn’t need or want support. My father was making arrangements to marry her, but she was a very headstrong woman who said, ‘Forget it. I don’t want to marry you. I don’t want to marry anybody.’ She was very unusual for 1950. It caused massive problems in the family.” So Paul Raymond had not deserted this woman. She turned him down. It was Paul who called the boy Derry. On his original birth certificate he was Darryl. He had it changed by deed poll later.

For 25 years, Derry idolised his father and suffered anguish at not being part of his life. They shared the same birthday, November 15, so every celebration reminded Derry of the absence. He devoured every newspaper article. “As a child, you excuse anything. When I read anything bad about him, I didn’t believe it.” Why didn’t he contact him? “It was like, if you see an attractive girl, while you don’t ask her out, you can fantasise about her. If you ask her out and she says no, the rejection can kill you.”

Finally, in 1975, when he was in London on business, Derry rang Raymond’s office. Raymond wasn’t there. Derry wrote a note on the headed paper of the crummy hotel in Bayswater where he was staying. Three or four weeks later, he got a letter on blue stationery, with Paul Raymond Organisation at the top. “I nearly dropped dead.” In it, Paul Raymond gave him his home address and home phone number and said he’d welcome the opportunity to meet.

They spent a day together at Raymond’s flat in Portman Square. Derry’s stomach was churning when he knocked on the door, but his father couldn’t have been nicer. They chatted and had coffee, then lunch, at the flat. Raymond’s girlfriend Fiona Richmond came home around 5pm and they carried on talking, had dinner. “We had a relaxed evening, a drink, talked about my family, my mother, what I was doing. I left at 8.30 floating on air. And then we never spoke again. Ever.”

Why not? “He didn’t contact me and I didn’t contact him.” Derry hoped Raymond would ring him or write again. But he didn’t. Perhaps he was thinking the same thing. Time went on, and it became awkward. And that day had been so perfect. “The memory was so good, I didn’t want it spoilt. My fears of rejection had not come true. In fact, the opposite had come true: I’d had acceptance. But if I pushed it, who knows?”

He heard about his father’s death like everyone else, on the news, on his car radio. He felt shaky, had to pull over. He appears to have been left nothing in the will. He doesn’t want to seem money-grubbing, and his first impulse is always to defend his father, but it clearly rankles. Until it was lost in a house move in 2002, he had carried his father’s letter around in his pocket, every single day for 27 years. “I’d folded it, put Sellotape on the back of it where it had started to crack. It was the only thing from him I ever had.”

) ) ) ) )

There was one thing Paul Raymond excelled at, and that was making money. He died with a vast fortune some have estimated at £650m. Quite an achievement for the son of a haulage contractor who left school at 15. He was ruthless, single-minded in his pursuit of wealth. But at what cost? “I think he was happy smoking and drinking at the Revuebar,” remembers his former publicist, Peter Thompson. “But it was sort of make-do happiness. Not ecstatic, just getting through. I think he was desperately lonely.”

Business was his pleasure, and he was never nervous about treading on people’s toes. Even old loyalties gave way to the bottom line. Gerard Simi, his choreographer at the Revuebar for 30 years, bought the lease and the name in 1997, but failed to revive its fortunes. Over a few years Raymond’s property company, Soho Estates, nearly doubled the rent and the enterprise went bust. Most people say the Revuebar had simply had its day, but Simi is still bitter. “I would have sold my soul for pennies to save the venue. But pennies and a lifelong commitment were not enough, and the greed of his organisation killed this extraordinary adventure,” he lamented.

“For Paul, when it came to business, there were no friends,” says the restaurateur Mark Fuller, co-owner of London’s Embassy club and godfather to Raymond’s grandchildren by Debbie. “He didn’t suffer fools gladly. He was a very hard landlord.” Fuller took the lease on a site in Greek Street and transformed it into Little Havana, and Raymond gave him a year to get the rent together. “If you told Paul a lie, that was the end of it. If you told him the truth, he would bend as much as he could. If he had a strong belief in you, he was fair. But I’m sure that had we not come through, he’d have brought the hammer down.”

“He was a supporter of the small person, not the big brand,” says another successful tenant, Nick Jones, founder of Soho House. “When I read something bad about him I get annoyed.” Soho House came into being when Jones was renting the premises below, what is now Cafe Boheme on Old Compton Street, from Raymond’s property company, co-run by John James. Jones was interested in the upper part of the building, but didn’t have the money to invest in it. So they agreed a deal whereby Raymond loaned Jones £1m to cover the refurbishment costs and added the repayments to the rent. “It was a clever way to finance it. There were times when the rent was hard to pay and I’d say, ‘Can I pay in instalments?’ They were always open to that. I’ll never forget leaving his office that day. That was my big chance. The world needs people like him to make areas happen. If he wasn’t buying up Soho, who would have? There was never an anti-Paul Raymond feeling around the place. You get landlords you can talk to, or landlords who are big corporations who don’t understand what you’re trying to do. And Paul did understand.”

Raymond knew what it was to put everything into achieving a goal. Born Geoffrey Anthony Quinn in 1925 to Irish parents, he was raised by his mother and aunts after his father left them when he was five years old. He had a Catholic education, played drums in dance bands and tried to dodge national service, once inducing a heart murmur by swallowing large amounts of saccharin and bread. Finally he did his national service, as a bandsman and switchboard operator in the RAF.

He had two brothers; one became a doctor, one a pilot. He didn’t keep in touch with them when he left their home in Glossop, Derbyshire. He changed his name to Paul Raymond when he was 22. He partnered Gay Dawn in a mind-reading act on Clacton pier, then began producing vaudeville around the country (the girls took their tops off for an extra two shillings a week). By 1951 he was putting on nude tableaux of women to an excitable post-war London audience. Private clubs were exempt from the government ruling that forbade naked women to move on stage. In 1958, he opened the Revuebar, offering legal striptease to private members. Within 10 years he had made over half a million pounds. He would admit he had been in the right place at the right time, been lucky to find a niche, but he was also an incredibly focused businessman. Carl Snitcher, Raymond’s lawyer and right-hand man, recalls: “He had the ability to spend hours drinking, and when everyone else’s ties were up round their necks and their shirts dishevelled, he’d be immaculate. We’d be slobbering all over the place, falling over, and he’d be in control. He wouldn’t let his guard down.”

Raymond persuaded Snitcher to work for him by offering him five times what he was earning in the early 1970s as assistant general secretary of the actors’ union Equity. It was a shrewd investment: with Snitcher on board he could win virtually any legal battle. He could afford to let disputes run on for years if necessary. “On a personal level, he was generous. As an employer he counted the pennies.” At home he could be frugal too, once giving his housekeeper an earful for spending £10 on pansies.

“He wasn’t particularly loyal to anyone,” admits Snitcher. “He was one-dimensional. If someone had a drink or drug problem, he was unsympathetic. He just thought, ‘If he can’t do his job, he should leave.’ He would never show any sign of what he would have seen as weakness. He would never concede that he was lonely. He never allowed those discussions to take place.”

Raymond never borrowed money, instead ploughing the returns from the theatres and then magazines into property. “He equated success and money,” says Snitcher. “He was a great entrepreneur. You never really needed contracts with him because he was a man of his word. He could cut through the bullshit. He didn’t read a lot, so his vocabulary was rather limited. But he had an instinct for seeing opportunities. And he never blamed anyone else for anything that went wrong.” Raymond was addicted to the buzz of making money. Once his energies had shifted into acquiring swathes of London – his organisation now owns an estimated 60 of Soho’s 87 acres, as well as properties in Notting Hill and Chelsea – he had no need for publicity. But he couldn’t duck the headlines – which newspaper could resist a flamboyant pornographer who seemed to be getting away with rather too much?

In 1968, Raymond was banned from driving for 12 months after driving erratically along the Embankment. The police report stated that “When he was given a Breathalyser test he sucked instead of blowing and was smoking at the time.” In 1972 the Obscene Publications Squad seized 260,000 of his “girlie mags”. In 1983, Raymond won a libel and slander action against a Westminster city councillor, Lois Peltz, who had given an interview to a South African magazine suggesting he used his business to launder money from the mafia. Snitcher insists: “There was this perception that we were involved in organised crime, and we were not. We were involved in racy entertainment that wasn’t everyone’s cup of tea.”

Even at his peak, Raymond preferred to have his daughter or a close associate represent him in public. Not least because he had a bad stammer (he once gave the Windmill Theatre over to a stammerers’ charity, free of charge). “He wouldn’t do interviews,” says Thompson, who did publicity for Raymond throughout the 1980s and early ’90s. Thompson was “staggered” to find himself liking the chain-smoker with the big hair. “I loved him. He didn’t behave how rich people tend to behave – rudely, like you don’t matter. He was incredibly generous, and not grandiose. He ate at the Piccadilly restaurant in Great Windmill Street: a nice Italian, but, you know, a £20 dinner. And he was caring. He wouldn’t dream of letting you get a cab home from a party – it was, ‘No, I’ll give you a lift. I want to make you get home safely.’ He wasn’t me-me-me. He drank – as did Debbie. She was his life. Her death finished him.”

Debbie Raymond was an ambitious, spontaneous party animal who adored her father. In 1986, Debbie and Paul were interviewed for this magazine’s Relative Values column. Debbie was 30 then. She had just had her first daughter, Fawn, by Duncan MacKay, keyboard player with the band 10cc. She and MacKay were not living together; she was dividing her time between a house in Chobham and a flat next to her father’s in London. Paul on Debbie: “Her weakness is falling in love the whole time, which I think is a disaster. A lot of girls do – men too, but I don’t believe it affects them as much.” Debbie on Paul: “He is an absolutely fantastic father to me – ever so caring. I can phone him up at four in the morning crying my eyes out over something, and he’s always there to help and listen.” On another occasion she said: “My father is the man I would marry if I could.”

When her parents divorced, Debbie took her father’s side. At 18 she went off to Las Vegas, got breast implants and a job as a showgirl. She tried singing too. Back in London, she persuaded her father to let her join his publishing firm, rising to become editor-in-chief of his magazine division. She had a white Rolls with the numberplate DR11. She would “kidnap” friends at 2am and drive them out to her house in Chobham, then be back in work the next day. Usually.

After she lost both her breasts to cancer, Debbie had reconstructive surgery and defiantly went back to wearing strappy tops. Her father said she was the toughest person he had ever met. She drank like a fish, did masses of cocaine. She was flamboyant, loyal to her friends, free with her favours. One former acquaintance calls her, affectionately, “a loony”. Another says she was “a moody old bitch”. Another recalls: “There was an unsettling edge to her, a feeling that high spirits could easily tip over into anger or bitterness. There was an inner anxiety. I never saw her truly relaxed.” Her younger brother, Howard, describes her as “a wild, wacky, loopy woman”. When he was learning to walk, she dug her mother’s high heels into his hands, jealous of the attention he was getting. Her twin died at birth. “She got two personalities,” says Howard. “She could be wonderful, and she could be the most ghastly person on Earth – vindictive, horrible. You never knew what you were going to get. Best to avoid her.”

Debbie divorced John James, with whom she’d had her second daughter, India Rose, and her trajectory took a turn for the worse. Her drug use went from recreational to fatal. When she was found dead of an overdose in a boyfriend’s flat in November 1992, she had been taking heroin as well as vodka and cocaine. “Deborah liked to play in the grey area,” says Howard, who thinks her death was an accident that would have happened sooner or later. “She was always pushing it, always.”

When Carl Snitcher picked up the phone on a cold November morning, he didn’t recognise his employer’s voice. Usually, Paul Raymond was calm, level. You couldn’t tell if he was going through a tough time. Now he was reduced to a hoarse whisper, repeating the words over and over again, stunned by what he had just found out. “She’s dead… She’s dead… She’s dead.”

) ) ) ) )

“He had a really infectious laugh, a razor-sharp mind and a fabulous self-deprecating sense of humour. He didn’t take himself too seriously.

I remember laughing a lot.” Julia Montgomery is a well-spoken lady who runs boutique guesthouses in Wiltshire and Grenada with her husband. In the 1970s she was Fiona Richmond, glamour model and actress and Paul Raymond’s girlfriend. Her lurid column in Men Only detailed her supposed sexual adventures with men and women around the world.

Ironically, Raymond the porn baron was quite prudish. When one of his cast farted loudly on stage, he put up a note backstage asking that they “desist” from being so vulgar. Snitcher says Raymond was “absolutely not lecherous at all. That sort of behaviour embarrassed him. He had old-fashioned manners”. At Christmas he would insist on watching the Queen’s speech on TV.

Richmond was starring in one of his farces at the Whitehall Theatre when they met. She was living in a flat with four other girls. Raymond used to go to Kentucky Fried Chicken in his Rolls-Royce and take them all supper. He and his wife Jean had separated and he was living in a hotel.

“I didn’t break up this idyllic marriage,” she says.

He bought a flat in Fitzhardinge House, Portman Square, and they moved in together. When Debbie was working in Las Vegas, Howard used to stay with Raymond and Richmond for weekends. “They were just another normal divorced family.” Jean Raymond received a lump sum thought to be £250,000 in their 1974 divorce. She told a reporter: “It is an agreement satisfactory to us both.” She also said: “It’s like chucking away your crutches.” Any bad feeling was put aside two years later when Paul took pictures of her, nude astride a stuffed tiger, for Men Only – for a generous fee. Raymond didn’t want Richmond to work, but she needed to. “I didn’t want to be just a rich man’s girl. And he was proud of that fact. He knew I wasn’t with him for his money. To me, he was vulnerable.

I shouldn’t think anyone else saw that side to him. To come from nothing to having such incredible wealth – and he had no one to share the burden of it. Because it is a burden.”

They would talk late into the night, scheming about shows they might put on. “He once said to me, ‘I don’t need to gamble – I do it every day!’ He sold his house and his car to open the Revuebar. Most people are not that brave. He didn’t know what to do, other than work. When he had a financial hiccup, he came to life. It galvanised him. He liked to battle. He liked to have his back up against the wall. That’s when he was most alive. When things were going well, he was bored.”

He was kind to her parents – sent his Rolls to pick them up from the airport when they went on holiday, passed his suits onto her father, who was a vicar. “I remember at a church fete one day my father wearing a dark suit with purple lining.” She remembers him falling into a Christmas tree and finding it funny. Raymond did have a temper. “I could make him angry.” But that wasn’t why they split up. She just grew tired of his pace.

It wasn’t an easy parting. “He was distraught; I was distraught. But I wanted more normality. I’d be tired after two shows nightly, and I just wanted to sleep. At first it was wildly exciting, but I was never a person to go to bed late and get up late. I just couldn’t handle the lifestyle.” Ironically, the money he was so good at making put her off.

“I like to see a dress in a window and think, ‘If I work hard enough, I can have that.’ You have to be a strong person not to let that overtake you. He wasn’t pleased I went, but he didn’t try to keep me.”

They remained friends, and Raymond used to phone her up and ask for advice. He was a guest at her wedding in 1983. Debbie and Fiona shared a flat in the Barbican for a while. “She was her father’s daughter. Bright and funny. It was a tragedy. If you have copious amounts of money, it takes away the impetus to do things for yourself.” Debbie’s death “knocked the stuffing out of him. He was a real person, with feelings. He faced it bravely, didn’t make a fuss”.

“Deborah’s death affected him very badly,” says John James, Debbie’s former husband, who runs Soho Estates with Mark Quinn, Raymond’s nephew, “as it did all of us. It took him a long time to get over it and if he talked about it, he got very upset. So he didn’t talk about it.” He pours water on rumours that Raymond let himself go at the end, didn’t cut his fingernails. “Complete nonsense. ‘Recluse’ is too easy a tag. He was a very proud man. He lived the life he wanted to lead. He was in constant communication with the office. He didn’t lose control of his business. Ask anyone what would happen if they lost their kid.”

James’s relationship with Raymond was professional as well as personal: he was already working for him when he met his daughter. Despite the father/daughter closeness (“They were best friends”), Raymond was not a possessive father-in-law. “There was never any ‘You’re taking away my daughter.’” Although Raymond continued to accompany James on family holidays even after James remarried in 1996 and had a son, there was a professional distance. “You didn’t know every facet of this man’s life. You knew your bit. He was a very private man. You knew what he wanted you to know and you didn’t know any more than that.”

James too never spoke to the press while Raymond was alive. “He wouldn’t have wanted you to.” They had hit it off the first time they went out on the town together, starting off watching a potential new act and ending up at Raymond’s flat till 5am. He offered him a job and that was that. “I’d been immediately welcomed in. It was a privilege. Wherever you went, it was an entrance. I just used to tag along.”

As a boss, Raymond was “direct-dealing. He was very clear-thinking and he made all the decisions. He would give you his remit. He didn’t want your details. If you were renting out a property, he’d say, ‘I want £250,000. Don’t bring me anybody who isn’t going to give me £250,000.’ He was very straight. We had forthright discussions, fuelled with alcohol, many a time. He commanded respect. You could get drunk with him, but you’d still be a little on your good behaviour”.

In the mid-1980s, Paul Raymond moved from his apartment in Portman Square to Arlington House near the Ritz, with views over Green Park. His old friend Ian Potter, who worked as a photographer for him for 15 years, says the Portman Square flat had been “like a hotel. Someone expensive had designed it. It had mirrors on the ceiling, blue shag-pile carpets, white leather sofas. One wall was a water feature. There was an electronic roof that never seemed to work. It was a flash place, but horrible. No soul”. By contrast, Raymond’s home in Arlington House “felt nice” and was done out in art-deco style. “I bought him a book for his birthday that was on his coffee table right up to the last time I was there, of art-deco statuettes, and I remember sitting at the bar, him with his Remy brandy, going through ticking off all the ones he’d got.”

Raymond acquired property in a similar fashion. Once, he told Potter he had done a good deal on a property, but paid over the odds for it.

“I owned a whole street apart from that one building. It was like having teeth with one missing. So I paid a bit extra.” It was like a game of Monopoly: “Leicester Square: I’ve got two-thirds of it, can’t get the other third!” Raymond didn’t care when the Raymond Revuebar sign came down, says Potter. It wasn’t his place any more.

Potter saw Raymond every few weeks up until his death. In the last couple of years, Raymond would normally be in bed, in his pyjamas – “I hadn’t seen him in a suit for about six years.” He was in his eighties, after all. He would have all the newspapers spread around him on his bed, and a brandy and a cigarette. He could reel off the current sales figures for different publications off the top of his head. “He’s portrayed as someone who didn’t have a heart, but he did. It just took a bit of getting to. He wasn’t a miserable old miser. But when you’ve got that much money, it’s a problem: who do you trust? Over the years you learn to put your defences up.

“I really miss him. I could tell him all my problems – emotional, business, anything – and his advice was always good. He was a good listener.” Raymond confided in Potter as well. He trusted him, knew he wasn’t after his money. He thinks the multimillionaire found that refreshing. Potter was always dead honest with him. Sometimes Potter slips into the present tense when he is talking about his friend.

“Paul always wondered if the money was responsible [for Debbie’s death]. Any father would tend to blame himself. It knocked him back, and a lot of people thought he’d never get over it. But I always knew he would. He’s a survivor.” Raymond worried about Debbie’s children, too. “Fawn and India were the apple of Paul’s eye. He used to say to me, ‘They’ll never have a poor day.’ But he was also concerned that money can be destructive as well as good.”

They didn’t speak for a year after Debbie’s death. Potter knew his friend was in mourning and didn’t want to disturb him. Then one night he saw a picture of him on Have I Got News for You. He rang him up. “Put it on, quick!” “What am I on there for?” “Paul, you’re famous! You’re a household name.” From then on they’d get together every few weeks for a brandy and to reminisce. Raymond didn’t go out any more.

“I don’t think he wanted to try to be something he wasn’t any more.”

Potter is glad his friend was carried off quickly – went into hospital on the Wednesday, was gone by the Sunday. “I would have hated to see him lose his mind.” His last memory of him, two weeks before his death, was of an elderly but sharp-witted man. It was right, he says, that the funeral was kept to close family only.

In his last few days, Raymond had no regrets. At least, none that he would voice. His son Howard, who saw him every day, says he claimed to be happy. At the very end, he said he wanted to see Jean, the wife he had divorced three decades earlier and who had died in 2002.

“I couldn’t believe it,” says Howard. “I said, ‘She won’t be pleased. She’s got the biggest frying pan waiting for you!’ They were both Scorpios. That tells the story, really. It was always ‘Get rid of the girl and you can come back’ with her. He was gobsmacked, thought she’d put up with it.”

Raymond’s will, which Howard helped him draw up, has already been read out, but its details remain secret, and nobody knows the full extent of his fortune, much of which it is assumed rests in offshore accounts. He left 80% of the estate to Debbie’s children. Perhaps he wanted to compensate in some way for the fact that they had lost their mother. Perhaps deep down he did feel Debbie’s death was his fault, since he had so encouraged her wild lifestyle. Or perhaps the two girls were just his favourite relatives and it was a rich man’s last whim. He wasn’t a huge benefactor, admits Howard. “He would only give to what he wanted to give to.” But he was generous to people and causes that he chose himself. He once donated enough money for a new wing to Brinsworth House, a retirement home in Twickenham for people who have worked in show business. He knew he owed his early success to his performers, and he never forgot his roots.

He took no interest in his own funeral arrangements, beyond wanting to be cremated. His son Howard has his ashes in a jar and is going to take them into Soho and spread them over the streets. “When the weather’s a bit nicer. He wasn’t bothered what happened to them. But I thought it would be nice. It’s his estate. It’s where he belongs.”

The King's men

Stephen Bleach worked with Paul Raymond for a decade. Here he recalls a boss who was ruthless but always decent

“Women will let you down, and friends will let you down,” my boss Paul Raymond told me. “Remember, Stephen, you have to look after yourself in life.”

Trust didn’t come easily to Paul. For over two decades he refused to give interviews or allow outsiders a glimpse inside his secretive organisation. I worked in it for 10 years. The first time I entered his office, in a slightly ramshackle building at the intractably seedy end of Soho, I was a chippy youth, he the grand old man of smut, but he was unfailingly courteous, gently witty, slightly camp. I couldn’t help warming to him.

He was ruthless, obsessed with money, and ultimately trapped by the riches he had worked so imaginatively to acquire. But Paul was also a decent man. As publishing director I oversaw his top-shelf magazines — from the pseudo-glamorous (Club International, Men Only) to the cheerfully brazen (Escort). They were tame by today’s standards, but they were porn nonetheless. Paul never used that word, though. We were publishing “adult entertainment”. It wasn’t hypocrisy. He was just old-school. He didn’t use the word “porn” for the same reason he didn’t swear, shout, renege on a deal or be rude to waiters. He paid his bills, kept his promises and was polite to everyone from the post boy up. But when I tried to negotiate salary rises for staff, he said: “No, £1,000 each is the maximum. As my old mum used to say, ‘Much wants more!’”

A day or two after his daughter Debbie’s death, I got a call from Paul. He was distraught. His gruff bass voice had turned weak and thin. He had lost the person he loved most, and the only one he felt he could trust. For the rest of his life, every relationship would be tainted by his fortune, by his wariness of others’ motives. He knew this. It was the most heart-rending phone call I’ve ever had.

In 1997, I left Paul, and porn – amicably, and with good memories of both. Once in a while, I’d call up for a chat. And then, a few years ago, he asked me over to his penthouse behind the Ritz. I’d been there many times — as he came into the offices less and less after Debbie died, it was the only place he’d have a meeting. The retractable sunroof, the mirrored walls, the 1970s disco decor were all still there, but the curtains were faded and frayed and the geraniums on the terrace were browning. Paul conducted the meeting in his dressing gown. His hair, though — a blow-dried bouffant creation — was as voluminous as ever, and he opened the day’s first bottle of champagne with the old relish.

After some preliminary fencing, the business became clear: he wanted me back. Magazine sales were plummeting and he thought I had the solution. I didn’t, of course.

Short of cutting every internet connection and trashing every satellite dish, nothing will return the top shelf to its former glory.

I said I’d think about it, but we both knew what that meant.

As we shook hands in the white marble hallway, I felt I’d let him down somehow, just as he predicted everyone would. As self-fulfilling prophecies go, it’s a bleak one. He turned and walked back to his loneliness and his money. I never saw him again.

Times recommends

• Jennifer Aniston, time to be a real bitch
• It may be hard to say so, but conversation is back
• News Review interview: Georgina Baillie