It came after a man, responding to publicity about the death, told police he had seen the actor at the New York apartment building where they were arrested on Tuesday. He said he believed that is where the 46-year-old got the heroin.
In searches of two flats in the building, police found hundreds of packets of the drug in one, according to a criminal complaint.
Prosecutors declined to pursue charges against one person, saying there was no evidence he had control of the drugs or the flat, a nd two others were charged only with possessing cocaine.
Only one, jazz musician Robert Vineberg, was facing a charge of heroin possession with intent to sell.
Lawyers for the three people charged denied their clients had any role in Philip's death and suggested they were being swept up in the attention surrounding it.
"This case and the charges against Mr. Vineberg have absolutely nothing to do with the death of Philip Seymour Hoffman. ... We're hoping the (district attorney) will not use Mr Vineberg as a scapegoat," said his lawyer, Edward Kratt.
The arrests came two days into the high-profile case, with all three people charged indicted within a day of their arrests and being held without bail.
The two charged with cocaine possession are Juliana Luchkiw and Max Rosenblum, a couple who are neighbours of Vineberg's.
"She's not a drug dealer. She's a college student," said Ms Luchkiw's lawyer, Stephen Turano. Mr Rosenblum's lawyer, Daniel Hochheiser, said his client "has nothing to do with Philip Seymour Hoffman".
The couple had two bags of cocaine, while investigators found about 300 packets of heroin, a bag of cocaine and about 1,200 dollars (£736) in cash in Mr Vineberg's flat, according to criminal complaints.
Investigators have determined that the Capote star made six ATM transactions for a total of 1,200 dollars (£740) inside a supermarket near his home the day before his death.
They are examining a computer and two iPads found at the scene and recovered syringes, a charred spoon and various prescription medications, including a blood pressure drug and a muscle relaxant.
Officers are waiting for a cause of death for the Oscar-winning actor from the medical examiner's office, which has said that more tests were needed.
He was found dead on Sunday with a needle in his arm, and tests found heroin in samples from at least 50 packets in his apartment in Manhattan's Greenwich Village.
A private funeral for relatives and close friends is set for Friday, and a larger memorial service will be held later this month, his publicist Karen Samfilippo said.
On Wednesday night, Broadway theatres dimmed their lights in memory of the Tony Award-nominated actor, and members of the theatre community held a candlelight vigil for him.
"Courage was his forte, always," said playwright and actor Eric Bogosian.
"In this world of creative enterprise, it is ultimately up to the artists to decide how high a bar he or she will set for themselves. Phil set his bar on the highest rung, on a rung above the highest rung."
Philip earned a Tony Award nomination each time he appeared on Broadway. He made his debut there in Sam Shepard's True West in 2000 and followed it up three years later with Eugene O'Neill's Long Day's Journey Into Night with Brian Dennehy and Vanessa Redgrave.
In 2012, he played a powerful Willy Loman in Death Of A Salesman by Arthur Miller opposite Andrew Garfield and under the direction of Mike Nichols.