The US Office star recently worked with the Bourne actor on the drama Promised Land and joked that he is now "totally in love with the guy".
The 33-year-old told People: "I would love to have a bromance with him but I don't even know if I could even use the word 'bromance' because he and Ben are so tight and like a married couple that I'll never get in there."
John, who is married to British actress Emily Blunt, got to spend plenty of time with Matt when they were working on their film, which they wrote together and co-starred in alongside Frances McDormand and Hal Holbrook.
But he joked that he felt like the actor's "bromance mistress" while his long-time friend Ben - who Matt rose to fame with when their film Good Will Hunting was a hit in the 90s - was the real thing.
Speaking at the New York premiere of Promised Land, John said: "I'm on the outside of the relationship, only working with him on the side while really he's suppose to be with Ben. I'm clearly the mistress and this has got to change!"