In a rare public speaking opportunity, Mr Mathieson urged men to have an examination to check for cancer, adding: "A small female Asian doctor is probably the best way."
The prime minister's smile vanished. She later said: "He's certainly acknowledged that the joke cracked last night was in poor taste."
Mr Mathieson has become an advocate for men's health issues, but is known for occasional gaffes, including saying "you again!" to the queen on their second meeting.
His latest comments were made at a reception for the West Indies cricket team. (© Daily Telegraph, London)