Wasps led 14-0, but a try from Matt Hopper and Evans' boot trimmed the gap to one at the break. The visitors moved four ahead with a Nicky Robinson penalty before Evans missed a penalty, scored one and missed another with two minutes to go as Wasps clung on.
On the first day of the season Evans struck a last-gasp penalty to see off Wasps 42-40, and Quins director of rugby Conor O'Shea acknowledged what goes around comes around.
"It's a disappointing loss as you want to win your home games, but we gave them a 14-point lead at the start of the match and then you're chasing," he said.
"We got the rub of the green at the start of the season and then we didn't get it here.
"We got ourselves back into it. We had plenty of ball, plenty of territory and plenty of opportunities but just couldn't seal it.
"Credit to them. We're very disappointed because we felt we had enough opportunity, but the energy of the lead always stayed with them."