An early goal from defender Sebastien Bassong put Norwich ahead and Anthony Pilkington slotted home a second after 37 minutes. But after conceding a goal on the stroke of half-time to Craig Gardner, the Canaries had to fight for much of the second half to deny Sunderland a share of the spoils.
Whilst Norwich extended their unbeaten league run to eight games, Sunderland have won just two of their last 22 Premier League fixtures. O'Neill said: "You need the points. We haven't at the minute got enough points on the board and we have to rectify that. Our position in the league would suggest just that (we are in a fight)."
Matthew Kilgallon missed the best of Sunderland's second-half efforts while Gardner rattled a post and saw another shot cleared off the line. O'Neill added: "It is a battle for us and we have to pull through - it was a battle last year and so from that viewpoint I'm hoping the experience they have got will stand them in good stead this season."
O'Neill bemoaned reckons the turnaround in the second half should give the team confidence going forward, although talismanic striker Steven Fletcher could be missing through injury having been brought off with a knock at half-time.
"I said to the players at the end that if we perform like we did in the second half we will be fine," O'Neill said. "We are certainly not good enough to gift goals to teams and be able to turn that around, the top sides are able to do that but we are certainly not a top side."
Norwich manager Chris Hughton admitted his players felt the strain of playing three matches in the space of a week as they struggled to contain Sunderland during the second half.
"I thought we were excellent in the first half and it is difficult to sustain that over 90 minutes, particularly with it being the third game in a week," he said. "We haven't made any changes to the team so to sustain that for 90 minutes is tough and of course their goal gave them the lift they needed.
"It was a real tough shift in that second half so testament to the players, there were saves that needed to be made and bodies that needed to be in front of some balls and they did that."