Mount scored to put Chelsea back but Norwich striker Teemu Pukki grabbed a second equaliser with his fifth goal of the campaign before half-time.
Abraham clinched the points in the second half but Chelsea legend Lampard, whose third child was born last year, had no grand plans to celebrate his maiden victory.
“I didn’t want a slow start. We had a tough start and I have got to say I am happy with a lot of the performances.
“But you can’t shout that too loudly if you aren’t winning games. So it changes it for me, today. So I am very happy.
“It gives me more determination to work. I saw a lot of good things, things we can improve but a real good belief in the players, the way they played today. I am happy, but determined, I suppose.”
Abraham has spent plenty of time out on loan and excelled at Aston Villa in the Championship last season before returning to Chelsea and becoming Lampard’s first-choice striker.