They didn't play perfectly all season. Several games were close contests (vs. Colts, Eagles, Ravens, Giants) and in a few of those instances (mainly the Ravens game) the officiating allowed the Pats to win. The Pressure of the defense was what made those passes hurried and off target. Brady never got into a Rhythm with pressure in his face most of the game. His completions did come when he had some time to set himself and throw the ball. Yeah, the Pats Offensive line went out the window in the biggest game of the season, and the Giants managed to shut down Moss who was double-covered most of the game. But it was the frontline that disrupted the passing game the most. Hard to be accurate when you are hit as you're throwing the ball (that one case they tried Moss deep - he had beaten the coverage but the pass was under thrown as Brady was hit as he released).
Good teams all season? 8 of their games were against teams who finished with more losses then wins (Jets, Dolphins, Ravens finishing in the bottom 6 overall, Buffalo and Bengals finishing well below the .500 mark). Very few teams lost to those first 3. Elis interception in the last game of the season is what sunk the Giants then (the only turnover in the game) in a game the Giants lost by 3.
Pats definitely struggled in the post-season. Giants played far better in January then the Pats did, and in the end thats when it matters the most.
EDIT: Brady has possible injury, didn't show in game. Speculation with Moss is that he just doesnt want to play in the meaningless Pro-Bowl.