Dielectric dispersion and NMRD experiments have revealed that a significant fraction of water molecules in the hydration shell of various proteins do not exhibit any slowing down of dynamics. This is usually attributed to the presence of the hydrophobic residues (HBR) on the surface, although HBRs alone cannot account for the large amplitude of the fast component. Solvation dynamics experiments and also computer simulation studies, on the other hand, repeatedly observed the presence of a non-negligible slow component. Here we show, by considering three well-known proteins (lysozyme, myoglobin and adelynate kinase), that the fast component arises partly from the response of those water molecules that are hydrogen bonded with the backbone oxygen (BBO) atoms. These are structurally and energetically less stable than those with the side chain oxygen (SCO) atoms. In addition, the electrostatic interaction energy distribution (EIED) of individual water molecules (hydrogen bonded to SCO) with side chain oxygen atoms shows a surprising two peak character with the lower energy peak almost coincident with the energy distribution of water hydrogen bonded to backbone oxygen atoms (BBO). This two peak contribution appears to be quite general as we find it for lysozyme, myoglobin and adenylate kinase (ADK). The sharp peak of EIED at small energy (at less than 2 $k_{B}T$) for the BBO atoms, together with the first peak of EIED of SCO and the HBRs on the protein surface, explain why a large fraction (∼ 80%) of water in the protein hydration layer remains almost as mobile as bulk water. Significant slowness arises only from the hydrogen bonds that populate the second peak of EIED at larger energy (at about 4 k$_B$T). Thus, if we consider hydrogen bond interaction alone, only 15-20% of water molecules in the protein hydration layer can exhibit slow dynamics, resulting in an average relaxation time of about 5-10 ps. The latter estimate assumes a time constant of 20-100 ps for the slow component. Interestingly, relaxation of water molecules hydrogen bonded to back bone oxygen exhibit an initial component faster than the bulk, suggesting that hydrogen bonding of these water molecules remains frustrated. This explanation of the heterogeneous and non-exponential dynamics of water in the hydration layer is quantitatively consistent with all the available experimental results, and provides unification among diverse features.