The monthly sunspot numbers compiled by Temmer et al. and the monthly polar faculae from observations of the National Astronomical Observatory of Japan, for the interval of March 1954 to March 1996, are used to investigate the phase relationship between polar faculae and sunspot activity for total solar disk and for both hemispheres in solar cycles 19, 20, 21 and 22. We found that (1) the polar faculae begin earlier than sunspot activity, and the phase difference exhibits a consistent behaviour for different hemispheres in each of the solar cycles, implying that this phenomenon should not be regarded as a stochastic fluctuation; (2) the inverse correlation between polar faculae and sunspot numbers is not only a long-term behaviour, but also exists in short time range; (3) the polar faculae show leads of about 50–71 months relative to sunspot numbers, and the phase difference between them varies with solar cycle; (4) the phase difference value in the northern hemisphere differs from that in the southern hemisphere in a solar cycle, which means that phase difference also existed between the two hemispheres. Moreover, the phase difference between the two hemispheres exhibits a periodical behaviour. Our results seem to support the finding of Hiremath (2010).