In this paper, detailed studies of structural, micro-structural, dielectric, electrical and magneto-electric characteristics of chemically synthesized a new double perovskite, BiSrFeCeO$_6$ (strontium cerate-modified bismuthferrite), are reported. The basic crystal data (unit cell dimension and orthorhombic crystal system) of the compound were obtained by analysing the room temperature X-ray powder diffraction data. Study of the surface microstructure of a sintered pellet sample, recorded by scanning electron microscope at room temperature, reveals the formation of highdensity sample with the homogeneously and uniformly distributed grains of different size and shape. Analysis of frequency- temperature dependence of dielectric parameters and nature of hysteresis loop has exhibited the ferroelectric characteristics of the material. Studies of complex impedance and modulus components at different temperatures and frequencies have provided electrical characteristics of grains and grain boundaries present in the material, which are closely related to its microstructure. Thus, the correlation between the crystal structure with electrical properties and relaxation mechanism of the prepared sample has been established. The type of dielectric relaxation process and mechanism occurred in the compound have also been studied not only by analysis of the dielectric properties (${\varepsilon}_r$, tan${\delta}$) and impedance parameters, but also from the study of frequency-temperature conductivity. The existence of negative temperature coefficient of resistance (NTCR) characteristics in the material has been confirmed from the temperature and dc conductivity analysis. Similarly, the frequency and ac conductivity analysis show that the studied compound follows Jonscher’s universal power law. Based on the structural, ferroelectric, magnetic and magneto-electric characteristics, multiferroism in the new composition of bismuth ferrite has been established.