It measures temperature change — and you have to teach it what that temperature change means in joules. The calibration factor is that translation key, unique to your specific apparatus. Change the CF and watch the temperature rise: the same reaction always releases the same 5000 J, so a calorimeter with a bigger CF shows a smaller temperature rise.
You run a calibration first by supplying a known amount of electrical energy and recording the temperature rise. If you put in 2500 joules and the temperature rises by 10 °C, your CF is 250 J °C⁻¹. This number absorbs all the quirks of your specific setup — the mass of water, the heat capacity of the container, even heat lost to the air. Once you know your CF, you can measure any reaction in that same calorimeter by multiplying CF by the observed temperature change.