They administer the building contract between you and your contractor: preparing the specification, running the tender, completing the JCT contract, inspecting the works, valuing what has been done, issuing interim and final certificates, pricing variations, and certifying practical completion and the final account. In short — they make the contract's protections actually operate.
For anything beyond minor works, yes — a standard-form contract such as JCT Minor Works defines the price, programme, payment terms, insurance and what happens when things change or go wrong. Most residential disputes we see as expert witnesses trace back to works run on a quote and a handshake. The contract costs little to put in place relative to what it prevents.
They are appointed and paid by you, and they advise you — but when certifying payments, extensions of time and completion they must act fairly between the parties. That impartiality is a feature, not a flaw: it is why contractors accept the certificates, why disputes are rarer, and why the paper trail holds up if a disagreement ever escalates.