Sounds easy, doesn't it? In order to save public expenditure, sack a few (or rather more than a few) civil servants and move the rest out of London.
Well, surprising as it may seem, civil servants are also people with certain rights. So when it comes to making them redundant, they are entitled to a pay-off, based on the length of their service. Alternatively, the more elderly may choose to retire, in which case HMG has to cough up a lump sum and a pension.
And, even if you move civil servants out of London, they are entitled to take their London weighting (in terms of additional salary) with them, at least for a year or two or three. Furthermore, renting office accommodation on the scale undertaken by the civil service is not like renting an individual flat; the terms of the lease will invariably preclude you from giving one month's notice before gadding off and you may be committed to the property for years.
So sacking or moving civil servants may well bring about public expenditure savings, but these may not be realised very quickly. In the meantime, treat the alleged savings quoted by Ministers with a pinch of salt.