That's a problem too, but department heads going to/from industry is mainly a problem of bad policy that doesn't align with what voters want, versus government contractors being problem of increasing government spending. If you want to reduce government payroll, by definition that means cutting services (politically unpopular and almost never happens) or paying someone else to do what the government used to. Paying government contractors to do what the government used to or could do almost always ends up costing more, because of the cut of profit they slice off the top.
A guy I went to MBA school with does this, he's got a team of contractors working on DoD projects and he collects $10-$20 per hour off each one. Makes over a million dollars a year basically doing fuck all at this point. Nice work if you can get it, but a perfect illustration of why we spend so much money and get so little in return.