None of this...
...requires any deep "conspiracy theory"
During the Cold War, spying was big business. A lot of money was spent by nations in the interests of "national security". Whole entities came into being as a result; GCHQ, MI6, NSA, etc etc, along with a vast industry to provide and support those services.
With the end of the Cold War, those financial interests were staring at The Wall and seeing their end in sight. With the events of 9/11 it was easy for those interests to create lobbying to convince a weak and ill-informed government that the threat was much larger and more cohesive than it appeared. Systems, technologies and methods were sold on the basis of that threat.
As time passed more systems were created, threats were expounded upon by NGOs, lobbyists and public groups that had all bought into the expertly created "null scenario" and the market expanded. New investment opportunities appeared and new business was created to take advantage. In an era of financial instability ANY new business was good business, even if it were founded upon complete nonsense. What was important was that money could be made.
Not only was it a "good day to bury bad news" but also a good day to turn a profit. Create a new tech, lobby the government to buy the tech, convince everyone that they need the tech to keep them safe, get the gov. to buy new, better tech and keep up the tales of imminent destruction and the need for security, pay for "research" to support that market, develop new tech..... repeat ad nauseum.
This is not without precedent, take a look at the drug industries and see how they "remap" pharmaceuticals every year to expand potential markets without the need to create new drugs. They just create new "conditions" that old drugs, it just so happens, treat.
No conspiracy required, no evil cabal plotting world domination, just day to day business-as-usual. Everything is exactly as it always is and all is driven the Great Good that is Profit.