Maverick's World

So what is going on with all this hacking?

As many of you have heard in the past few months, a large number of US government agencies, major US retailers/companies, and credit reporting services have been hacked by persons unknown.

In this post, I hope to discuss with you why this is happening. (Again standard disclaimers apply - this is my own opinion, and not that of any others I may or may not know. If you heard it elsewhere, it was likely because it is all so bloody obvious that any fool - even myself can see it.)

For the past thirty years, the US Intelligence Community (USIC) devoted an extraordinary amount of resources to the creation of a giant sea of information. When you listened to stories from Edward Snowden, about Echelon, Pvt. Manning and Wikileaks etc... you were hearing garbled narratives about this sea. All that meta-data, real data, SIGINT, HUMINT, MASINT etc.. that the USIC collects via various channels is "fed" into this sea.

All those people that rail against the USIC for creating the sea are missing the point - life as we know it is impossible without such a sea. The sea is merely a physical manifestation of collective human knowledge which is otherwise stored in poorly-networked computers we call "our brains". We cannot run a society this global and complex without a better way of sharing common knowledge regardless of how "private" any one of us might feel it is.

After 9/11 the "feeds" to the sea became real-time and more numerous, and the size of the sea grew beyond imagination. All those stories you have heard about tunnel boring machines, underground bunkers, the airport and takeovers of stores etc... these are very common narratives that one comes across when building large server farms for shuffling around secure data. A secure data farm usually consists of a large building over/under ground and connected by tunnels which house secure data links. If you want to keep the data secret, you tend to air-gap the system.

In my travels through various parts of the world, I came across a number of people working on Quantum Information Processing (QIP). I saw the promise of QIP, the massive parallelism of a quantum information system making computation incredibly faster. I too believe QIP can make the human experience infinitely richer.  In my interactions with the QIP crowd, after hours and hours of talking, and peeling layer after layer of the lies that we physicists tell ourselves, I realized that QIP as it stood had little to do with my views on quantum mysticism and its benefits to society as a whole. I finally came to understand that QIP was about an elephant in the room i.e. without a massive increase in computational power, the sea of information so carefully collected is basically useless.

I also realized interest in QIP was a good way to track who faced this problem and given the nationalities of the various physicists working on the issue, I realized the problem was not unique to the USIC. People in other nations had similar problems and from the glee that I saw in their eyes when I spoke about QIP, I grasped that some countries actually desperately needed such processing merely to stay viable. I was stupid I hadn't realized that the burden that a large population and scarce resources can impose on a government. Without information there cannot be order, without order - civilized society will be impossible. 

In order to make this giant information sea useful one needs to be able to sort through an unsorted database and build indexes. Once the indexes are built, you need to construct a very fast correlation mapper that can distinguish between real data and noise. Above that level of code, you need a certain level of higher cognitive functionality which identifies threats based on the correlations that they present as.  

Some of this capability exists in the private sector. Without going into details I don't know, your credit card company uses a similar system to keep track of your purchases and detect fraud. However this capability cannot embrace the massive sea of information that the USIC has collected. The sea is simply too big to be managed with such small scale code. How to manage this sea remains a puzzle.

Various programs in the USIC have focused exclusively on different parts of this puzzle, some names like Able Danger are public, others obviously are not.  A very large amount of science fiction has been published about AIs and how they might view the world through the lens of correlation analysis, but until two weeks when an AI actually outperformed human correlation mappers, all this work was done by human hands - painstakingly and slowly. Not only is this mapping done by humans but security of the entire information sea and all its tributaries is in the hands of humans. There is a massive human element at play here.

Outside of any major computational failures - that human element remains the vulnerability in the system as it stands now. Compromising a human factor will most likely lead to an air-gap being bridged. Once the gap is bridged, a route to the sea will emerge and as demonstrated by the Snowden saga, a hyper-empowered individual will arise from the exploitation of that route.

This sea of information, the feeds that contribute to it and the analytical short-comings (human or AI) that prevent timely threat predictions, all add up and create the biggest vulnerability to national security in known history. This sea and everything associated with it is now a giant target for anyone with hacking capabilities. Every nation on earth will treat such it as a legitimate target for intelligence operations. Every private entity with an agenda or ambition for world domination will try to take a crack at it. The creation of this entity has sparked a global war in cyberspace. And as cyberspace increasingly connects all aspects of our lives, this war will become very real and hit very close to home.

If nation states succeed in keeping control over this sea and associated tributaries, then we will see the established order continue, although I imagine we may see major economic shifts as the resources needed to defend control over the sea are mobilized on a national scale. If private parties succeed where national forces fail, then we will slide into a world like that described in post-nationalist post-cyberpunk classics like Ghost in a Shell. If nations and private parties vie for power and fail to reach a decisive victory - then the future will be more dystopian. There is no limit to how dark it can get.

I hope this helps people in understanding what is happening. We are facing the greatest threat to humanity since the day the first atomic explosive device was demonstrated.

Perhaps if we understand the problem - we may be better placed to solve it.

posted by maverick @ 4:31 AM