Tuesday, October 27, 2015

So what is going on with all this hacking?

As many of you have heard in the past few months, a large number of US government agencies, major US retailers/companies, and credit reporting services have been hacked by persons unknown.

In this post, I hope to discuss with you why this is happening. (Again standard disclaimers apply - this is my own opinion, and not that of any others I may or may not know. If you heard it elsewhere, it was likely because it is all so bloody obvious that any fool - even myself can see it.)

For the past thirty years, the US Intelligence Community (USIC) devoted an extraordinary amount of resources to the creation of a giant sea of information. When you listened to stories from Edward Snowden, about Echelon, Pvt. Manning and Wikileaks etc... you were hearing garbled narratives about this sea. All that meta-data, real data, SIGINT, HUMINT, MASINT etc.. that the USIC collects via various channels is "fed" into this sea.

All those people that rail against the USIC for creating the sea are missing the point - life as we know it is impossible without such a sea. The sea is merely a physical manifestation of collective human knowledge which is otherwise stored in poorly-networked computers we call "our brains". We cannot run a society this global and complex without a better way of sharing common knowledge regardless of how "private" any one of us might feel it is.

After 9/11 the "feeds" to the sea became real-time and more numerous, and the size of the sea grew beyond imagination. All those stories you have heard about tunnel boring machines, underground bunkers, the airport and takeovers of stores etc... these are very common narratives that one comes across when building large server farms for shuffling around secure data. A secure data farm usually consists of a large building over/under ground and connected by tunnels which house secure data links. If you want to keep the data secret, you tend to air-gap the system.

In my travels through various parts of the world, I came across a number of people working on Quantum Information Processing (QIP). I saw the promise of QIP, the massive parallelism of a quantum information system making computation incredibly faster. I too believe QIP can make the human experience infinitely richer.  In my interactions with the QIP crowd, after hours and hours of talking, and peeling layer after layer of the lies that we physicists tell ourselves, I realized that QIP as it stood had little to do with my views on quantum mysticism and its benefits to society as a whole. I finally came to understand that QIP was about an elephant in the room i.e. without a massive increase in computational power, the sea of information so carefully collected is basically useless.

I also realized interest in QIP was a good way to track who faced this problem and given the nationalities of the various physicists working on the issue, I realized the problem was not unique to the USIC. People in other nations had similar problems and from the glee that I saw in their eyes when I spoke about QIP, I grasped that some countries actually desperately needed such processing merely to stay viable. I was stupid I hadn't realized that the burden that a large population and scarce resources can impose on a government. Without information there cannot be order, without order - civilized society will be impossible. 

In order to make this giant information sea useful one needs to be able to sort through an unsorted database and build indexes. Once the indexes are built, you need to construct a very fast correlation mapper that can distinguish between real data and noise. Above that level of code, you need a certain level of higher cognitive functionality which identifies threats based on the correlations that they present as.  

Some of this capability exists in the private sector. Without going into details I don't know, your credit card company uses a similar system to keep track of your purchases and detect fraud. However this capability cannot embrace the massive sea of information that the USIC has collected. The sea is simply too big to be managed with such small scale code. How to manage this sea remains a puzzle.

Various programs in the USIC have focused exclusively on different parts of this puzzle, some names like Able Danger are public, others obviously are not.  A very large amount of science fiction has been published about AIs and how they might view the world through the lens of correlation analysis, but until two weeks when an AI actually outperformed human correlation mappers, all this work was done by human hands - painstakingly and slowly. Not only is this mapping done by humans but security of the entire information sea and all its tributaries is in the hands of humans. There is a massive human element at play here.

Outside of any major computational failures - that human element remains the vulnerability in the system as it stands now. Compromising a human factor will most likely lead to an air-gap being bridged. Once the gap is bridged, a route to the sea will emerge and as demonstrated by the Snowden saga, a hyper-empowered individual will arise from the exploitation of that route.

This sea of information, the feeds that contribute to it and the analytical short-comings (human or AI) that prevent timely threat predictions, all add up and create the biggest vulnerability to national security in known history. This sea and everything associated with it is now a giant target for anyone with hacking capabilities. Every nation on earth will treat such it as a legitimate target for intelligence operations. Every private entity with an agenda or ambition for world domination will try to take a crack at it. The creation of this entity has sparked a global war in cyberspace. And as cyberspace increasingly connects all aspects of our lives, this war will become very real and hit very close to home.

If nation states succeed in keeping control over this sea and associated tributaries, then we will see the established order continue, although I imagine we may see major economic shifts as the resources needed to defend control over the sea are mobilized on a national scale. If private parties succeed where national forces fail, then we will slide into a world like that described in post-nationalist post-cyberpunk classics like Ghost in a Shell. If nations and private parties vie for power and fail to reach a decisive victory - then the future will be more dystopian. There is no limit to how dark it can get.

I hope this helps people in understanding what is happening. We are facing the greatest threat to humanity since the day the first atomic explosive device was demonstrated.

Perhaps if we understand the problem - we may be better placed to solve it.


At 9:03 PM, Blogger Nanana said...

Mav, how easy is it for the Pakistanis to have tested these supposed tactical nukes without detection (or public detection) ? Or the uncertainity is part of the deterrence doctrine?

At 8:38 AM, Blogger maverick said...

Dear Nanana,

I would be very skeptical of undeclared tests.

The whole point of deterrence is to let the enemy know of your capability - otherwise it doesn't deter anything.

A rational power would shy away from such choices, but then even between rational powers there can be miscommunication.

Vipin Narang etal. have commented extensively on the potential for miscommunication on account of the technical ambiguity between India's stated "no first use" and "guaranteed second strike" capabilities.

If Pakistanis want to pursue secret testing of pre-designs of physics packages, then that is up to them. There are a variety of cold test ideas out there and the Pakistanis can easily choose something that works for them, but when it comes to a "tactical nuke" we are talking about a weaponized configuration. That needs to be tested overground to demonstrate visible military effects.

At the present time Pakistan does not have such a test site in landmass or territorial waters. It is also a signatory to the PTBT at this time. If Pakistan participates in a North Korean overground test, then it is unlikely that people will consider Pakistan a viable candidate for membership of the NSG and other like minded groups.

At 4:37 AM, Blogger maverick said...

I know some of you are concerned that Pakistan has a plutonium based physics package in its possession and that they are able to test it - it is a valid concern, and it is difficult to imagine why Pakistan would build up so much infrastructure around a plutonium based route to weapons if it didn't have something resembling a viable physics package.

I get all that and I sympathize with such worries, but I simply point out to you that Pakistan's actions in this regard are driven purely by its sense of imminent hostility with GoI. If the Pakistani potential on a Pu based package is to remain recessed, the GoI needs to reign in the loudmouths (this includes certain ministers who seem to talk before they can actually think). If GoI can't do that - then we will see a major nuclear escalation in the coming months. As all such escalations are punctuated by terror strikes in India and a rise in the conventional tension along the border - one should also expect that. Things will start shaking and jangle around for years to come afterwards.

I sort of take a distant view of these matters as I am on the outside looking in. I don't know who decides what in New Delhi and what if any coherence they can muster in the face of determined high-yield-touting-thermonuclear-know-it-alls!!!

I am going to take the path of least resistance. I am going to sit back and let the thermonuclear-bullshit-brigade burn itself out. Like their physical counterpart, they will imo exceed Lawson's criteria for a nanosecond in a highly inefficient way and then gloriously crap out!

I am going to focus the blog towards the challenges that accompany the use of AI based correlation mappers to produce data driven analysis of national security matters.


Post a Comment

<< Home