Observing Thinking

Observing Thinking
Observing Thinking

Tuesday, August 27, 2013

Metadata = Data? You bet. (for Sept 8 PR column)

As I mentioned in last month’s column, the story of NSA’s domestic surveillance made public by Edward snowden has legs. In fact, if the story were an insect, it would be a millipede. Now before you send me a nasty correction, let the record show that I know that, by definition, an insect is limited to six legs but “millipede” sounds so much better than “arachnid. It would seem that in this brave new digital age there should be not only millipedes but mega, giga, tera and even petapedes. No matter. Suffice to say that this story shows no signs of ending well or soon.

As of Aug 21, the latest twist to this thriller revealed that two years ago the FISA court strongly admonished the NSA for sweeping up domestic along with foreign intelligence gathering. The crux of the issue was that, without a warrant, the NSA had no authority to spy on US citizens and in fact, were violating the fourth amendment protecting citizens from unreasonable search.

I have spent some weeks researching the method that NSA must have used to intercept US citizen’s phone calls, emails and other Internet transactions and could only find the political and economic aspects --- how they pressured Internet providers like Verizon and AT&T to “share” their data unbeknownst to US users. There was very little information about the actual techniques applied to the data once the NSA had it in their hands.. So I decided to abandon the experiential approach and apply deduction instead. After all, I had taught the Database Management course in my career as Computer Science professor so why not put to use what I had learned? Here’s the way I think it went.:

Once the NSA had all of this data safely stored on their collection of disks they could make the first pass over the data to create their database. The three main functions of a database system are: Create, Update, and Interrogate. In the Create phase the raw data is usually indexed for rapid retrieval during the Update and Interrogate phases. Indexing is a fairly straightforward operation; if you’re of a certain age, you remember thumb-indexed dictionaries to faciitlate the Interrogate function. For example if you needed the definition of “mendacious” you could start your search immediately in the “M” section of the dictionary thanks to the handy thumb indentations rather than begin on page 1 and search sequentially from there. Techniques similar to this are embodied in computer programs whose job it is to update and interrogate large databases --- similar in kind but not in degree. These programs not only allow for multiple indexes as links to the database but are degrees of magnitude faster than manual methods.

For example, if I am the program looking at one of your emails, I can record the time and date it was sent, your and the recipient’s email addresses as well as any keywords that have been deemed important like: “bomb”, “Egypt”, “Syria”, “China”....you get the idea. Next, I determine the location in disk memory where this email will be stored but before I store it I make a note, in the form of a list which associates each of the keywords with that disk location. This process is repeated for all of the emails in the database and when it’s finished we have created a table of keywords and the disk locations of the emails that contain that word:

Keyword / Location

aardvark 636542

bomb 124679, 001489, 789325

... ...

zygote 987654, 123321

Now imagine that I’m the Interrogate program and my human NSA agent wants to look at all emails that contain the word “bomb”, all I have to do to make him happy is consult my table of associations between keywords and disk locations, go to each location (124679, 001489, 789325) and display the full email located there.

By this time dear reader, you may have surmised that these keywords that link to and allow rapid access to individual emails are the metadata the NSA originally claimed to be outside the purview of the fourth amendment as they are not the actual data itself. If you believe that, I have a lottery prize for you to claim.

Sunday, August 11, 2013

Privacy, Security and Sympathy

I don’t want anyone to feel sorry for me today, Friday July 19, as I sit on my deck, temperature 95 in the shade, sweating out yet one more column. You can if you wish but I don’t want you to. I do this not only to let my editor know that I don’t always write the column the night before but to call your attention to the fact that the Privacy vs Security issue that I discussed in last month’s column is a news story with legs (that’s what we in the news biz call “persistence”) and I want to continue it in this column even though you won’t be reading it until August 11 when, if the current trend continues, the temperatures will hover in the 135 degree range. Or not --- they could be in the thirties and I would not be surprised.

To mix a metaphor, lots more words have flowed under the bridge/over the dam since we first learned from Edward Snowden’s leaks to  the Manchester Guardian about how the NSA was, if you’re of the liberal or libertarian persuasion, invading our right to privacy and if you’re not then “enhancing our security” might be the way you would describe it. In either case,  Snowden’s actions may have the unintended side effect of uniting the left and the right, the liberals and the conservatives, the Democrats and the Republicans in Congress under the banner of “Freedom to be Left Alone”.“

By the way, I just want all my readers to know that I’ve decided to move me, my fan and iced coffee from the deck down to the basement where it’s a frosty 80 degrees and I should  be able to concentrate better.  Please, please do not worry about me. I’ll be fine. I’m fine, really. Ok, where was I? Ah, yes,, the story with legs.

The Aug 1 edition of the PR published an AP story that the surveillance had “3 hops” to it. That is, NSA could, if they felt it was necessary, search your phone calls for up to 40 people and, for each of those 40, 40 more, and for each of those 1600 oblivious souls, 40 more --- resulting in up to “12.5 million” searches..By my calculation: 40 + (40x40) + (40x40x40) is only 65,640 searches --- nowhere near the 12.5 million but still enough to get excited about.

If you feel that your privacy is in jeopardy, there are several things you can do:

Sign an online petition against this surveillance  at: https://secure.38degrees.org.uk/page/s/stop-government-snooping#petition or at:

If you are web-savvy you can use a proxy or anonymous remailer service.

You can use self-destructing emails similar to the mobile photo apps http://news.cnet.com/8301-17938_105-57591562-1/this-e-mail-will-self-destruct-in-five-seconds/?part=rss but this app may still be in the review.

In more recent developments, on July 24, 2013, Representative  Justin Amash, a Republican from Michigan presented the House this question: “Do you oppose the suspicionless collection of the phone records of every American? “.

Here's how they answered: 94 Republicans and 111  Democrats agreed (they voted Yes) while  134 Republicans and 83 Democrats disagreed (they voted No). So the vote was 205 For and 217 Against (12 members did not vote).(http://clerk.house.gov/evs/2013/roll412.xml). It’s easy to conclude that if our Representatives truly represent us there is no consensus on this question.
While I basically oppose dragnets, I have mixed feelings on this issue. When I receive an automated call from my credit card company questioning purchases I have made that do not fit my spending “profile” should I be enraged that my privacy has been violated or thankful that they’re looking out for me?  Frankly, I’m thankful because I know about and have agreed to this policy. In the case of the NSA vs Snowden, to be honest, I was not surprised but I was disturbed that a government agency, even a spy agency, is spying on me under a “law” that I didn’t know existed.. One of the hallmarks of a representative democracy is transparency, is it not?

Search This Blog