How are you going to deduce the encryption key, Holmes?
December 12, 2011 11:04 AM Subscribe
Espionage/Police Procedural Filter: how do the police, or CIA, or FBI, MI5, MI6, private eyes, or whoever do their job in a world of encryption, cloud storage, and teeny tiny little storage devices?
Lots of classic detective fiction hinges on finding some stolen letter or photo--think Sherlock Holmes in A Scandal in Bohemia or Poe's Purloined Letter, and countless others, but those came to mind immediately.
I loved the Sherlock reboot, and I was surprised to see that one of the episodes for its second season is "A Scandal in Belgravia" (which I assume is an update of A Scandal in Bohemia).
But how? If Irene Adler from the Holmes story were to copy a digitized image--hell, an entire encyclopedia and a 30-minute incriminating video--to a 2gb Micro-SD card that's smaller than my thumbnail, encrypt it with a 256-bit Blowfish key (or whatever), and THEN hide that Micro-SD card in any of 10 million physical locations, how on earth would Holmes get anywhere on the case?
I hasten to add that this question is not just about that one episode of a TV show. Rather, can anyone give a rundown of what the police do when they are searching for incriminating digital files? Or when the FBI or CIA are looking for spies with top secret stuff? When--literally--every single book I've ever read can fit in a virtually uncrackable archive on a micro SD that's 1/4 the size of a postage stamp and can be hidden almost anywhere (to say nothing of storing somewhere on the cloud), how on earth does anything get done?
I think of the last scene in The Conversation where Hackman is searching for some tiny mic--but 1000x more impossible: not only do you have to find a Micro SD card, but even if you find it, it's encrypted to the Nth degree, and would take a supercomputer 1000 years to brute force the password.
This seems highly unsatisfying. Is this really how things work?
Lots of classic detective fiction hinges on finding some stolen letter or photo--think Sherlock Holmes in A Scandal in Bohemia or Poe's Purloined Letter, and countless others, but those came to mind immediately.
I loved the Sherlock reboot, and I was surprised to see that one of the episodes for its second season is "A Scandal in Belgravia" (which I assume is an update of A Scandal in Bohemia).
But how? If Irene Adler from the Holmes story were to copy a digitized image--hell, an entire encyclopedia and a 30-minute incriminating video--to a 2gb Micro-SD card that's smaller than my thumbnail, encrypt it with a 256-bit Blowfish key (or whatever), and THEN hide that Micro-SD card in any of 10 million physical locations, how on earth would Holmes get anywhere on the case?
I hasten to add that this question is not just about that one episode of a TV show. Rather, can anyone give a rundown of what the police do when they are searching for incriminating digital files? Or when the FBI or CIA are looking for spies with top secret stuff? When--literally--every single book I've ever read can fit in a virtually uncrackable archive on a micro SD that's 1/4 the size of a postage stamp and can be hidden almost anywhere (to say nothing of storing somewhere on the cloud), how on earth does anything get done?
I think of the last scene in The Conversation where Hackman is searching for some tiny mic--but 1000x more impossible: not only do you have to find a Micro SD card, but even if you find it, it's encrypted to the Nth degree, and would take a supercomputer 1000 years to brute force the password.
This seems highly unsatisfying. Is this really how things work?
I'd imagine the answer is very similar to the way you or I conduct electronic discovery. We get a massive dump of an unbelievable amount of files, and we start sorting through it. Search engines can be made to go through file partitions. If we still come up with a huge amount of stuff that's potentially relevant, well, that's why paralegals don't get paid as much as lawyers do, and why digital forensics firms exist.
This, of course, assumes that no encryption is being used, but other than that, the principle seems the same. And Holmes might well be able to deduce the password anyway, given how sorry most people's passwords actually are.
One thing to remember here is that if it's impossible to find a relevant file, it's going to be a pain in the neck for the authorized user too. It may be possible to completely swamp the signal with noise, but that reduces the usefulness of the information for all users. If there's a shortcut or something for finding what you want, someone will figure it out. Especially someone like Holmes.
posted by valkyryn at 11:16 AM on December 12, 2011 [1 favorite]
This, of course, assumes that no encryption is being used, but other than that, the principle seems the same. And Holmes might well be able to deduce the password anyway, given how sorry most people's passwords actually are.
One thing to remember here is that if it's impossible to find a relevant file, it's going to be a pain in the neck for the authorized user too. It may be possible to completely swamp the signal with noise, but that reduces the usefulness of the information for all users. If there's a shortcut or something for finding what you want, someone will figure it out. Especially someone like Holmes.
posted by valkyryn at 11:16 AM on December 12, 2011 [1 favorite]
Most crimes have evidence that can't be digitally stored (murder weapons, for example). There is, of course, other evidence (used to establish motive, alibi etc.) that can be very important to the case, but it is important to remember that there are many pieces to a crime puzzle and only a few can be hidden digitally.
Espionage is a different sort of game, but similar principles apply: even if a particular document is really, really unobtainable, there are other ways to get close to the information you need.
posted by vidur at 11:19 AM on December 12, 2011 [1 favorite]
Espionage is a different sort of game, but similar principles apply: even if a particular document is really, really unobtainable, there are other ways to get close to the information you need.
posted by vidur at 11:19 AM on December 12, 2011 [1 favorite]
Or when the FBI or CIA are looking for spies with top secret stuff?
A lot of it is just people being bad about security, like you would find in any organization. For example, in espionage one of the basic security principles is that temporary sensitive documents should be destroyed after they are no longer needed. So if a spy gets his orders from a secret encrypted physical document and decrypts it by hand, they should memorize the message and destroy both the encrypted document and any plaintext they created. But there have been plenty of cases where a spy has been found with these kinds of documents that should have been destroyed, because they wanted to keep the messages to refer to later or for other reasons. In that case it's a completely unbeatable security scheme if implemented properly, because you can't crack data that no longer exists, but a great security scheme does little to no good if it's not implemented correctly. A lot of security in general comes down to these sorts of things, because you can use as much encryption and secrecy as you like as the core of your scheme, but any weak link in the chain can be exploited to defeat the security, and a lot of times the weakest link a person or people who have the data or are in charge or running the scheme.
posted by burnmp3s at 11:24 AM on December 12, 2011 [2 favorites]
A lot of it is just people being bad about security, like you would find in any organization. For example, in espionage one of the basic security principles is that temporary sensitive documents should be destroyed after they are no longer needed. So if a spy gets his orders from a secret encrypted physical document and decrypts it by hand, they should memorize the message and destroy both the encrypted document and any plaintext they created. But there have been plenty of cases where a spy has been found with these kinds of documents that should have been destroyed, because they wanted to keep the messages to refer to later or for other reasons. In that case it's a completely unbeatable security scheme if implemented properly, because you can't crack data that no longer exists, but a great security scheme does little to no good if it's not implemented correctly. A lot of security in general comes down to these sorts of things, because you can use as much encryption and secrecy as you like as the core of your scheme, but any weak link in the chain can be exploited to defeat the security, and a lot of times the weakest link a person or people who have the data or are in charge or running the scheme.
posted by burnmp3s at 11:24 AM on December 12, 2011 [2 favorites]
The common or garden variety criminal is still caught for the same reason they have been caught in the past -- they make a mistake, often of assuming that they are smarter than the police. Most people use weak passwords, and when they don't, they write them down somewhere so they won't forget them. Sure, you don't put the label "password for super-secret USB stick" on the Post-It that contains the password for your super-secret USB stick." Doesn't matter. The police just take every Post-It on your monitor and try every word or phrase on each of them to see if it's the password. They have lots of people and lots of time and are very strongly motivated to find the evidence.
Also, criminals regularly believe that because they haven't been arrested that the authorities haven't noticed their activities at all, when in reality they may be under suspicion and the police have already installed a rootkit on their computer and a tap on their phone and are collecting all their communications as evidence.
That's without having partners or rivals who will turn you in for the right incentive, or facing a detective who is very good at noticing small signs (such as Holmes) or even simply bluffing.
It's similar to the lion vs. the gazelle: the lion is running for his dinner, while the gazelle is running for his life. The police can miss most of the clues and still get their man, while the criminal must successfully hide every single clue to be sure he'll get away.
posted by kindall at 11:25 AM on December 12, 2011 [2 favorites]
Also, criminals regularly believe that because they haven't been arrested that the authorities haven't noticed their activities at all, when in reality they may be under suspicion and the police have already installed a rootkit on their computer and a tap on their phone and are collecting all their communications as evidence.
That's without having partners or rivals who will turn you in for the right incentive, or facing a detective who is very good at noticing small signs (such as Holmes) or even simply bluffing.
It's similar to the lion vs. the gazelle: the lion is running for his dinner, while the gazelle is running for his life. The police can miss most of the clues and still get their man, while the criminal must successfully hide every single clue to be sure he'll get away.
posted by kindall at 11:25 AM on December 12, 2011 [2 favorites]
the DOJ in july tried to claim that keeping your encryption password to yourself isn't an option. they're arguing that they can't force you to hand it over, but that the 5th amendment doesn't protect against them forcing you to type it in.
beyond that, i imagine detective stories will continue to hinge on human error. i can't find the article, but sometime in the last couple of years a lawyer was brought up on child porn charges because his unencrypted USB stick fell out of his pocket without him noticing.
posted by nadawi at 11:42 AM on December 12, 2011 [1 favorite]
beyond that, i imagine detective stories will continue to hinge on human error. i can't find the article, but sometime in the last couple of years a lawyer was brought up on child porn charges because his unencrypted USB stick fell out of his pocket without him noticing.
posted by nadawi at 11:42 AM on December 12, 2011 [1 favorite]
As others have alluded to, the weak link is almost always meat-based, not bit-based. I'm a bit surprised that this XKCD hasn't come up yet. :-) It's humorous, but exposes the rather profound truth that any weak link at all in an encryption scheme (really, any system for hiding or obscuring information) results in the entire scheme becoming more-or-less useless.
If anyone knows the encryption key or some other way of getting at the information, they're vulnerable to blackmail, extortion, torture, or other form of coercion. The police forces and courts in the UK already exploit this. It is a punishable crime in the UK simply to refuse to turn over encryption keys: Link
posted by jdwhite at 11:43 AM on December 12, 2011
If anyone knows the encryption key or some other way of getting at the information, they're vulnerable to blackmail, extortion, torture, or other form of coercion. The police forces and courts in the UK already exploit this. It is a punishable crime in the UK simply to refuse to turn over encryption keys: Link
posted by jdwhite at 11:43 AM on December 12, 2011
The eavesdropping side has become stronger as well.
If the cryptography is too strong to brute force (it almost always is), use side channel attacks. The power consumption of a processor, the sounds a keyboard makes when someone is typing a password or reflections of the screen content on mugs or glasses can all be used to gain information about the data someone is trying to hide.
In the time of Holmes, the processing power wasn't available to squeeze information from these signals, now it is.
With the right equipment, it is possible to catch the electromagnetic waves created by monitors or keyboards and watch the screen or inputs from afar.
Additionally, social media opens up new attack vectors. If the target is using these, it is now easier to impersonate acquaintances or discern routines, which in turn can be used for either social engineering or infecting someone's computer with keyloggers or rootkits.
posted by Triton at 11:45 AM on December 12, 2011
If the cryptography is too strong to brute force (it almost always is), use side channel attacks. The power consumption of a processor, the sounds a keyboard makes when someone is typing a password or reflections of the screen content on mugs or glasses can all be used to gain information about the data someone is trying to hide.
In the time of Holmes, the processing power wasn't available to squeeze information from these signals, now it is.
With the right equipment, it is possible to catch the electromagnetic waves created by monitors or keyboards and watch the screen or inputs from afar.
Additionally, social media opens up new attack vectors. If the target is using these, it is now easier to impersonate acquaintances or discern routines, which in turn can be used for either social engineering or infecting someone's computer with keyloggers or rootkits.
posted by Triton at 11:45 AM on December 12, 2011
I work in digital forensics in the UK. Primarily with mobile phones, but since most of them have MicroSD cards these days, I examine those also. I also know a fair amount about what my colleagues upstairs with the computers do.
The primary recourse for us is the owner of the device. Many people, when arrested, think they may have safely covered their tracks - or are indeed innocent - and will provide their passcodes. For those that know they are guilty, the Regulation of Investigatory Powers Act 2000 includes the ability to jail suspects for non-provision of passwords for (I believe) up to 3 years. In this time they can be interviewed to obtain the passcodes again or brute force attacks can be used on the encrypted containers.
In mobile devices, the situation varies hugely. Few mobile handsets outside of RIM's BlackBerry range routinely employ encryption. Most locks can be bypassed with the right tools. Even BlackBerry handsets are, with the right tools, potentially at the mercy of the forensic/intelligence expert.
As far as procedures for investigating devices go, that's far too complex for me to put into a MeFi post, but you could look at the materials provided by the developers of major digital forensics tools such as FTK by AccessData and EnCase by Guidance Software.
In my experience, if a suspect has incriminating evidence on a digital device, the problem is not with finding it, but proving how it got there. Sure, you can encrypt all your illegal porn in an uncrackable container, but Windows LOVES making copies of things all over the damn place, and unless you know where all of them are, we'll find them.
posted by fearnothing at 12:37 PM on December 12, 2011 [6 favorites]
The primary recourse for us is the owner of the device. Many people, when arrested, think they may have safely covered their tracks - or are indeed innocent - and will provide their passcodes. For those that know they are guilty, the Regulation of Investigatory Powers Act 2000 includes the ability to jail suspects for non-provision of passwords for (I believe) up to 3 years. In this time they can be interviewed to obtain the passcodes again or brute force attacks can be used on the encrypted containers.
In mobile devices, the situation varies hugely. Few mobile handsets outside of RIM's BlackBerry range routinely employ encryption. Most locks can be bypassed with the right tools. Even BlackBerry handsets are, with the right tools, potentially at the mercy of the forensic/intelligence expert.
As far as procedures for investigating devices go, that's far too complex for me to put into a MeFi post, but you could look at the materials provided by the developers of major digital forensics tools such as FTK by AccessData and EnCase by Guidance Software.
In my experience, if a suspect has incriminating evidence on a digital device, the problem is not with finding it, but proving how it got there. Sure, you can encrypt all your illegal porn in an uncrackable container, but Windows LOVES making copies of things all over the damn place, and unless you know where all of them are, we'll find them.
posted by fearnothing at 12:37 PM on December 12, 2011 [6 favorites]
Also, on the mobile phone side of things, the number of people who use their day/month or year of birth for their 4-digit password is scarily high. So, Holmes might see that the owner's calendar has July 23rd circled and the legend "My 50th" and from there deduce that the unlock code for the iPhone of the murder suspect is either 2307 or 1962.
posted by fearnothing at 12:51 PM on December 12, 2011 [1 favorite]
posted by fearnothing at 12:51 PM on December 12, 2011 [1 favorite]
Most people use crap passwords, and worse yet, reuse them. There is software available that will literally take everything on your hard drive and try using it to open an encrypted file. So if you have ever written that password down anywhere in unencrypted form, or if it was taken from a file (an ebook, say), or if it's used as a password to a website and stored in your browser's credential cache (unless you encrypt it, which most people don't), etc., it could be lifted fairly easily.
That would crack open files encrypted by all but the most paranoid users; the only way around it is to use a completely new, high-entropy password, generated using a method that you're sure doesn't leave any traces on the hard drive that could be recovered later. Most people don't go to nearly that much effort.
posted by Kadin2048 at 1:38 PM on December 12, 2011 [1 favorite]
That would crack open files encrypted by all but the most paranoid users; the only way around it is to use a completely new, high-entropy password, generated using a method that you're sure doesn't leave any traces on the hard drive that could be recovered later. Most people don't go to nearly that much effort.
posted by Kadin2048 at 1:38 PM on December 12, 2011 [1 favorite]
Response by poster: This is all sorts of fascinating--thanks to all so far.
Nadawi--that case looks really interesting; jdwhite, the UK law is interesting too.
I think the NSA / spy angle was probably a wild goose chase in my question, since I'm sure there are all sorts of things they can do that Holmes or your average PD can't. I'm imagining more of The Wire-level of crimes and sophistication. While I think Avon Barkesdale would be sophisticated enough to use encryption to protect his Master File of Criminal Doings (and, let's assume, sophisticated enough to use a strong key, and not just his birthday or something), I don't think the Baltimore PD has access to the kinds of technologies that would be required to break the key (yet).
posted by Admiral Haddock at 1:40 PM on December 12, 2011
Nadawi--that case looks really interesting; jdwhite, the UK law is interesting too.
I think the NSA / spy angle was probably a wild goose chase in my question, since I'm sure there are all sorts of things they can do that Holmes or your average PD can't. I'm imagining more of The Wire-level of crimes and sophistication. While I think Avon Barkesdale would be sophisticated enough to use encryption to protect his Master File of Criminal Doings (and, let's assume, sophisticated enough to use a strong key, and not just his birthday or something), I don't think the Baltimore PD has access to the kinds of technologies that would be required to break the key (yet).
posted by Admiral Haddock at 1:40 PM on December 12, 2011
I believe this xkcd answers your question.
Sherlock Holmes would cleverly guess where the encrypted disk was hidden and what the password is. There are other attacks on encryption, based on things like guessing what some of the encrypted text is or if a weak pseudo-random number generator was used. But if Moriarty had followed all the best security practices, Holmes would be out of luck. The NSA is probably about 20 years ahead of current published mathematics, so maybe they would be able to break something like Blowfish, but maybe not.
posted by qxntpqbbbqxl at 1:59 PM on December 12, 2011
Sherlock Holmes would cleverly guess where the encrypted disk was hidden and what the password is. There are other attacks on encryption, based on things like guessing what some of the encrypted text is or if a weak pseudo-random number generator was used. But if Moriarty had followed all the best security practices, Holmes would be out of luck. The NSA is probably about 20 years ahead of current published mathematics, so maybe they would be able to break something like Blowfish, but maybe not.
posted by qxntpqbbbqxl at 1:59 PM on December 12, 2011
To summarize the comments above: data leaks at the seams. If it has to pass from one hand to another, one system to another, one place to another, from one encrypted state to another, then data is vulnerable to copying, to theft and to detection.
posted by Mo Nickels at 4:36 PM on December 12, 2011 [1 favorite]
posted by Mo Nickels at 4:36 PM on December 12, 2011 [1 favorite]
Any time the baddies actually use their encrypted data, it's vulnerable to screen-capping, TEMPEST, even shoulder surfing. And even if they used a smart password, there's always keystroke loggers, etc. And when that fails, you've got either a.) threats of jail time, b.) rubber hoses, and/or c.) rooms with floor drains.
posted by codswallop at 10:10 PM on December 12, 2011
posted by codswallop at 10:10 PM on December 12, 2011
oops jdwhite and qxntpqbbbqxl beat me to it...sorry
posted by lalochezia at 8:35 AM on December 13, 2011
posted by lalochezia at 8:35 AM on December 13, 2011
You may be interested in The Code Book which contains a history of cryptography.
+ 1 everyone who said that they don't necessarily try to crack the encryption or find the tiny little device - they deal with people wherever possible.
- Keyloggers are almost impossible to detect, all you need is a person on the inside.
- It was recently discovered that Android phones ship with a Rootkit that logs every swipe, every number dialed, every letter typed and sends it off without your knowledge.
posted by MesoFilter at 1:19 PM on December 14, 2011
+ 1 everyone who said that they don't necessarily try to crack the encryption or find the tiny little device - they deal with people wherever possible.
- Keyloggers are almost impossible to detect, all you need is a person on the inside.
- It was recently discovered that Android phones ship with a Rootkit that logs every swipe, every number dialed, every letter typed and sends it off without your knowledge.
posted by MesoFilter at 1:19 PM on December 14, 2011
This thread is closed to new comments.
More practically, and still thinking like a writer with a problem, here: you can encipher anything you wish to the nth degree, and I'll even grant you that it's truly "unbreakable", but so what? Weak points abound nonetheless:
What's the passphrase? What might it be? Who knows it? Can they be reached or coerced or threatened or otherwise compromised? Is it written down? Where was it last typed, and how? What else happened before and after on that computer that last used the key? Were there any cameras in view the last time the passphrase was typed? Audio recorders? Vibration detectors?
There are many ways around this problem without brute-force hacking.
posted by rokusan at 11:15 AM on December 12, 2011 [3 favorites]