The Wall Street Journal published an article today citing a source at the FBI is planning to tell the White House that “it knows so little about the hacking tool that was used to open terrorist’s iPhone that it doesn’t make sense to launch an internal government review”. If true, this should be taken as an act of recklessness by the FBI with regards to the Syed Farook case: The FBI apparently allowed an undocumented tool to run on a piece of high profile, terrorism-related evidence without having adequate knowledge of the specific function or the forensic soundness of the tool.
Best practices in forensic science would dictate that any type of forensics instrument needs to be tested and validated. It must be accepted as forensically sound before it can be put to live evidence. Such a tool must yield predictable, repeatable results and an examiner must be able to explain its process in a court of law. Our court system expects this, and allows for tools (and examiners) to face numerous challenges based on the credibility of the tool, which can only be determined by a rigorous analysis. The FBI’s admission that they have such little knowledge about how the tool works is an admission of failure to evaluate the science behind the tool; it’s core functionality to have been evaluated in any meaningful way. Knowing how the tool managed to get into the device should be the bare minimum I would expect anyone to know before shelling out over a million dollars for a solution, especially one that was going to be used on high-profile evidence.
A tool should not make changes to a device, and any changes should be documented and repeatable. There are several other variables to consider in such an effort, especially when imaging an iOS device. Apart from changes made directly by the tool (such as overwriting unallocated space, or portions of the file system journal), simply unlocking the device can cause the operating system to make a number of changes, start background tasks which could lead to destruction of data, or cause other changes unintentionally. Without knowing how the tool works, or what portions of the operating system it affects, what vulnerabilities are exploited, what the payload looks like, where the payload is written, what parts of the operating system are disabled by the tool, or a host of other important things – there is no way to effectively measure whether or not the tool is forensically sound. Simply running it against a dozen other devices to “see if it works” is not sufficient to evaluate a forensics tool – especially one that originated from a grey hat hacking group, potentially with very little actual in-house forensics expertise.
It is highly unlikely that any agency could effectively evaluate the forensic soundness of any tool without having an understanding of how it works. The FBI’s arguments to the White House with regards to this appear to many as an attempt to simply skirt the vulnerabilities equities process.
There are only two possible conclusions to draw from all of this: either the FBI is lying to the White House (misleading the President), and actually does possess enough knowledge about the tool to warrant a review, or the FBI never evaluated and validated the safety of this tool, never learned how it worked, and recklessly used it on a piece of terrorism-related evidence so high profile that it warranted an egregious abuse of the constitution when ordering Apple to assist… yet was so inconsequential a piece of evidence that the FBI didn’t have a problem running an ordinary jailbreak tool on it. This would not fall short of misleading the court.
The Syed Farook case has been wrought with recklessness. Numerous mistakes were made early on, as I’ve written about, such as changing the iCloud password and possibly even powering down the device (or letting it die), locking the encryption. When the FBI demonstrated that only a mere 30 days was necessary in order to get into the iPhone, many interpreted this as proof that adequate due diligence had not been done prior to filing for an All Writs Act order against Apple. Beyond this case, the FBI has pulled out of their NY iPhone case after the passcode was given to them – further suggesting the FBI’s unwillingness or inability to do their job, to the degree of abusing the All Writs Act as an alternative to good police work.
Ironically, the NY case highlighted the DOJ’s reluctance to use an undocumented hacking tool named IP-BOX, which was essentially a “black box” to brute force PINs on iOS 7/8 devices, and listed as one major reason Apple’s help was required. Ironically, the FBI is claiming to have done the very thing here that they argued they shouldn’t do with regards to the NY case: Use an undocumented, opaque hacking tool that they were unable to fully understand. It would seem that situation ethics are in play here.
This sets a dangerous practice in motion: The FBI has offered this tool to any other law enforcement agencies that need it. So the FBI is endorsing the use of an untested tool that they have no idea how it works, for every kind of case that could go through our court system. A tool that was also only tested, if at all, for one very specific case now being used on a very broad set of types of data and evidence, which it could easily damage, alter, or – more likely – see thrown out of cases as soon as it’s challenged. If the FBI truly does not know how this tool works, they’ve created an extremely dangerous situation for all of us.