From nude pictures of celebrities to politicians caught in compromising positions, verifying the authenticity of images online is often no easy task. To address this problem, a team at Duke University looking has developed software called YouProve that can be integrated into the Android operating system to track changes made to images or audio captured on an Android smartphone. The software then produces a non-forgeable "fidelity certificate" that uses a "heat-map" to summarize the degree to which various regions of the media have been modified compared to the original image.
Landon Cox, a computer scientist at Duke, and students Peter Gilbert, Henry Qin, Kyungmin Lee and DJ Sharkey collaborated with Jaeyeon Jung of Microsoft Research and Anmol Sheth of Technicolor Research to design YouProve. They altered the Android operating system so that it keeps copies of images or audio clips that are opened in apps, such as Facebook and Photoshop Express for Android, and then tracks what modifications the app makes.
If the app saves a modified version of the media to a file on a phone or sends it over a wireless network, YouProve compares the original data to the modified one using advanced audio and image analysis algorithms. It then produces a fidelity certificate that identifies and displays to what degree various regions of the media have been preserved or modified.
Cox says the fidelity certificates are produced using emerging tamper-resistant, "trusted" hardware on mobile devices that guarantees they are generated securely and cannot be fabricated. He says that the hardware is a standard feature on PCs and new smartphones but it remains largely unused on both platforms.
The fidelity certificate, along with the modified media, can then be posted online so individuals or citizen reporting services, such as CNN iReports or Al Jazeera's Sharek, can check a photo's authenticity. Opening the file in YouProve's online Photo Analysis Visualizer produces a "heat-map" showing the extent to which various areas of the image differ from the original.
"With the Arab Spring and the Iranian protests in 2009, we relied on citizen journalists for information," said Cox. "But as crowd-sourced content plays an increasingly important role in world affairs, falsified media could have severe consequences. It's important that we make sure the information we are getting is accurate."
With the software monitoring media files for compression, cropping and blurring, it is able to complete an analysis of a five-minute audio clip in less than 70 seconds and a 5-megapixel photo in under 30 seconds without interrupting simultaneous use of other applications on the phone. In tests, the developers say that YouProve correctly identified edited regions of photos or audio clips with an accuracy of 99 percent.
For YouProve to be deployed on smartphones and other devices, manufacturers will need to make their devices' trusted hardware accessible to the software. Cox is optimistic this will happen in the near future.
Cox and his collaborators presented their results on YouProve at the Association for Computing Machinery Conference on Embedded Networked Sensor Systems in Seattle earlier this month. The related paper, "YouProve: Authenticity and Fidelity in Mobile Sensing," can be downloaded here.
Here's a short video showing how YouProve works.
