Real Data Corpus - Revision history

Simson at 15:46, 23 October 2012

2012-10-23T15:46:28Z

← Older revision		Revision as of 08:46, 23 October 2012
Line 1:		Line 1:
	The Real Data Corpus (RDC) is a collection of raw data extracted from data-carrying devices that were purchased on the secondary market around the world. Many studies have shown that hard drives, cell phones, USB memory sticks, and other data-carrying devices are frequently discarded by their original users without the data first being cleared or purged. By purchasing these devices and extracting their data, we have created a data set that closely mimics data as it is found in the real world.		The Real Data Corpus (RDC) is a collection of raw data extracted from data-carrying devices that were purchased on the secondary market around the world. Many studies have shown that hard drives, cell phones, USB memory sticks, and other data-carrying devices are frequently discarded by their original users without the data first being cleared or purged. By purchasing these devices and extracting their data, we have created a data set that closely mimics data as it is found in the real world.

	Potential Uses		Potential Uses

Simson: /* Current Contents */

2011-02-21T15:33:58Z

Current Contents

← Older revision		Revision as of 08:33, 21 February 2011
Line 14:		Line 14:

	==Current Contents==		==Current Contents==
	As of ~~August 1~~, ~~2008~~, the Corpus consists of the following:		As of February 21, 2011, the Non-US Person's Corpus consists of the following:
	* ~~Approximately 2000~~ hard drive images ranging in size from 500MB to 80GB.		* 1,289 hard drive images ranging in size from 500MB to 80GB.
	* ~~Approximately 800~~ flash memory images (USB, Sony Memory Stick, SD and other), ranging from 128MB to 4GB.		* 643 flash memory images (USB, Sony Memory Stick, SD and other), ranging from 128MB to 4GB.
	* ~~Approximately 10 digital camera memory images~~.		* 98 CDROMs
	* Approximately 40 GSM SIM chip memory images.		For a total of 36TB of data (uncompressed).

	==Access and Availability==		==Access and Availability==
	Real Data Corpus can be distributed to sponsors and collaborators as a set of encrypted AFF files. Encryption is with AES 256 and can be based on either a pass phrase or X.509 PKI using AFF encryption:		Real Data Corpus can be distributed to sponsors and collaborators as a set of encrypted AFF files. Encryption is with AES 256 and can be based on either a pass phrase or X.509 PKI using AFF encryption:

Simson at 12:42, 8 February 2011

2011-02-08T12:42:24Z

← Older revision		Revision as of 05:42, 8 February 2011
Line 8:		Line 8:
	* Exploring and characterizing real-world computing practices, configuration choices, and option settings.		* Exploring and characterizing real-world computing practices, configuration choices, and option settings.
	* Studying the storage allocation strategies of file systems under real-world conditions		* Studying the storage allocation strategies of file systems under real-world conditions

			For more information about the Real Data Corpus, please see:
			* [http://digitalcorpora.org/corpora/disk-images Disk Images] on the [http://digitalcorpora.org DigitalCorpora.org] website.
			* [http://digitalcorpora.org/corpora/disk-images/rdc-faq Real Data Corpus FAQ]

	==Current Contents==		==Current Contents==
	As of August 1, 2008, the Corpus consists of the following:		As of August 1, 2008, the Corpus consists of the following:

Simson: 4 revisions

2009-05-03T07:11:56Z

4 revisions

← Older revision	Revision as of 00:11, 3 May 2009
(No difference)

Simson at 17:49, 21 April 2009

2009-04-21T17:49:05Z

@@ Line 1: / Line 1: @@
-In 2005 more than 210 million hard drives were retired from primary service. Although many of these drives were destroyed, many are repurposed within organizations, donated to charities, or sold on the secondary market.
+The Real Data Corpus (RDC) is a collection of raw data extracted from data-carrying devices that were purchased on the secondary market around the world. Many studies have shown that hard drives, cell phones, USB memory sticks, and other data-carrying devices are frequently discarded by their original users without the data first being cleared or purged. By purchasing these devices and extracting their data, we have created a data set that closely mimics data as it is found in the real world.
-We are purchasing a statistically-significant quantity of used hard drives on the secondary market and analyzing the content of these drives. Many of the drives contain remnant information from their previous users --- in many cases no effort whatsoever has been made to purge this data from the drives. To date we have purchased more than 2000 hard drives on the secondary market and have created a unique resource with more than a terabyte of compressed disk images.
+The Real Data Corpus is a one-of-a-kind scientific resource for:
+* Developing and validating forensic and data recovery tools.
 We have used this unique resource for a variety of research purposes:
 # Our first publication <ref>S. Garfinkel. and A. Shelat. "Remembrance of Data Passed: A Study of Disk Sanitization Practices," IEEE Security and Privacy, January/February 2003. </ref> alerted the community to the scale of the problem of data on repurposed hard drives. Following this publication, the US government passed legislation creating an affirmative responsibility on the part of American businesses to purge consumer information from hard drives before discarding them <ref> US Congress. Fair and Accurate Credit Transactions Act of 2003</ref>, and two major products were introduced that use cryptography to rapidly "shred" information on stored magnetic media <ref> Seagate Technology. Momentus Family Overview, 2006</ref><ref> Decru. DataFort Security Appliances, 2005</ref>.
@@ Line 13: / Line 31: @@
 ==References==
 <references/>

Simson at 20:12, 12 April 2009

2009-04-12T20:12:46Z

← Older revision		Revision as of 13:12, 12 April 2009
Line 10:		Line 10:
	# We have used the corpus to evaluate the effectiveness of today's computer forensic tools.		# We have used the corpus to evaluate the effectiveness of today's computer forensic tools.
	# We are using the corpus to create new computer forensic tools and techniques.		# We are using the corpus to create new computer forensic tools and techniques.

			==References==
			<references/>

Simson at 20:12, 12 April 2009

2009-04-12T20:12:26Z

← Older revision		Revision as of 13:12, 12 April 2009
Line 2:		Line 2:

	We are purchasing a statistically-significant quantity of used hard drives on the secondary market and analyzing the content of these drives. Many of the drives contain remnant information from their previous users --- in many cases no effort whatsoever has been made to purge this data from the drives. To date we have purchased more than 2000 hard drives on the secondary market and have created a unique resource with more than a terabyte of compressed disk images.		We are purchasing a statistically-significant quantity of used hard drives on the secondary market and analyzing the content of these drives. Many of the drives contain remnant information from their previous users --- in many cases no effort whatsoever has been made to purge this data from the drives. To date we have purchased more than 2000 hard drives on the secondary market and have created a unique resource with more than a terabyte of compressed disk images.

	We have used this unique resource for a variety of research purposes:		We have used this unique resource for a variety of research purposes:
	# Our first publication <ref>S. Garfinkel. and A. Shelat. "Remembrance of Data Passed: A Study of Disk Sanitization Practices," IEEE Security and Privacy, January/February 2003. </ref> alerted the community to the scale of the problem of data on repurposed hard drives. Following this publication, the US government passed legislation creating an affirmative responsibility on the part of American businesses to purge consumer information from hard drives before discarding them <ref> US Congress. Fair and Accurate Credit Transactions Act of 2003</ref>, and two major products were introduced that use cryptography to rapidly "shred" information on stored magnetic media <ref> Seagate Technology. Momentus Family Overview, 2006</ref><ref> Decru. DataFort Security Appliances, 2005</ref>.		# Our first publication <ref>S. Garfinkel. and A. Shelat. "Remembrance of Data Passed: A Study of Disk Sanitization Practices," IEEE Security and Privacy, January/February 2003. </ref> alerted the community to the scale of the problem of data on repurposed hard drives. Following this publication, the US government passed legislation creating an affirmative responsibility on the part of American businesses to purge consumer information from hard drives before discarding them <ref> US Congress. Fair and Accurate Credit Transactions Act of 2003</ref>, and two major products were introduced that use cryptography to rapidly "shred" information on stored magnetic media <ref> Seagate Technology. Momentus Family Overview, 2006</ref><ref> Decru. DataFort Security Appliances, 2005</ref>.

Simson: New page: In 2005 more than 210 million hard drives were retired from primary service. Although many of these drives were destroyed, many are repurposed within organizations, donated to charities, o...

2009-04-12T20:05:00Z

New page: In 2005 more than 210 million hard drives were retired from primary service. Although many of these drives were destroyed, many are repurposed within organizations, donated to charities, o...

New page

In 2005 more than 210 million hard drives were retired from primary service. Although many of these drives were destroyed, many are repurposed within organizations, donated to charities, or sold on the secondary market.

We are purchasing a statistically-significant quantity of used hard drives on the secondary market and analyzing the content of these drives. Many of the drives contain remnant information from their previous users --- in many cases no effort whatsoever has been made to purge this data from the drives. To date we have purchased more than 2000 hard drives on the secondary market and have created a unique resource with more than a terabyte of compressed disk images.

We have used this unique resource for a variety of research purposes:
# Our first publication <ref>S. Garfinkel. and A. Shelat. "Remembrance of Data Passed: A Study of Disk Sanitization Practices," IEEE Security and Privacy, January/February 2003. </ref> alerted the community to the scale of the problem of data on repurposed hard drives. Following this publication, the US government passed legislation creating an affirmative responsibility on the part of American businesses to purge consumer information from hard drives before discarding them <ref> US Congress. Fair and Accurate Credit Transactions Act of 2003</ref>, and two major products were introduced that use cryptography to rapidly "shred" information on stored magnetic media <ref> Seagate Technology. Momentus Family Overview, 2006</ref><ref> Decru. DataFort Security Appliances, 2005</ref>.
# We conducted a "trace-back" study in which 20 organizations were contacted who had data on the hard drives that we obtained. Based on interviews, we were able to identify the technical and organizational failures that resulted in the data compromises.<ref>S. Garfinkel. "Design Principles and Patterns for Computer Systems that are Simultaneously Secure and Usable," PhD Thesis, Massachusetts Institute of Technology, June 2005</ref>
# We have identified patterns and principles for promoting secure human-computer interaction.
# As part of developing this resource, we have developed a new file format for storing disk images<ref> S. Garfinkel. "AFF: A New Format for Storing Hard Drive Iamges," Communications of the ACM, February, 2006</ref><ref>S. Garfinkel and D. Malan and K. Dubec and C. Stevens and C. Pham. "Disk Imaging with the Advanced Forensics Format, Library and Tools," The Second Annual IFIP WG 11.9 International Conference on Digital Forensics, National Center for Forensic Science, Orlando, Florida, USA January 29 - February 1 2006. </ref>, and we are developing a new technique for mapping social networks among individuals whose data is on captured hard drives. These approaches could be used, for example, to allow the rapid and automated analysis of disk drives seized during the course of a police investigation or obtained as part of military operations.
# We have used the corpus to evaluate the effectiveness of today's computer forensic tools.
# We are using the corpus to create new computer forensic tools and techniques.