CS4920: Automated Document and Media Exploitation

Wed Oct 15, 2008

SQL and Lucene: Indexing human documents

Seminar 4 <<                     Seminar 5                    >> Seminar 6

• SQL
• Lucene
• Google Search
• Prefix arrays

Lesson Plan

• You will be given a set of files (file00.txt through file99.txt). Each file has words and email addresses. Among the email addresses are the email addresses of everyone in the class. We would like to answer the following questions:
  • In which file(s) does your email address appear?
  • How many NPS email addresses are in all of the files?
  • Which file has the most NPS email addresses?
  • Which email address(s) is in every file?
  • Which two files have the most email addresses in common?
• download the names and the make.py program

References

• SQLite 3 Data Types

Readings

• A Gentle Introduction to SQL
• Embedding an SQL database with SQLite, Michael Owens, Linux Journal, Volume 2003, Issue 110 (June 2003) Page 2
• Threats to privacy in the forensic analysis of database systems, International Conference on Management of Data archive, Proceedings of the 2007 ACM SIGMOD international conference on Management of data

Optional Readings

• The book Learning SQL is available trhough ACM Online Book Program and Safari (NPS has licenses for both)