Simson L. Garfinkel
Smokey


Flames are an environmental hazard of cyberspace. They sneak up on
you, drop into your mailbox, and---wham---explode across your screen
minutes before that big annual review meeting with your boss's boss.
Flames are everywhere, they're insidious, and everybody hates them.
And that's what drove Ellen Spertus to write Smokey, an AI-based system
for automatically detecting them.

Spertus created Smokey last summer while she was working at Microsoft
Research.  The system, which uses an experimental Microsoft
natural-language parser, a decision tree, and a bunch of a Lisp code,
can actually pick out the flames from ordinary email messages. This
lets you save them for reading them for reading at a later time ... or
never at all.

For raw material, Spertus obtained messages sent to the webmasters of
three controversial Internet sites---Fairness and Accuracy in
Reporting (http://www.fair.org), NewtWatch
(http://www.cais.com/newtwatch), and The Right Side of the Web
(http://www.clark.net/pub/jeffd/index.html). With the raw data in
hand, she quickly learned that a one site's flame was another site's
fan mail. Her solution was to develop a set of linguistic rules for
detecting flames, and then a set of site-specific plug-in villains and
heroes. Thus, an email to NewtWatch bemoaning Slick Willy (President
Clinton) is quickly identified as a flame, whereas the same note sent
to The Right Side of the Web gets high marks for accuracy.

But even though flamers have different enemies, Spertus learned, they
do share a common grammar. For example, flamers tend to use noun
phrases as appositions---for example, "you bozos," "you flamers", and
"you people." (The phrase "you guys" was an exception to the rule.)
Likewise, flamers tend to use the word "get" followed within 10
character by the words "life," "lost," "real," "clue" "wit it," or
"used to it." Nearly a hundred such rules, firing in rapid succession,
help Smokey isolate the flames from the fan mail.
 
Smokey's isn't perfect, notes Spertus, who is currently finishing up a
PhD at the MIT Artificial Intelligence Laboratory. Because of spelling
errors, ambiguities in English, and the fact that the program really
doesn't understand English, Smokey fails to identify roughly half the
flames that it sees. The good news, though, is that the program has an
astonishingly low false positive rate---less than 1% of regular
messages are mis-identified as being incendiary. As for what to do with
the flames, Spertus recommends against deleting them or sending them
back to the author. Instead, she says, flames should just be flagged,
so that web master can mentally prepare him or herself before dipping
into the deluge.

Although working Smokey has been fun, says Spertus, what companies
really need right now are systems that can automatically route e-mail
to the right group within a large company or government agency. That
would let you send mail to an address like help@ibm.com and have a
prayer of the message ending up in the mailbox of the right person ...
or the right computer. 


=== 

Note to editor:

Ellen Spertus
ellens@ai.mit.edu