Mail::SpamAssassin::Plugin::ExtractText - extracts text from documenmts.
NAME SYNOPSIS DESCRIPTION CONFIGURATION Tools MetadataNAME
ExtractText − extracts text from documenmts.
SYNOPSIS
loadplugin Mail::SpamAssassin::Plugin::ExtractText
ifplugin Mail::SpamAssassin::Plugin::ExtractText
extracttext_external
pdftotext /usr/bin/pdftotext −nopgbrk −layout
−enc UTF−8 {} −
extracttext_use pdftotext .pdf application/pdf
# http://docx2txt.sourceforge.net
extracttext_external docx2txt /usr/bin/docx2txt {} −
extracttext_use docx2txt .docx application/docx
extracttext_external antiword /usr/bin/antiword −t
−w 0 −m UTF−8.txt {}
extracttext_use antiword .doc
application/(?:vnd\.?)?ms−?word.*
extracttext_external unrtf /usr/bin/unrtf
−−nopict {}
extracttext_use unrtf .doc .rtf application/rtf text/rtf
extracttext_external odt2txt /usr/bin/odt2txt
−−encoding=UTF−8 {}
extracttext_use odt2txt .odt .ott
application/.*?opendocument.*text
extracttext_use odt2txt .sdw .stw
application/(?:x−)?soffice
application/(?:x−)?starwriter
extracttext_external tesseract {OMP_THREAD_LIMIT=1}
/usr/bin/tesseract −c page_separator= {} −
extracttext_use tesseract .jpg .png .bmp .tif .tiff
image/(?:jpeg|png|x−ms−bmp|tiff)
# QR−code decoder
extracttext_external zbar /usr/bin/zbarimg −q −D
{}
extracttext_use zbar .jpg .png .pdf image/(?:jpeg|png)
application/pdf
add_header all ExtractText−Flags _EXTRACTTEXTFLAGS_
header PDF_NO_TEXT X−ExtractText−Flags =˜
/\bpdftotext_NoText\b/
describe PDF_NO_TEXT PDF without text
score PDF_NO_TEXT 0.001
header DOC_NO_TEXT X−ExtractText−Flags =˜
/\b(?:antiword|openxml|unrtf|odt2txt)_NoText\b/
describe DOC_NO_TEXT Document without text
score DOC_NO_TEXT 0.001
header EXTRACTTEXT exists:X−ExtractText−Flags
describe EXTRACTTEXT Email processed by extracttext plugin
score EXTRACTTEXT 0.001
endif
DESCRIPTION
This module uses external tools to extract text from message parts, and then sets the text as the rendered part. External tool must output plain text, not HTML or other non−textual result.
How to extract text is completely configurable, and based on MIME part type and file name.
CONFIGURATION
All
configuration lines in user_prefs files will be ignored.
extracttext_maxparts (default: 10)
Configure the maximum mime parts number to analyze, a value of 0 means all mime parts will be analyzed
extracttext_timeout (default: 5 10)
Configure the timeout in seconds of external tool checks, per attachment.
Second argument speficies maximum total time for all checks.
Tools
extracttext_use
Specifies what tool to use for what message parts.
The general syntax is
extracttext_use "name" "specifiers"
name
the internal name of a tool.
specifiers
File extension and regular expressions for file names and MIME types. The regular expressions are anchored to beginning and end.
Examples
extracttext_use
antiword .doc application/(?:vnd\.?)?ms−?word.*
extracttext_use openxml .docx .dotx .dotm
application/(?:vnd\.?)openxml.*?word.*
extracttext_use openxml .doc .dot
application/(?:vnd\.?)?ms−?word.*
extracttext_use unrtf .doc .rtf application/rtf text/rtf
extracttext_external
Defines an external tool. The tool must read a document on standard input or from a file and write text to standard output.
The special keyword "{}" will be substituted at runtime with the temporary filename to be scanned by the external tool.
Environment variables can be defined with "{KEY=VALUE}", these strings will be removed from commandline.
It is required that commandline used outputs result directly to STDOUT.
The general syntax is
extracttext_external "name" "command" "parameters"
name
The internal name of this tool.
command
The full path to the external command to run.
parameters
Parameters for the external command. The temporary file name containing the document will be automatically added as last parameter.
Examples
extracttext_external
antiword /usr/bin/antiword −t −w 0 −m
UTF−8.txt {} −
extracttext_external unrtf /usr/bin/unrtf
−−nopict {}
extracttext_external odt2txt /usr/bin/odt2txt
−−encoding=UTF−8 {}
Metadata
The plugin adds some pseudo headers to the message. These headers are seen by the bayes system, and can be used in normal SpamAssassin rules.
The headers are also available as template tags as noted below.
Example
The fictional
example headers below are based on a message containing
this:
1 A perfectly normal PDF.
2 An OpenXML document with a word document inside. Neither
Office
document contains text.
Headers
X−ExtractText−Chars
Tag: _EXTRACTTEXTCHARS_
Contains a count of characters that were extracted.
X−ExtractText−Chars: 10970
X−ExtractText−Words
Tag: _EXTRACTTEXTWORDS_
Contains a count of "words" that were extracted.
X−ExtractText−Chars: 1599
X−ExtractText−Tools
Tag: _EXTRACTTEXTTOOLS_
Contains chains of tools used for extraction.
X−ExtractText−Tools: pdftotext openxml_antiword
X−ExtractText−Types
Tag: _EXTRACTTEXTTYPES_
Contains chains of MIME types for parts found during extraction.
X−ExtractText−Types: application/pdf; application/vnd.openxmlformats−officedocument.wordprocessingml.document, application/ms−word
X−ExtractText−Extensions
Tag: _EXTRACTTEXTEXTENSIONS_
Contains chains of canonicalized file extensions for parts found during extraction.
X−ExtractText−Extensions: pdf docx
X−ExtractText−Flags
Tag: _EXTRACTTEXTFLAGS_
Contains notes from the plugin.
X−ExtractText−Flags: openxml_NoText
X−ExtractText−Uris
Tag: _EXTRACTTEXTURIS_
Contains uris extracted from the plugin.
X−ExtractText−Uris: https://spamassassin.apache.org
Rules
Example:
header
PDF_NO_TEXT X−ExtractText−Flags =˜
/\bpdftotext_Notext\b/
describe PDF_NO_TEXT PDF without text