[feedback-report] reality check, was Revised draft charter

[Douglas Otis]

On 9/28/09 4:04 PM, J.D. Falk wrote:
> Douglas Otis wrote:
>
>> It should be stated as a good practice for providers receiving ARF
>> reports, to visually review on a sampling basis, the messages contained
>> to confirm an ARF generator's proper message categorizations.
>
> Except for a few test environments, I'm not aware of any ARF generator using
> a Feedback-Type other than "abuse".  This is why I proposed removing most of
> the other types.
>
> I'll make that proposal again once we've got a working group.

J.D.

Limiting category selections would fine.  However, the concern regarding 
the proper categorization of a message remains a concern, even when 
limited to messages being categorized as spam/abusive or not.  Here 
auto-responses in foreign languages being marked as spam are one example 
of a common mistake.

Spammers often use harvested email addresses in their spam's return 
address.  When this address represents a honey-pot, then determining 
which messages are auto-responses becomes important.  Often mistakes can 
be visually obvious, even when the language of the message is not 
understood.

Language recognition and translation tools are making this task easier, 
however support for some languages still remains limited.  In languages 
where support is good, recognition errors still represent several 
percentage points.  Message categorization has not reached a point of 
certainty that allows an assumption of categorization errors being so 
few as to be negligent, IMHO.

Strides are being made at improving this situation in our own product, 
with emphasis on several Asian and Middle Eastern languages. 
Nevertheless, spammers leverage any gray area in an automated process, 
where often no two messages are identical to thwart statistical 
analysis.  With vast computational resources at their disposal, spammers 
enjoy an advantage when it comes to automation.

-Doug