Skip to main content

Ethical and Social Challenges with developing Automated Methods to Detect and Warn potential victims of Mass-marketing Fraud (MMF)

Whitty, Monica ; Edwards, Matthew ; Levi, Michael ; Peersman, Claudia ; Rashid, Awais ; Sasse, Angela ; Sorell, Tom ; Stringhini, Gianluca (2017) — Proceedings of the 26th International Conference on World Wide Web Companion - WWW '17 Companion

View Original Source (opens in a new tab)

DOI: 10.1145/3041021.3053891 (opens in a new tab)

Type:   Journal Article
Country:   Global
Tags:   Automated Detection, Ethical AI, Machine Learning, Mass Marketing Fraud, Online Dating, Privacy, Romance Scams, Warning Systems
Citation tools
Download RIS Download BibTeX 


              
              

            

          

        


      

      

    

        
Transparency

        
Evidence and review status

        
This page contains AI-generated content. No human content review or subject-matter-expert review is recorded.

    


    

                    

                
Source basis

                

                    Downloaded PDF
                

            

                    

                
AI-generated page content

                

                    Yes
                

            

                    

                
Automated checks

                

                    Passed
                

            

                    

                
Administrative approval

                

                    Yes
                

            

                    

                
Human content review

                

                    Not recorded
                

            

                    

                
Subject-matter-expert review

                

                    Not recorded
                

            

            


    

        
            How this was prepared
        
        

                            

                    
Source basis

                    

                    
RSRC downloaded and privately stored a copy of the paper for internal analysis. The PDF is not offered to viewers from this page.


                                            
    
                                                            
  • 
                                    PDF added to RSRC:
                                    
                                
    • 
                                                    

                    
                                        

                

                            

                    
AI-generated page content

                    

                    
AI-generated research notes displayed on this page: Synopsis, Identified gaps, Methods, Limitations, Future work. The paper itself is not described as AI-generated.


                                            
    
                                                            
  • 
                                    Document analysis recorded:
                                    
                                
    • 
                                                            
  • 
                                    Synopsis generation recorded:
                                    
                                
    • 
                                                            
  • 
                                    Page record updated:
                                    
                                
    • 
                                                    

                    
                                        

                

                            

                    
Automated checks

                    

                    
The current, source-bound synopsis passed the recorded versioned publication checks.


                                            
    
                                                            
  • 
                                    Checks completed:
                                    
                                
    • 
                                                    

                    
                                            

                            
                                View passed checks (3)
                            
                            
    
                                                                    
  • Length, completeness, repetition, refusal, boilerplate, and active-markup screening
    • 
                                                                    
  • Numerical claims checked against the available source text
    • 
                                                                    
  • English-source lexical grounding check
    • 
                                                            

                        

                                        

                

                            

                    
Administrative approval

                    

                    
An authenticated administrator approved the bibliographic record for public Library display. This is not a review of every research claim.


                                            
    
                                                            
  • 
                                    Approved for public display:
                                    
                                
    • 
                                                    

                    
                                        

                

                            

                    
Human content review

                    

                    
No human review is recorded for the AI-generated content displayed on this page.


                    
                                        

                

                            

                    
Subject-matter-expert review

                    

                    
RSRC has not recorded review of this content by a subject-matter or methods expert.


                    
                                        

                

                    

    


    

        Review-state definitions

        Found a possible error?
        Request a correction.
    



    



              

        
                  

            
Synopsis

            
The publication presents an interdisciplinary effort to develop automated methods for detecting mass-marketing fraud (MMF) and warning potential victims, with a focus on mass-market scams including romance scams. Its central aim is to prevent victimization by enabling early identification of deceptive communication and grooming strategies across multiple online channels. The authors describe an ongoing project that blends psychology, media studies, criminology, linguistics, and human-computer interaction to identify signals of deception, grooming, and persuasive requests within end-user communications and profiles. The paper outlines a research program that would use personal data from dating sites, employment pages, emails, and other sources to train and test machine-learning and pattern-recognition approaches. Proposed methods include supervised learning (e.g., random forests) and clustering to distinguish scam-related behavior and to uncover linguistic and stylistic indicators typical of victims and scammers. They also consider analyzing sociotechnical features such as response patterns, profile descriptions, and multi-channel interactions. Ethical and social challenges are given substantial attention: data anonymization, informed consent, potential false positives, and the need to balance protection with user autonomy. The authors acknowledge the difficulty of making automated judgments about authenticity and the risk of harming the very individuals they aim to help, suggesting human-in-the-loop or warning-message strategies as possible mitigations. The discussed implications emphasize that while automated MMF detection could reduce financial and psychological harm, it raises privacy concerns and governance questions about how such systems are deployed, how risks of misclassification are managed, and how user trust is maintained. The publication stops short of presenting empirical results, instead outlining the conceptual framework, methodological directions, and ethical constraints guiding future work.

          

        
                  

            
Identified Gaps

            
Existing scam education focused on idealized individual behavior and knowledge may not prevent victimization; awareness can coexist with vulnerability. MMF detection is difficult because offenders use personalized, adaptive, long-term communication across multiple channels. The paper also identifies an unresolved need to design detection and warning systems that balance prevention with privacy, autonomy, false-positive harms, and trust.

          

        
                  

            
Methods

            
This extended abstract describes early planning for an interdisciplinary MMF detection project. The proposed research analyzes dating and employment profiles and communications from victims, scammers, and noncriminal contacts. It plans to examine psychological, linguistic, behavioral, and socio-technical indicators, using anonymization, supervised machine learning (including random forests), and profile clustering. If detection proves effective, the team plans a browser-extension proof of concept and HCI-informed tests of warning messages.

          

        
                  

            
Limitations

            
The paper reports planned work rather than completed empirical findings, validation results, or performance measures. Detection may produce false positives that harm genuine users and relationships. The proposed system would require analysis of highly personal data and may make authenticity judgments without the other person's knowledge. Victims may also distrust or disregard warnings, limiting intervention effectiveness.

          

        
                  

            
Future Work

            
Develop and evaluate a proof-of-concept browser extension that detects suspected scam accounts and warns users. Test the effectiveness of alternative warning-message designs. Explore warning approaches that retain human judgment, such as guided authenticity checks, and consider whether tools should be situated on dating sites or support networks.

          

              

    
    

        
Related Research Insights

        

            See how this publication connects to RSRC's living evidence syntheses through current citations and research-topic mapping.