Skip to main content

Phishing and Social Engineering in the Age of LLMs

Sean Gallagher ; Ben Gelman ; Salma Taoufiq ; Tamás Vörös ; Younghoo Lee ; Adarsh Kyadige ; Sean Bergeron (2024) — Large Language Models in Cybersecurity: Threats, Exposure and Mitigation, chapter 8, pp.81–86

Citation activity

Total citations · Google Scholar
Unavailable
Average per calendar year
Citation count unavailable

Counts reflect Google Scholar’s coverage and do not measure research quality. Source, calculation and limitations

View Original Source (opens in a new tab)

DOI: 10.1007/978-3-031-54827-7_8 (opens in a new tab)

Country:   Global
Tags:   Credential Theft, Cryptocurrency Scam, Large Language Models, Multimodal Generative AI, Phishing, Pig Butchering, Social Engineering, Spearphishing
What do these research terms mean?
Preprint
A manuscript shared before formal peer review and publication. Check whether a later published version is available.
Dataset
A collection of data or examples for others to inspect or reuse. It can appear in Library search and topic mapping, but RSRC does not use dataset records as evidence in Research Insights.
Dissertation or thesis
Research submitted for an academic degree. This describes its format, not its reliability.
Journal article
An article published in a journal. This label alone does not establish peer review, study quality, or how well the findings apply elsewhere.
Qualitative research
Examines experiences, meanings, or processes, often through interviews or observations. It can explain how something happens without estimating how common it is.
Quantitative research
Uses numerical measurements to describe patterns or test relationships. A relationship between two measurements does not by itself show that one causes the other.
Systematic review
Uses a planned, documented method to find and assess research addressing a question. Its conclusions still depend on the included studies and what the search covered.
Meta-analysis
Statistically combines results from multiple studies. Combining studies does not remove weaknesses in their design or make unlike populations interchangeable.
Not classified
This record has no recognized label in this filter. It does not mean the publication used no method, or that no research exists.

Definitions draw on DataCite resource types; Cochrane review methods; NLM: association and causation. RSRC’s dataset and classification rules are explained in our methodology.

Citation tools
Download RIS Download BibTeX 


              
              

            

          

        


      

      

    

        
Transparency

        
Evidence and review status

        
This page contains AI-generated content. No human content review or subject-matter-expert review is recorded.

    


    

                    

                
Source basis

                

                    Downloaded PDF
                

            

                    

                
Source updates

                

                    No notice found at last check
                

            

                    

                
AI-generated page content

                

                    Yes
                

            

                    

                
Automated checks

                

                    Passed
                

            

                    

                
Administrative approval

                

                    Yes
                

            

                    

                
Human content review

                

                    Not recorded
                

            

                    

                
Subject-matter-expert review

                

                    Not recorded
                

            

            


    

        
            How this was prepared
        
        

                            

                    
Source basis

                    

                    
RSRC downloaded and privately stored a copy of the paper for internal analysis. The PDF is not offered to viewers from this page.


                                            
    
                                                            
  • 
                                    PDF added to RSRC:
                                    
                                
    • 
                                                    

                    
                                        

                

                            

                    
Source updates

                    

                    
No incoming update notice was found in the dated Crossref response. Coverage is incomplete, particularly for corrections and expressions of concern; this is not a guarantee that the source is valid or unchanged.


                                            
    
                                                            
  • 
                                    Last source-status attempt:
                                    
                                
    • 
                                                    

                    
                                        

                

                            

                    
AI-generated page content

                    

                    
AI-generated research notes displayed on this page: Synopsis, Identified gaps, Methods, Limitations, Future work. The paper itself is not described as AI-generated.


                                            
    
                                                            
  • 
                                    Document analysis recorded:
                                    
                                
    • 
                                                            
  • 
                                    Synopsis generation recorded:
                                    
                                
    • 
                                                            
  • 
                                    Page record updated:
                                    
                                
    • 
                                                    

                    
                                        

                

                            

                    
Automated checks

                    

                    
The current, source-bound synopsis passed the recorded versioned publication checks.


                                            
    
                                                            
  • 
                                    Checks completed:
                                    
                                
    • 
                                                    

                    
                                            

                            
                                View passed checks (3)
                            
                            
    
                                                                    
  • Length, completeness, repetition, refusal, boilerplate, and active-markup screening
    • 
                                                                    
  • Numerical claims checked against the available source text
    • 
                                                                    
  • English-source lexical grounding check
    • 
                                                            

                        

                                        

                

                            

                    
Administrative approval

                    

                    
An authenticated administrator approved the bibliographic record for public Library display. This is not a review of every research claim.


                                            
    
                                                            
  • 
                                    Approved for public display:
                                    
                                
    • 
                                                    

                    
                                        

                

                            

                    
Human content review

                    

                    
No human review is recorded for the AI-generated content displayed on this page.


                    
                                        

                

                            

                    
Subject-matter-expert review

                    

                    
RSRC has not recorded review of this content by a subject-matter or methods expert.


                    
                                        

                

                    

    


    

        Review-state definitions

        Found a possible error?
        Request a correction.
    



    



              

        
                  

            
Synopsis

            
The chapter examines how large language models and multimodal generative technologies amplify phishing and social engineering threats, focusing on how AI can generate convincing, personalized content at scale and blend text, images, and audio to deceive targets. It frames the human factor as a persistent vulnerability in cybersecurity and notes that advances in LLMs make attacks harder to detect and more profitable by enabling tailored communications that mimic legitimate entities and urgent calls to action. Two case studies illustrate the practical reach of these tools. The first demonstrates an automated phishing campaign that uses an LLM to create and manage fraudulent websites, complete with product catalogs, testimonials, and a simulated checkout flow, powered by AutoGPT and other generative models to minimize human involvement. The second case study highlights a Shā Zhū Pán or “pig butchering” scam that leverages LLM-generated responses to sustain a romantic-interest narrative, culminating in financial theft. The authors emphasize that attackers can orchestrate multi-modal deception, including synthesized voices and realistic images, to reinforce trust and lure victims. The chapter concludes that AI-enabled phishing and social engineering are likely to grow in sophistication and scale, driven by the ease of generating tailored content and the ability to combine modalities. It underscores the need for heightened awareness and improved defenses, including automated first-line responses, to mitigate emerging threats while acknowledging the limitations of human-centered defenses when confronted with increasingly convincing AI-driven manipulation.

          

        
                  

            
Identified Gaps

            
The chapter demonstrates technical feasibility but does not evaluate real-world prevalence, victim susceptibility, detection accuracy, or the effectiveness of proposed awareness and automated defenses. Its romance-scam evidence is limited to illustrative Sophos case material rather than a systematic victim or offender study.

          

        
                  

            
Methods

            
The chapter uses two case studies. First, the Sophos AI team developed a proof of concept using an LLM and generative models to create a fraudulent e-commerce phishing website and associated content. Second, it describes a reported pig-butchering case in which an anomalously polished reengagement message was assessed as showing LLM-text hallmarks.

          

        
                  

            
Limitations

            
The phishing proof of concept still required manual human support, including integrations for image and audio generation and an existing e-commerce template. The chapter provides illustrative case-study evidence rather than systematic measurement of scam prevalence, victim outcomes, or comparative effectiveness of LLM-enabled attacks.

          

        
                  

            
Future Work

            
Develop automated, data-driven first-line defenses against LLM-enabled social engineering. Increase public awareness of multimodal phishing and romance-themed cryptocurrency scams.

          

              

    
    

        
Related Research Insights

        

            See how this publication connects to RSRC's living evidence syntheses through current citations and research-topic mapping.