Trust & Safety
Specialists in this role develop detection systems and enforcement strategies to identify and mitigate emerging abuse patterns across AI products, working at the intersection of data science, policy, and operations. They balance competing priorities—detecting sophisticated threat actors while maintaining platform usability—by building scalable detection pipelines, conducting rapid investigations, and collaborating with policy and engineering teams to implement mitigations. Unlike policy-focused roles, these positions emphasize technical implementation and quantitative analysis; unlike pure engineering roles, they require deep domain expertise in specific abuse vectors and threat actor behavior. These analysts typically sit within dedicated Trust & Safety or Safeguards teams that operate cross-functionally with research, product, and legal to stay ahead of evolving misuse techniques.
Skills
What companies are looking for in this role.
Monitoring and investigating content and behavior that violates terms of service
Detecting, investigating, and disrupting malicious use of AI platforms
Developing abuse signals and tracking strategies to proactively detect harmful activities
Providing data labeling, annotations, and inputs for safety protocols
Designing and implementing enforcement workflows and review processes
Analyzing large datasets to identify patterns and coordinated networks
Building and scaling detection systems for fraud and abuse
Processing appeals and auditing automated systems
Responding to urgent escalations and participating in on-call rotations
Conducting threat intelligence analysis and threat actor investigations
Conducting root cause analyses and deep-dive investigations
Creating monitoring dashboards, alerts, and internal administrative interfaces
Leading safety assessments and threat modeling for new products
Managing vendor relationships and third-party content moderation services
Training and refining large language models for safety and policy enforcement
Building multi-layered defenses and real-time safety mechanisms for AI systems
Conducting safety evaluations and assessments of AI models
Developing AI-specific detection capabilities and behavioral clustering techniques
Creating automated enforcement systems that scale with AI platform growth
Collaborating across cross-functional teams including engineering, policy, and legal
Communicating complex technical concepts to non-technical stakeholders
Leading and mentoring teams of safety operations analysts
Technology
The tools and technologies that define this role.
Open Jobs
37 open Trust & Safety jobs across 5 companies.
Other Security roles
Identifies and mitigates security vulnerabilities in applications and products.
Secures cloud infrastructure, networks, and systems.
Generalist security engineering role spanning multiple security domains. For security engineers who work across application, infrastructure, and cloud security without a single dominant specialization. The default home for "Security Engineer" titles when the function is clearly Security.
Builds detection systems, investigates security incidents, and leads incident response efforts.
Conducts offensive security assessments including red teaming, penetration testing, and adversarial simulation.