Assess Risk in Third-Party AI Models

Carriers increasingly build underwriting and claims tools on third-party models, open-source components, and vendor services. Vendors may withhold documentation, treat outputs as a black box, and change model behavior without notice. PointGuard AI evaluates these components before and after adoption, giving insurers evidence about what they depend on and when that dependency changes.

Insurers remain accountable for outcomes produced by models they did not build and cannot fully inspect.

• Vendors provide little documentation about training data or model changes.
• Open-source models and MCP servers can carry vulnerabilities or malicious code.
• License terms can conflict with commercial or regulatory use.
• Model updates can change behavior without notifying the carrier.

Supply Chain Security and AI Red Teaming evaluate third-party components and track how their risk changes over time.

1. Inventory AI dependencies. Identify models, datasets, libraries, and MCP servers used across underwriting, claims, and service applications. Generate an AI bill of materials for each system.

2. Score component risk. Evaluate publisher reputation, known vulnerabilities, unsafe file formats, and license terms. Flag components that fail the carrier's acceptance criteria before deployment.

3. Test vendor behavior. Run adversarial and harmful-output probes against third-party models in the carrier's own context. Document results for vendor reviews and model approval decisions.

4. Watch for changes. Re-score components and retest models when versions change. Alert owners when a dependency's risk rises so the carrier can respond before outcomes are affected.

• OWASP Top 10 for LLMs: LLM03 Supply Chain and LLM04 Data and Model Poisoning, covering compromised or unvetted components from outside vendors.
• OWASP Top 10 for Agentic Applications: ASI04 Agentic Supply Chain Vulnerabilities, addressing vulnerabilities introduced through third-party agentic components.
• NIST AI RMF: GOVERN sets third-party acceptance criteria; MAP identifies dependencies; MEASURE evaluates component risk; MANAGE responds to changes.