shutterstock_1144251818.jpg

Model Validation

ComplyGenics possesses a dedicated team of professionals specializing in the application of analytics to risk and compliance model development and validation testing. Our team has developed a set of proprietary methodologies and tools for model development and validation testing that we will apply as core enablers to execute efficient and effective services.

Specifically, we understand that you are interested in FCC model validation testing related to:

  • Conceptual soundness of the design and construction of the model

  • The accuracy of the data sourcing, mapping, and quality of the data

  • Performance testing of FCP Transaction Monitoring system

  • Effectiveness assessment of a financial institution’s monitoring strategy

  • Reporting to our client and continuous knowledge transfer to the Model Risk Management team

ComplyGenics’s Methodology for Model Validation Testing 

Assessment of Conceptual Soundness:

  • Assess whether the design is consistent with the transaction monitoring team expectations and FCC Compliance’s policies and risk tolerance

  • Assess completeness of transaction monitoring scenario selection and coverage of risks identified in FCC Risk Assessment

  • Assess the accuracy of scenario logic by replicating model logic for a sample of scenarios, and generating testing alerts

  • Conduct empirical analyses to compare to commonly used vendor models

  • Assess soundness of design of model governance procedures​

shutterstock_1221068065.jpg
Attribute
Grouping
shutterstock_1292827159.jpg
Criticality
Assessment
shutterstock_740863696.jpg
Data
Quality
Rules
shutterstock_1559487254.jpg
Document

Data Quality Assessment:

  • Assess the quality of data input to the models by conducting targeted testing

  • Assess controls for proper identification and notification of data quality issues

  • Analyze end to end data flow and data mapping from source to the target system, and perform analysis of the ETL (Extract, Transform & Load) processes

  • Assess the quality of data input to the model by testing the consistency of data from source and surveillance databases

FCC TMS Validation

Assess Pre-Production Tuning Methodology


Assess client’s tuning methodology and pre-implementation tuning plan for consistency with our understanding of current industry practices




Assess Threshold Settings


  • Assess the sufficiency of support for threshold settings by assessing pre-production tuning analysis and output
  • Conduct analysis of thresholds settings with available data
  • Conduct distributional analysis to assess whether the percentage of the test population that is currently alerting by using initial threshold setting is consistent with our client's risk assessment and our understanding of current industry practices




Establish Performance Measures


Assess the design of financial institution's ongoing monitoring and reporting framework and establish relevant performance measures




Effective Challenge


  • Assess whether an alternate approach to monitoring suspicious activities may be employed
  • Assess any additional activity outside of current scope of FCC monitoring that may potentially be added to enhance the current approach




Report of Results


  • Compile a report containing scope of testing methodology, assumptions, testing results, findings, and recommendations
  • Describe model testing activities and results of model validation testing: Data quality, Model Design, Model Implementation, Model Performance and Model Governance





Types of Model Validation

shutterstock_635474195.jpg
Transaction Monitoring (TM)
shutterstock_478870435.jpg
Sanctions
shutterstock_755088883.jpg
Customer Risk
Rating

Data validation procedures:

ComplyGenics follow various steps that will be executed to assess the overall data quality

Contact Us

1100 Cornwall Rd. Ste. 205, Monmouth Junction NJ 08852

609-924-3400

  • LinkedIn
  • Facebook
  • Twitter

© 2020 by ComplyGenics