Validation of Public and Consumer Finance Models

FI was tasked with validating a large financial institutions stress models.

Client: Large Financial Institution with $300+ Billion Dollars in Assets

Challenge: A large bank’s Model Risk Management team was in the cycle of reviewing and validating their internal models to comply with the guidelines set by the Comprehensive Capital Analysis and Review (CCAR). FI offered a combination of CCAR business knowledge and model validation skills to validate the C&I Public and Consumer Finance probability of default models. FI Consulting worked closely with the Model Risk Management team members to gather the proper data sets, documentation, and code that was used to develop the original models. The focus of the project was centered upon replicating the model results, assessing the performance of the models and stress testing the models under the Baseline and Severely Adverse economic scenarios.

FI provided expert advisory support to the Model Risk Management Group in the areas of model theory, data quality, testing, implementation, and regulatory compliance. Model Validation activities were consistent with regulatory guidance, internal validation policies, and industry best practice. Throughout the validation process, FI Consulting met with model developers, conducted effective challenge, and performed full replication and testing of all datasets and model components. FI Consulting’s efficient discovery of model findings and significant issues allowed the Credit Policy Committee to remediate issues and implement changes in production in time for CCAR reporting season.

FI Solution: FI conducted multiple statistical tests, in addition to assessing model assumptions and methodologies, replicating model results and data cleansing procedures, stress testing for Base and Severely Adverse economic scenarios, and assessing model performances. FI generated multiple visualizations to display the results of the analysis and tests conducted for each of the models.

FI Impact: FI found the models to be sound in methodologies and performance; however, there were suggestions that should be considered by the model developers and the Model Risk Management team. The suggestions were tested and indicated that they could positively affect the performance of the models, and improve the methodologies that were used to develop the original models. FI was able to identify several documentation gaps, observations, and alternate methodologies that would be more in line with industry best practices to assess variable selection, feature engineering, and model construction methods. The extensive and thorough tests conducted by FI provided the Model Risk Management team and the model developers several items to consider for improving the models.