Model Validation and Independent Estimate

Client: Department of Housing and Urban Development OIG

Estimating the Impact of Significant Model Changes, and Generating an Independent Point Estimate

Challenge: The U.S. Department of Housing and Urban Development (HUD) Office of Inspector General (OIG) retained FI Consulting to generate independent point estimates for some of Ginnie Mae’s accounting estimates, and assess various modeling changes made by Ginnie Mae during the Fiscal Year (FY) 2019 financial statement audit that decreased projected guaranty fees from $8.958 billion to $6.918 billion, a net decrease of $2.04 billion. ​

FI Solution: After reviewing model documentation, model code, and executing Ginnie Mae’s models in SAS, , FI pinpointed aspects of the Ginnie Mae’s models that could be improved. FI proposed several modifications, and re-specified Ginnie Mae’s models with the proposed modifications, generating an independent point estimate. Moreover, going into the FY 2019 audit, Ginnie Mae had made substantial changes to their models, but did not provide an attribution analysis outlining the impact of each change. FI decomposed Ginnie Mae’s major model changes and approximated the impact that each modeling change had on the $2.04 billion decrease in projected guaranty fees. ​

FI Impact: HUD OIG was able to understand the approximate magnitude and directionality of each major modeling change on the $2.04 billion guaranty fee decrease. ​HUD OIG was able to understand the magnitude and directionality of each FI re-specification on accounting estimates. FI discovered additional modeling concerns and helped HUD OIG understand how Ginnie Mae’s major model changes specifically impacted model outputs, and how proposed model improvements approximately impacted accounting estimates. These new insights aided HUD OIG in determining whether or not an accounting restatement was warranted.