To promote the stability of the financial system, the Federal Reserve Board (FRB) is responsible for regulating and supervising various financial entities. One of the tools the FRB uses to fulfill its mandate is stress testing of financial institutions.
The FRB has adopted a formal standard that, the severely adverse scenario of the Comprehensive Capital Analysis and Review (CCAR) program, the Global Market Shock (GMS) component should consider “hypothetical but plausible outcomes.”1 Unlike the macroeconomic component of CCAR, where the FRB has adopted formal unemployment rate and house price decline quantitative targets for the severely adverse scenario, the FRB has not adopted any quantitative thresholds for determining what severely adverse scenario shocks are “plausible” in the GMS component. The Study attempts to answer a simple question: does the available evidence indicate that GMS and Large Counterparty Default (LCD) shocks from recent CCAR cycles are plausible?
To answer this question, the Study evaluated the probability of various GMS/LCD shocks from the past several years. In many cases, the statistical probability of a GMS/LCD shock is extremely low. For example, as summarized in the pages that follow, the statistical probability of 2019 GMS spreads for certain corporate bonds occurring is 0.001%. The Study begs the question: should a one-in-one hundred thousand probability be deemed to be “plausible” within the meaning of the FRB’s scenario design standard?
The Study does not propose a formal quantitative threshold for determining what GMS/LCD shocks should be deemed plausible. However, for reference, it is worth keeping in mind that the FRB modeled the Macroeconomic component’s unemployment rate and house price decline quantitative targets on observed economic data from recent severe recessions. the 10 percent unemployment rate target, for example, is “the average level to which it has increased in the most recent three severe recessions.”2 The FRB’s approach to calibrating severity in the macroeconomic component suggests that “plausible” shocks under the GMS should align with observed stress market conditions from recent severe recessions.
The Study concludes that, in a number of significant areas, GMS/LCD shocks are not reasonably plausible. In many areas, GMS/LCD shocks do not replicate, or approximate levels of severity observed in recent severe recessions, and in some key areas effectively impose double counting that exaggerates loss assumptions well beyond historic experience. In other cases, the assumptions that govern GMS/LCD shocks appear to be illogical, resulting in exaggerated stress loss estimates that do not align with plausible assessments of market or counterparty risk. Reasonably plausible GMS shocks are critical to the GMS and LCD components of the stress test, as they are the single tool that the FRB uses to size the test’s severity. Unnecessarily severe calibration of shocks results in excessive capital requirements relative to potential risk, which create uneconomic outcomes for firms, their clients and the market. These exaggerated stress loss estimates include instances where particular trading assets are subject to multiple, unreconciled stress loss assumptions, which can result in capital requirements that exceed trading assets’ carrying value or other double counting that are analytically unsupported. The effect of implausible or illogical GMS/LCD shock calibrations and assumptions should be assessed not only in isolation, but also cumulatively; the coherence and risk management value of the GMS component are weakened if the GMS is not grounded in a reasonable assessment of banking institutions’ actual vulnerabilities.
The Study provides detailed calculations in support of its conclusions, with the objective that open, transparent engagement will result in future GMS/LCD shocks that more clearly meet the FRB’s stated objective of “hypothetical but plausible outcomes.” In time, with further data analysis, we believe that the formal standards governing the GMS component could evolve to include quantitative targets analogous to the unemployment and house price decline ratios, thereby improving the coherence of CCAR stress loss analysis and strengthening its utility in risk management.
1 12 C.F.R. § 252 Appendix A Section 5.2.3(c).
2 12 C.F.R. § 252 Appendix A Section 4.2.2.
To support the FRB’s recent effort to review and improve the transparency, coherence and volatility of the CCAR program,3 SIFMA4 performed an analysis of the GMS and LCD components of CCAR. These efforts are described in this paper as: “Global Market Shock and Large Counterparty Default Study,” or the “Study”.
The primary goals of the Study were to understand and analyze the:
- Assumptions underlying the GMS, including its severity, plausibility, stability and degree of conservatism;
- Calibration and correlation assumptions embedded in the GMS;
- Coherence of incorporating the GMS in the LCD; and
- Transparency and coherence of the overarching CCAR framework.
The Study’s empirical and statistical analysis reviewed the range of GMS shocks individually and collectively from the inception of CCAR through CCAR 2019. It also attempted where possible to understand and consider the impact of the new GMS approaches announced in February 2019 pursuant to which future scenarios would derive from historical, hypothetical
and hybrid sources. The analytical approaches that the Study used to perform an assessment and draw conclusions included:
- Estimating the probability and severity of a subset of material GMS shocks using historical data, and assuming a conservative fat-tailed distribution;
- Evaluating the plausibility of the GMS shocks occurring simultaneously based on historical data;
- Analyzing the severity of GMS shocks individually against historical factor activity;
- Assessing trading volume data as a proxy for asset class liquidity during the crisis period;
- Examining regulatory approaches to estimating market dislocation duration; and
- Evaluating the use of GMS factor shocks on the LCD component’s outcome.
- The severity of single-factor GMS shocks — including their extended calibration — are not considered reasonably plausible based on empirical analysis and unlike the macroeconomic scenario, do not appear closely tied to a detailed scenario;
- The correlation assumptions underlying the construct of annual GMS shocks cannot be empirically justified as reasonably plausible;
- The use of GMS factor shocks to size losses in the LCD component is not appropriate because of the factor shocks’ severity, calibration and correlation assumptions;
- The use of the GMS and LCD components and the pre-provision net revenue (PPNR) framework to estimate losses leads to an inherent overestimation of losses and underestimation of available capital; and
- The limited transparency and disclosure regarding the assumptions which underly the GMS and LCD components impede public study and improvements in methodology.
Each of the material conclusions including the approach is briefly summarized below.
The severity of single-factor GMS shocks — including their extended calibration — are not considered reasonably plausible based on empirical analysis.
To assess the calibration and the plausibility of individual GMS factor shocks, the Study used several analytical and statistical tests, including by evaluating the:
- Severity of a sub-population of material shocks based on historical factor activity;
- Relationships between shock factor moves;
- Probability of factor shocks occurring based on historical and crisis data;
- Trading volumes for a sample of asset classes across the crisis period as a proxy for market receptivity; and
- Access and capacity of specific markets to execute transactions following a material market event.
The Study found that the probability of a 2019 GMS factor shock occurring on a single day were extremely remote. For instance, the probability of market movements assumed by the 2019 GMS spreads for A/BBB/B- corporate bonds, single A rated Credit Card ABS, GBP/USD and EUR/USD were less than or equal to 0.001%. Similarly, the probability of the 2019 GMS S&P 500 Index shock occurring on a single day estimated using a historical time series was 0.002%. Estimating the same probability using only crisis data (126 trading days of the second half of 2008) and assuming a crisis occurs every 10 years, the probability that the S&P index would move 20.3% in one day was similarly low, implying a frequency of once in every four crisis periods. As further discussed below, the severity of the GMS shocks is compounded when applied simultaneously. Although stress test shocks should be severe, the extent of any such severity should not be so extreme as to be implausible. Introducing discipline into these assumptions would increase the comparability of CCAR results year-over-year and support more robust risk analysis and conservative capital management.
The study also employed a historical analysis to assess the plausibility of the severity of GMS shocks. Specifically, the study compared the most adverse GMS factor shocks used since 2013 with the most adverse performance for 10 asset classes. The analysis indicated that GMS factor shocks were more severe than the actual most adverse six-month period for five of the ten asset classes sampled. When reviewing the most adverse three-month period, the Study found that the GMS factor shocks were more severe than the most adverse three-month period for six of the ten asset classes reviewed. Finally, when reviewing the most adverse ten-day periods, the GMS factor shocks were more severe than the most adverse ten-day period for seven out of the ten asset classes sampled. These results underscore the severe calibration of the GMS shocks across the 10-day, three month and six-month time periods.
Lastly, the Study examined the market receptivity of certain asset classes immediately following a large- scale adverse event (such as the Lehman default, AIG bailout and Brexit). Using volume as a proxy for market receptivity, the Study found that U.S. Treasuries, corporate bonds, and FX pairs exhibited reasonable, and arguably even healthy, volumes immediately preceding and following the studied event. To test this hypothesis further, the Study used a Transaction Cost Model to test market capacity and depth. Ultimately, the analysis revealed that the markets in these asset classes were deep enough to support execution of large transactions, suggesting that certain asset classes should not be subject to the most severe calibrations.
Taking into account the results of the single factor analysis, the plausibility of historical GMS shock versus observed experience and our market function analysis, the Study concluded that the severity and calibration of many GMS shock factors were not plausible. Based on this finding, the FRB should more actively consider plausibility and consistency when calibrating the severity of the GMS shocks. An empirically supported range of severity and calibration metric would increase the credibility of CCAR’s GMS and LCD components, and moreover, result in outcomes that support capital and risk management.
The correlation assumptions underlying the construct of annual GMS shocks cannot be empirically justified as reasonably plausible.
As currently constructed, the GMS assumes that multiple, if not most, asset classes will experience their most adverse or near-most adverse performance simultaneously. To assess the plausibility of this assumption, the Study conducted various correlation assessments including:
- Cross-factor relationship historical analysis of largest one-day and six-month movements across a subset of asset classes;
- Three-month rolling historical correlation analysis between select factor pairs; and
- Statistical analysis of the relative probabilities of joint tail events for correlated variables.
The Study demonstrated that simultaneously applying the GMS shocks would require a set of market movements that have never been experienced together in history. Moreover, the FRB’s underlying assumption that extreme events for various asset classes are highly correlated was not supported by historical statistical evidence and more importantly was not empirically supported.
The Study found no material evidence of simultaneous shifts in asset class performance in the magnitude represented by the 2019 GMS scenario in a cross factor historical correlation analysis of one-day and six-month periods. While the Study identified positive correlations between certain asset classes, such correlations were never of the same magnitude as implied by the GMS shocks, and never occurred on the same date. On a six-month basis, the Study revealed some increased positive correlations among certain asset classes versus the one-day cross factor historical analysis, but, again, such correlations were not nearly of the same magnitude implied by the relevant asset class GMS shock tested.
Another test for correlation produced similar results. In an analysis of rolling historical factor pairs, the Study revealed that the existence of some positive correlation depended on the asset classes in question. For example, over a 20-year historical time series, the analysis revealed very low correlation between the S&P 500 and the 10-year U.S. Treasury notes (roughly 0.29 on average). Moreover, the degree or direction of correlation was not constant over time. While the GBP/USD and the EUR/USD demonstrated higher correlation (0.63 on average), the correlation relationship similarly varied significantly over time. For both asset class comparisons, the variation in correlation suggests the magnitude of any positive and negative correlation is bounded at levels much lower than suggested in the GMS. Consequently, the high correlations assumptions found in the GMS factor shocks are significantly overstated.
Lastly, the Study estimated the relative likelihood of different extreme events occurring simultaneously based on the correlations between those extreme events (using perfect correlation as a benchmark). The Study found that correlation across factors would need to exceed 0.85 before the relatively likelihood of joint extreme events would reach at least 50%, suggesting that the correlation assumptions underlying the GMS factors are implausible. This analysis again supports the conclusion that the correlation assumptions underlying the application of the most severe GMS factors across most asset classes are overstated and empirically remote.
Given the implausibility of the correlation assumptions embedded in the GMS factor shocks and the impact that these assumptions have on the severity of the overall outcome, the FRB should establish explicit, reasonable bounds on these correlation assumptions. Using more empirically grounded correlation assumptions in sizing GMS shocks would result in more precise and credible outcomes.
The use of GMS factor shocks to size losses in the LCD component is not suitable, because of the implausible calibration and correlation assumptions embedded in the GMS.
The use of the GMS to size losses in the LCD component implies a margin period of risk (MPOR) that is considerably longer than both recognized market practice and applicable regulatory requirements, even in extreme market conditions. To assess the reasonableness of the MPOR implied by GMS factor shocks, the Study reviewed market practice and regulatory requirements, and estimated the cost and speed of the liquidation of a large hypothetical portfolio of “typical” collateral. Although the Study was unable to assess the speed to rebalance positions following a large counterparty default, market feedback indicated that the largest counterparty exposures typically are rates and FX derivative contracts, and firms would be able to access exchanges to initiate risk and loss mitigating hedges as typical during events in 2008 and 2009.
The analysis indicated that the MPOR implied by the GMS factor shock calibration can be as extreme as 120 business days. While an extended MPOR may be appropriate for certain transaction or collateral types as outlined in point in time capital rules (ranging from 10 to 20 days in general), the MPOR implied by the GMS factor shocks can be six to twelve times greater than applicable MPOR point-in-time regulatory requirements. Moreover, the extended MPOR calibration implied by the GMS calibration is considerably outside of market observation Additionally, the transaction cost analysis estimated that, under a 99% confidence interval, a firm could liquidate a large pool of typical collateral within five days and within the applied haircuts, even if the pool of collateral included relatively illiquid securities within a given asset class.
Based on these studies, the FRB should revise the LCD component to increase its risk sensitivity and more closely mirror established market practices around liquidation of collateral. At a minimum, the FRB should support an estimation of LCD losses based on extreme-but-plausible price moves over conservatively defined closeout periods, but not as draconian as those implied in many of the GMS shocks. Calibrating the LCD based on a new set of price shocks defined independently of the GMS, or by applying scalars that adjust the GMS down to closeout periods appropriate for counterparty default exposures are potential alternatives that would be appropriately risk sensitive and reflect market practice.
The Study’s findings regarding calibration of the GMS shocks and its use in LCD lend additional support to an FRB review of other aspects of the GMS and LCD where concerns have been noted. These other aspects include the impact of using both the GMS and LCD components and the PPNR process, and the lack of transparency of GMS and LCD compared to other elements of CCAR testing.
The use of both the GMS and LCD components and the PPNR framework leads to overestimation of losses and underestimation of available capital.
The requirement to apply both the GMS and PPNR components to many of the same assets results in overestimation of losses and underestimation of available capital through double counting of losses for those assets. Many of the same trading assets that are written down under the GMS are then subject to additional losses as part of the PPNR forecasting requirement, which the Study indicates can have severe impacts. The impacts are further exacerbated by the treatment of deductions for certain asset classes.
Accordingly, the FRB should revise the GMS and/or PPNR approaches to eliminate any such double counting either by developing a supervisory workaround, or by “zeroing out” the GMS and LCD component results in the first quarter of the PPNR. Further, the FRB should implement a max loss cap (losses and deductions) to avoid capital requirements that could exceed the current value of securities and investments. With regards to the CCAR treatment of non-SSFA assets that require a capital deduction, firms are required to employ pre-stress assets values when calculating post-stress loss capital. The Study concluded it is more appropriate to calculate deductions in post-stress loss capital ratios using poststress loss asset values.
The limited transparency and disclosure regarding the GMS and LCD components, including assumptions, and the lack of a coherent scenario type approach impede public study and improvements in methodology.
A continued disparity in transparency and disclosure exists between the GMS and LCD components and other elements of CCAR such as scenario design, loss-modeling and PPNR. This opacity impedes public study of approaches and
outcomes and consequently hinders process and approach improvements. Greater openness on the part of the FRB to share the justification for its approaches also could help preserve and promote its independence.
Currently, GMS factor shocks are simply provided by asset class without regard to the events the GMS factor shocks have been sized to capture. Providing transparency around the broader scenario like how the macroeconomic scenario provides context to the banking book and PPNR components would promote better understanding of potential risk and capital needs. Linking the GMS factor shocks to a detailed “scenario” or event type would support the year-on-year analysis and potentially limited back testing of results.
For these reasons, the FRB should continue to promote transparency and, moreover, apply the same level of transparency and disclosure employed in other aspects of CCAR to the GMS and LCD components. Moreover, the heightened transparency recently applied to the macroeconomic scenario design process and the development of controls and metrics to guide annual volatility should be applied to the GMS shock development and evaluation.
Policy Commentary and Recommendations
SIFMA supports stress testing as a valuable capital planning and risk management tool. It also appreciates the goals of CCAR, including the incorporation of fire sale risk as a critical consideration of the GMS and LCD. However, the Study’s empirical and historical analysis, coupled with the related conclusions regarding the plausibility of assumptions underlying the GMS and LCD, warrant further FRB consideration. Important improvements could be made to the GMS and LCD to support their continued creditability, appropriateness and precision. Moreover, it is particularly appropriate to improve the precision of the GMS and LCD components given the anticipated implementation of the Stress Capital Buffer (SCB). This sequencing is important because reducing unnecessary and unwarranted severity in the GMS and LCD will promote more appropriate inputs to the CCAR framework and, in turn, the size of the SCB.
It is critical that the GMS and LCD are appropriately calibrated, because these components have a significant measurable impact on banks’ capital allocation and business strategy, which in turn affect the health, capacity and effectiveness of the U.S. capital markets. The effect of any revisions to the GMS and LCD should be to increase the value of the GMS and LCD components as capital planning and risk management tools, to apply the lessons learned from previous GMS and LCD reviews, and to improve the precision and credibility of the framework’s outcomes.
3 Although supervisory stress testing exists independently from CCAR, and the paper’s comments focus on the FRB’s supervisory stress testing more generally, this paper refers to CCAR and the FRB’s supervisory stress testing framework interchangeably for convenience.
4 The thoughts expressed in the document are that of SIFMA, however, BlackRock Financial Market Advisory assisted with the quantitative analysis and Debevoise & Plimpton LLP assisted in the preparation of this document.
|Recommendation #1||GMS factor shocks should be tailored to reflect the liquidity and price stability characteristics of particular asset classes.|
|Recommendation #2||Correlation assumptions need to be rationalized and validated based on empirical analysis and historical observation in order to meet the “severe but plausible” standard.|
|Recommendation #3||Year-over-year shock volatility should not be extreme or random, but rather should reflect more gradual adjustments to emerging views of risk that are linked to feedback from the prior years' analysis.|
|Recommendation #4||Replace the blunt three- to six-month GMS with a framework that accounts for the liquidity and price stability of assets. Assets with high price stability and liquidity should be calibrated towards historical observation, at a maximum 30 days or fewer.|
|Recommendation #5||Review and justify the LCD component to improve risk sensitivity and more closely mirror market practice.|
|Recommendation #6||At a minimum, estimate LCD losses on the basis of extreme-but-plausible price moves over conservatively defined closeout periods, either by basing the LCD on a new set of price shocks defined independently of the GMS, or by applying scalars that adjust the GMS shocks down to closeout periods appropriate for counterparty default exposures.|
|Recommendation #7||Increase the transparency and disclosure regarding recovery rate modeling, assumptions and data including empirical support. Recovery rates used in CCAR should be disclosed annually.|
|Recommendation #8||Apply similar transparency to the determination of GMS shocks as is currently applied to the FRB’s macroeconomic scenarios. Additionally, employ controls or “guardrails” on GMS factor severity, correlation, volatility and calibration similar to what the FRB implemented with respect to House Price Index (HPI) and unemployment. Moreover, apply the same process type controls to the administration and communication of the GMS AND LCD components’ annual process.|
|Recommendation #9||Revise the GMS and LCD components and the PPNR framework to eliminate the double counting issue either by a developing a transparent supervisory workaround, or by zeroing out the GMS and LCD components results in the first quarter of the PPNR. Simultaneously, address the underestimation of available capital by implementing a max loss cap (losses and deductions) to avoid capital requirements that could exceed the current value of securities and investments, and allow firms to calculate deductions in post-stress loss capital ratios using post-stress loss asset values.|
|Recommendation #10||Omit non trading-centric asset types from the GMS, as is currently the case with the carve-out of fair value non-trading loans. These assets would remain subject to PPNR modelling.|