Since the financial crisis, stress tests have become an important supervisory and financial stability tool (Constâncio 2017). Against this background, an important question is whether stress tests contribute to financial stability by promoting risk reduction in the banking sector, as recent evidence suggests (Cortes et al. 2017, Acharya et al. 2018, Steri and Pierret 2018). Stress tests offer deep insights into banks’ vulnerabilities to supervisors and the public through an intense supervisory process. Recent evidence suggests that supervisory scrutiny decreases banks’ risk-taking activities (Rezende and Wu 2014, Hirtle et al. 2018, Bonfim et al. 2018, Kandrac and Schlusche 2019). In the same vein, in a recent study, we show that higher supervisory scrutiny led to a disciplining effect for banks in the part of the European Banking Authority (EBA) EU-wide stress test conducted by the ECB in 2016.

How supervisory scrutiny is exerted in stress tests 

In Europe, stress tests involve interactions between banks and supervisors on banks' risk management practices as well as confidential communications about best stress-testing practices and techniques. We use data on these confidential interactions to approximate how much scrutiny was exerted on Single Supervisory Mechanism (SSM) banks under the direct supervision of the ECB during the 2016 EU-wide stress test. These interactions arise as part of the constrained bottom-up approach pursued in the EBA-coordinated exercises (see Figure 1). In this context, banks use their own internal models to generate projections (e.g. for credit losses). Meanwhile, banks' projections are challenged by the competent supervisory authorities typically by applying top-down models and other challenger tools. In the presence of material deviations between these two sets of projections, ‘flags’ are triggered and are later discussed between supervisors and banks. Banks need to either comply with or explain the issues raised in the interactions with the ECB. We construct the scrutiny measure by counting the flags related to credit risk projections. Intuitively, banks that received more flags had to work harder on their resubmissions and had lengthier and probably more intense interactions with supervisors while banks that received no flags in principle had no further interactions with supervisors. 

Figure 1 Simplified illustration of one quality assurance cycle under the constrained bottom-up approach

Source: Own illustration based on Mirza and Zochowski (2017).

Scrutiny measures the intensity of stress tests

We apply a differences-in-differences approach where we use the stress test as a treatment and the involved scrutiny as a measure of the intensity of the treatment. In a first step, we compare the credit risk of banks that were part of the stress test and banks that were not part of the stress test four quarters before and four quarters after the 2016 stress test. In a second step, we compare credit risk of banks that were subject to a more intense supervisory scrutiny and banks that received less or none.¹ The 2016 EU-wide stress test was executed on significant institutions (SIs). Less significant institutions (LSIs) were not tested and we therefore use them as the control group.²   

The effect of supervisory scrutiny on credit risk

We focus our analysis on credit risk, which accounts for a large part of the stress testing projections and on average for 86% of risk exposure amounts in bank balance sheets. To measure credit risk at the bank level, we use the risk-weight density (RWD), i.e. the aggregate risk weight assigned to total credit risk exposures according to regulatory standards.

Figure 2 Estimate and 90% confidence interval of the differential effect on RWD between tested and non-tested banks before and after the reported quarter. 

We find no significant difference in RWD between the treatment and the control group before the stress test (2015q1 and 2015q4; see Figure 2) but significant negative differences for the period after the test (2017q1 to 2017q4). The reduction in RWD of tested banks after the stress test was on average 4.2 percentage points lower than the reduction of not-tested banks. This effect is economically material as it amounts to a change of about 20% of the standard deviation of RWD.³ These results confirm the findings based on US data that ‘treating’ banks with stress tests can affect their risk.

Figure 3 Marginal effect and average effect estimates with 90% confidence intervals of the differential effect of scrutiny intensity on RWD 

Second, we show that the more interactions banks had with supervisors, the higher their reduction in RWD after the stress test exercise (see Figure 3). We find that those banks that received more scrutiny (the half with scrutiny intensity above the median) exhibit a 5.6 percentage point greater decrease in credit risk than the half that received less scrutiny.⁴ All in all, these findings provide novel evidence that the tighter and more intrusive supervisory scrutiny associated to the EU-wide stress-test has the potential to enhance banks' risk management practices and to induce lower bank risk.

Policy implications

We contribute to the emerging evidence on the effectiveness of supervisory scrutiny. Our results suggest that stress tests conducted applying a robust quality assurance of banks' projections and models have disciplining effects on stress-tested banks’ risk. However, it has to be noted that one of the stress tests' primary objectives is to correctly assess banks' risk profiles. Our findings do not provide information on how well this objective is met. The possible strategic underreporting of banks' vulnerabilities under a bottom-up approach could undermine the reliability of the stress test outcomes from this perspective (Niepmann and Stebunovs 2018). Pursuing a more unbiased top-down approach while retaining supervisory interactions with banks during and after the stress test might be more suitable to achieve this goal. We, therefore, with our analysis, only deliver one insight among many that could serve the policy discussion on the future stress test design in Europe.  

References

Acharya, V, A Berger and R Raluca (2018), “Lending Implications of U.S. Bank Stress Tests: Costs or Benefits?”, Journal of Financial Intermediation 34: 58—90.

Bonfim, D, G Cerqueiro, H Degryse and S Ongena (2020), “On-site inspecting zombie lending”, CEPR Discussion Paper No. 14754.

Constâncio, V (2017), “Macroprudential stress tests: A new analytical tool”, VoxEU.org, 22 February.

Cortés, K, Y Demyanyk, L Li, E Loutskina and P E Strahan (2020), “Stress tests and small business lending”, Journal of Financial Economics 136(1): 260—279.

ECB (2019), “What makes a bank significant?”, European Central Bank Banking Supervision.

Hirtle, B, A Kovner and M Plosser (2020), “The Impact of Supervision on Bank Performance”, The Journal of Finance 75(5): 2765—2808.

Ivanov, I and J Wang (2019), “The impact of bank supervision on corporate credit”, Working Paper.

Kandrac, J and B Schlusche (2017), “The effect of bank supervision on risk taking: Evidence from a natural experiment”, Finance and Economics Discussion Series No. 79, Board of Governors of the Federal Reserve System.

Kok, C, C Müller, S Ongena and C Pancaro (2021), “The Disciplining Effect of Supervisory Scrutiny in the EU‑wide Stress Test”, CEPR Discussion Paper No. 16157.

Mirza, H and D Zochowski (2017), “Stress Test Quality Assurance from a Top-Down Perspective”, Macroprudential Bulletin 3.

Niepmann, F and V Stebunovs (2018), “How EU banks modelled their stress away in the 2016 EBA stress tests”, VoxEU.org, 30 July.

Pierret, D and R Steri (2019), “Stressed Banks”, Swiss Finance Institute Research Paper No. 58.

Rezende, M and J Wu (2014), “The effects of supervision on bank performance: Evidence from discontinuous examination frequencies”, Midwest Finance Association 2013 Annual Meeting Paper.

Endnotes

1 We also compare banks that received more treatment to banks that received less treatment excluding banks that received no treatment at all, i.e. the effect of supervisory scrutiny within the tested banks.

2 Significant institutions are those SSM banks that fulfil certain criteria (ECB 2019). LSIs are SSM banks that do not fulfil any of the criteria. LSIs are directly supervised by the national competent authorities under the oversight of the ECB which ensures the consistency of the regulatory framework and supervisory practices applied to these banks. To account for the differences between SIs and LSIs, we include robustness checks where we use matching and sample exclusions to estimate the effect in a sample with minimized differences. 

3 The mean of RWD of tested banks before the stress test is 43.7 percentage points with a standard deviation of 22.5 percentage points.

4 An increase in supervisory scrutiny intensity by 10% decreases RWD on average by around 0.27 percentage points.