Banks that participated in the 2016 EU-wide stress test subsequently
reduced their credit risk relative to banks that were not part of this
exercise. Relying on new metrics for supervisory scrutiny, it also shows
that the disciplining effect is stronger for banks subject to more
intrusive supervisory scrutiny during the stress test.
Since the financial crisis, stress tests have become an
important supervisory and financial stability tool (Constâncio 2017).
Against this background, an important question is whether stress tests
contribute to financial stability by promoting risk reduction in the
banking sector, as recent evidence suggests (Cortes et al. 2017, Acharya
et al. 2018, Steri and Pierret 2018). Stress tests offer deep insights
into banks’ vulnerabilities to supervisors and the public through an
intense supervisory process. Recent evidence suggests that supervisory
scrutiny decreases banks’ risk-taking activities (Rezende and Wu 2014,
Hirtle et al. 2018, Bonfim et al. 2018, Kandrac and Schlusche 2019). In
the same vein, in a recent study, we show that higher supervisory
scrutiny led to a disciplining effect for banks in the part of the
European Banking Authority (EBA) EU-wide stress test conducted by the
ECB in 2016.
How supervisory scrutiny is exerted in stress tests
In Europe, stress tests involve interactions between banks and
supervisors on banks' risk management practices as well as confidential
communications about best stress-testing practices and techniques. We
use data on these confidential interactions to approximate how much
scrutiny was exerted on Single Supervisory Mechanism (SSM) banks under
the direct supervision of the ECB during the 2016 EU-wide stress test.
These interactions arise as part of the constrained bottom-up approach
pursued in the EBA-coordinated exercises (see Figure 1). In this
context, banks use their own internal models to generate projections
(e.g. for credit losses). Meanwhile, banks' projections are challenged
by the competent supervisory authorities typically by applying top-down
models and other challenger tools. In the presence of material
deviations between these two sets of projections, ‘flags’ are triggered
and are later discussed between supervisors and banks. Banks need to
either comply with or explain the issues raised in the interactions with
the ECB. We construct the scrutiny measure by counting the flags
related to credit risk projections. Intuitively, banks that received
more flags had to work harder on their resubmissions and had lengthier
and probably more intense interactions with supervisors while banks that
received no flags in principle had no further interactions with
Figure 1 Simplified illustration of one quality assurance cycle under the constrained bottom-up approach
Source: Own illustration based on Mirza and Zochowski (2017).
Scrutiny measures the intensity of stress tests
We apply a differences-in-differences approach where we use the
stress test as a treatment and the involved scrutiny as a measure of the
intensity of the treatment. In a first step, we compare the credit risk
of banks that were part of the stress test and banks that were not part
of the stress test four quarters before and four quarters after the
2016 stress test. In a second step, we compare credit risk of banks that
were subject to a more intense supervisory scrutiny and banks that
received less or none.1 The 2016 EU-wide stress test was
executed on significant institutions (SIs). Less significant
institutions (LSIs) were not tested and we therefore use them as the
The effect of supervisory scrutiny on credit risk
We focus our analysis on credit risk, which accounts for a large part
of the stress testing projections and on average for 86% of risk
exposure amounts in bank balance sheets. To measure credit risk at the
bank level, we use the risk-weight density (RWD), i.e. the aggregate
risk weight assigned to total credit risk exposures according to
Figure 2 Estimate and 90% confidence interval of the
differential effect on RWD between tested and non-tested banks before
and after the reported quarter.
We find no significant difference in RWD between the treatment and
the control group before the stress test (2015q1 and 2015q4; see Figure
2) but significant negative differences for the period after the test
(2017q1 to 2017q4). The reduction in RWD of tested banks after the
stress test was on average 4.2 percentage points lower than the
reduction of not-tested banks. This effect is economically material as
it amounts to a change of about 20% of the standard deviation of RWD.3 These results confirm the findings based on US data that ‘treating’ banks with stress tests can affect their risk.
Figure 3 Marginal effect and average effect
estimates with 90% confidence intervals of the differential effect of
scrutiny intensity on RWD
Second, we show that the more interactions banks had with
supervisors, the higher their reduction in RWD after the stress test
exercise (see Figure 3). We find that those banks that received more
scrutiny (the half with scrutiny intensity above the median) exhibit a
5.6 percentage point greater decrease in credit risk than the half that
received less scrutiny.4 All in all, these findings provide
novel evidence that the tighter and more intrusive supervisory scrutiny
associated to the EU-wide stress-test has the potential to enhance
banks' risk management practices and to induce lower bank risk.
We contribute to the emerging evidence on the effectiveness of
supervisory scrutiny. Our results suggest that stress tests conducted
applying a robust quality assurance of banks' projections and models
have disciplining effects on stress-tested banks’ risk. However, it has
to be noted that one of the stress tests' primary objectives is to
correctly assess banks' risk profiles. Our findings do not provide
information on how well this objective is met. The possible strategic
underreporting of banks' vulnerabilities under a bottom-up approach
could undermine the reliability of the stress test outcomes from this
perspective (Niepmann and Stebunovs 2018). Pursuing a more unbiased
top-down approach while retaining supervisory interactions with banks
during and after the stress test might be more suitable to achieve this
goal. We, therefore, with our analysis, only deliver one insight among
many that could serve the policy discussion on the future stress test
design in Europe.
Acharya, V, A Berger and R Raluca (2018), “Lending Implications of U.S. Bank Stress Tests: Costs or Benefits?”, Journal of Financial Intermediation 34: 58—90.
Bonfim, D, G Cerqueiro, H Degryse and S Ongena (2020), “On-site inspecting zombie lending”, CEPR Discussion Paper No. 14754.
Constâncio, V (2017), “Macroprudential stress tests: A new analytical tool”, VoxEU.org, 22 February.
Cortés, K, Y Demyanyk, L Li, E Loutskina and P E Strahan (2020), “Stress tests and small business lending”, Journal of Financial Economics 136(1): 260—279.
ECB (2019), “What makes a bank significant?”, European Central Bank Banking Supervision.
Hirtle, B, A Kovner and M Plosser (2020), “The Impact of Supervision on Bank Performance”, The Journal of Finance 75(5): 2765—2808.
Ivanov, I and J Wang (2019), “The impact of bank supervision on corporate credit”, Working Paper.
Kandrac, J and B Schlusche (2017), “The effect of bank supervision on
risk taking: Evidence from a natural experiment”, Finance and Economics
Discussion Series No. 79, Board of Governors of the Federal Reserve
Kok, C, C Müller, S Ongena and C Pancaro (2021), “The Disciplining Effect of Supervisory Scrutiny in the EU‑wide Stress Test”, CEPR Discussion Paper No. 16157.
Mirza, H and D Zochowski (2017), “Stress Test Quality Assurance from a Top-Down Perspective”, Macroprudential Bulletin 3.
Niepmann, F and V Stebunovs (2018), “How EU banks modelled their stress away in the 2016 EBA stress tests”, VoxEU.org, 30 July.
Pierret, D and R Steri (2019), “Stressed Banks”, Swiss Finance Institute Research Paper No. 58.
Rezende, M and J Wu (2014), “The effects of supervision on bank
performance: Evidence from discontinuous examination frequencies”,
Midwest Finance Association 2013 Annual Meeting Paper.
1 We also compare banks that received more treatment to banks that
received less treatment excluding banks that received no treatment at
all, i.e. the effect of supervisory scrutiny within the tested banks.
2 Significant institutions are those SSM banks that fulfil certain
criteria (ECB 2019). LSIs are SSM banks that do not fulfil any of the
criteria. LSIs are directly supervised by the national competent
authorities under the oversight of the ECB which ensures the consistency
of the regulatory framework and supervisory practices applied to these
banks. To account for the differences between SIs and LSIs, we include
robustness checks where we use matching and sample exclusions to
estimate the effect in a sample with minimized differences.
3 The mean of RWD of tested banks before the stress test is 43.7
percentage points with a standard deviation of 22.5 percentage points.
4 An increase in supervisory scrutiny intensity by 10% decreases RWD on average by around 0.27 percentage points.