Nash–Sutcliffe model efficiency coefficient

The Nash–Sutcliffe model efficiency coefficient (NSE) is used to assess the predictive skill of hydrological models. It is defined as:

${\displaystyle NSE=1-{\frac {\sum _{t=1}^{T}\left(Q_{o}^{t}-Q_{m}^{t}\right)^{2}}{\sum _{t=1}^{T}\left(Q_{o}^{t}-{\overline {Q}}_{o}\right)^{2}}}}$

where ${\textstyle {\overline {Q}}_{o}}$ is the mean of observed discharges, and ${\textstyle Q_{m}}$ is modeled discharge. ${\textstyle Q_{o}^{t}}$ is observed discharge at time t.^[1]

The Nash–Sutcliffe efficiency is calculated as one minus the ratio of the error variance of the modeled time-series divided by the variance of the observed time-series. In the situation of a perfect model with an estimation error variance equal to zero, the resulting Nash–Sutcliffe Efficiency equals 1 (NSE = 1). Conversely, a model that produces an estimation error variance equal to the variance of the observed time series results in a Nash–Sutcliffe Efficiency of 0.0 (NSE = 0). In reality, NSE = 0 indicates that the model has the same predictive skill as the mean of the time-series in terms of the sum of the squared error. In the case of a modeled time series with an estimation error variance that is significantly larger than the variance of the observations, the NSE becomes negative. An efficiency less than zero (NSE < 0) occurs when the observed mean is a better predictor than the model. Values of the NSE nearer to 1, suggest a model with more predictive skill. Subjective application of different NSE values as thresholds of sufficiency have been suggested by several authors.^[2][3][4][5] For the application of NSE in regression procedures (i.e. when the total sum of squares can be partitioned into error and regression components), the Nash–Sutcliffe efficiency is equivalent to the coefficient of determination (R²), thus ranging between 0 and 1.

In some applications such as automatic calibration or machine learning, the NSE lower limit of (-∞) creates problems. To eliminate this problem and re-scale the NSE to lie solely within the range of {0,1} normalization, use the following equation that yields a Normalized Nash–Sutcliffe Efficiency (NNSE) ^[6]

${\displaystyle NNSE={\frac {1}{2-NSE}}}$

Note that NSE=1 corresponds to NNSE=1, NSE=0 corresponds to NNSE=0.5, and NSE=-∞ corresponds to NNSE=0. This convenient re-scaling of the NSE allows for easier interpretation, and use of the NSE measure in parameter estimation schemes used in model calibration.

The NSE coefficient is sensitive to extreme values and might yield sub-optimal results when the dataset contains large outliers. To address this a modified version of NSE has been suggested where the sums of squares in the numerator and denominator of NSE are raised to 1 instead of 2 and the resulting modified NSE values compared to the original NSE values to assess the potential effect of extreme values.^[7] Importantly, this modification relies on the absolute value in lieu of the square power:

${\displaystyle NSE_{1}=1-{\frac {\sum _{t=1}^{T}\left|Q_{o}^{t}-Q_{m}^{t}\right|}{\sum _{t=1}^{T}\left|Q_{o}^{t}-{\overline {Q}}_{o}\right|}}}$

A test significance for NSE to assess its robustness has been proposed whereby the model can be objectively accepted or rejected based on the probability value of obtaining NSE greater than some subjective threshold.

Nash–Sutcliffe efficiency can be used to quantitatively describe the accuracy of model outputs other than discharge. This indicator can be used to describe the predictive accuracy of other models as long as there is observed data to compare the model results to. For example, Nash–Sutcliffe efficiency has been reported in scientific literature for model simulations of discharge; water quality constituents such as sediment, nitrogen, and phosphorus loading.^[5] Other applications are the use of Nash–Sutcliffe coefficients to optimize parameter values of geophysical models, such as models to simulate the coupling between isotope behavior and soil evolution.^[8]

Criticism

The Nash–Sutcliffe Coefficient masks important behaviors that if re-cast can aid in the interpreted as the different sources of model behavior in terms of bias, random, and other components.^[9] The alternate "Kling-Gupta" efficiency does not have the same bounds as the NSE^[10]

See also

• Coefficient of determination

References

1. Nash, J. E.; Sutcliffe, J. V. (1970). "River flow forecasting through conceptual models part I — A discussion of principles". Journal of Hydrology. 10 (3): 282–290. Bibcode:1970JHyd...10..282N. doi:10.1016/0022-1694(70)90255-6.
2. McCuen, R.H.; Knight, Z; Cutter, A.G. (2006). "Evaluation of the Nash–Sutcliffe efficiency index". Journal of Hydrologic Engineering. 11 (6): 597–602. doi:10.1061/(ASCE)1084-0699(2006)11:6(597).
3. Criss, R.E; Winston, W.E (2008). "Do Nash values have value? Discussion and alternate proposals". Hydrological Processes. 22 (14): 2723–2725. Bibcode:2008HyPr...22.2723C. doi:10.1002/hyp.7072.
4. Ritter, A.; Muñoz-Carpena, R. (2013). "Performance evaluation of hydrological models: statistical significance for reducing subjectivity in goodness-of-fit assessments". Journal of Hydrology. 480 (1): 33–45. Bibcode:2013JHyd..480...33R. doi:10.1016/j.jhydrol.2012.12.004.
5. Moriasi, D. N.; Arnold, J. G.; Van Liew, M. W.; Bingner, R. L.; Harmel, R. D.; Veith, T. L. (2007). "Model Evaluation Guidelines for Systematic Quantification of Accuracy in Watershed Simulations" (PDF). Transactions of the ASABE. 50 (3): 885–900. doi:10.13031/2013.23153.
6. Nossent, J; Bauwens, W (2012). "Application of a normalized Nash–Sutcliffe efficiency to improve the accuracy of the Sobol'sensitivity analysis of a hydrological model". EGUGA: 237. Bibcode:2012EGUGA..14..237N.
7. Legates, D.R.; McCabe, G.J. (1999). "Evaluating the use of "goodness-of-fit" measures in hydrologic and hydroclimatic model validation". Water Resour. Res. 35 (1): 233–241. Bibcode:1999WRR....35..233L. doi:10.1029/1998WR900018.
8. Campforts, Benjamin; Vanacker, Veerle; Vanderborght, Jan; Baken, Stijn; Smolders, Erik; Govers, Gerard (2016). "Simulating the mobility of meteoric 10 Be in the landscape through a coupled soil-hillslope model (Be2D)". Earth and Planetary Science Letters. 439: 143–157. Bibcode:2016E&PSL.439..143C. doi:10.1016/j.epsl.2016.01.017. ISSN 0012-821X.
9. Gupta, H.V.; Kling, H (2011). "On typical range, sensitivity, and normalization of Mean Squared Error and Nash‐Sutcliffe Efficiency type metrics". Water Resources Research. 47 (10): W10601. Bibcode:2011WRR....4710601G. doi:10.1029/2011WR010962.
10. Knoben, W.J; Freer, J.E.; Woods, R.A. (2019). "Inherent benchmark or not? Comparing Nash–Sutcliffe and Kling-Gupta efficiency scores". Hydrology and Earth System Sciences. 23 (10): 4323–4331. doi:10.5194/hess-23-4323-2019.