[Update 15 Nov 2016: Calculations available as a spreadsheet]
RORB provides 7 statistics to summarise the match between calculated and actual hydrographs. A comparison of:
- Peak discharge (m3/s)
- Time to peak (hour)
- Volume (m3)
- Average absolute coordinate error (m3/s)
- Time to centroid (hour)
- Lag (centre of mass to centre of mass) (hour)
- Lag to peak (hour)
An example is shown in Figure 1 below which compares a calculated and actual hydrograph for the Werribee River. This is taken from section 10.2.1 of the RORB manual. The difference between the hydrographs is summarised as an absolute error and a percentage error for each statistic.
During calibration, the losses and kc can be varied until the modeller is satisfied there is a reasonable match between the calculated and actual hydrographs across the range of statistics. Generally modellers focus on peak discharge and volume but the other error measures are also useful. For example, problems in the speed of the hydrograph (time to centroid and lag) can be related to the number of sub-catchments used in the RORB model (see the discussion here).
There is a brief summary of these statistics in Section 18.104.22.168 of the RORB manual. In this blog I’ll discuss them in more detail.
Figure 1: Error statistics provided by RORB based on the differences between calculated and actual hydrographs
Example data set
Example calculations are made using data from the Werribee River worked example (Section 10.2 of the RORB manual). The actual and calculated hydrographs are listed on page 111 and 112. See Figure 2.
Figure 2: Calculated and actual hydrographs for the Werribee worked example (p111 – 112 of the RORB manual)
The error in peak discharge is the difference between the maximum of the calculated RORB hydrograph and the maximum of the actual hydrograph provided as input to RORB.
For the Werribee example:
max(actual) = 356 m3/s
max(calculated) = 337.999 m3/s
Absolute difference = max(calculated) – max(actual) = 337.999 – 356 = – 18 m3/s
Percentage difference = 100×(max(calculated) – max(actual))/max(actual)
Note that these error measures are expressed in a the opposite way to that used by statisticians. In statistics, residuals are usually expressed actual – calculated e.g. . Here we are reporting calculated – actual. So if the difference is negative, the calculated value is too small.
Time to peak
This is where it starts to get interesting. Looking at the data in Figure 2, the peak flow for both the actual and calculated hydrographs occurs at the 9th time increment, which Figure 2 suggests, is at 18 hours. However, look at the statistics in Figure 1; the time to peak is listed as occurring at 16 hours. What’s going on?
We can get a clue by looking at the RORB control vector for the Werribee case study (see Figure 3). The first line highlighted in green show that the time increment is 2 hours and calculations will be undertaken for 28 increments. The second line in green shows that the two actual hydrographs (Melton Reservoir outflow and Werribee weir) both start at the zeroth time increment and end at the 28th time increment. The trick is that increments start at zero not 1. The increments shown in Figure 2 start at 1 not zero. This is an error in reporting. The first flows listed in Figure 2 actually occur at time zero so the maximum flows occur after 16 hours, not 18 hours. Its a bit hard to tell, but if you look closely at the graph in Figure 1, the maximum flow occurs at 16 hours not 18 hours.
Figure 3: RORB control vector from the Werribee case study (p106 of the RORB manual)
There seems to be an ‘out by one’ error in the RORB output file. Other case studies in the RORB manual have the same issue. This creates an inconsistency between the on-screen statistics, such as those shown in Figure 1, and the RORB text output.
RORB estimates hydrograph volumes as the mean flow times the length of the hydrograph.
For the data shown in Figure 2, the means are:
- Calculated 119.0093 m3/s
- Actual 122.4138 m3/s
To convert these mean flows to volumes (m3) multiply by the length of the hydrograph. In this case, there are 29 increments (remember we start at zero and finish at 28). Each increment is 2 hours long so we need to multiply by 2 × 60 × 60 to convert hours to seconds.
- Volume of calculated hydrograph 119.0093 × 29 × 2 × 60 × 60 = 24848021 ≈ 0.25 × 108 m3
- Volume of actual hydrograph 122.4138 × 29 × 2 × 60 × 60 = 25560000 ≈ 0.26 × 108 m3
The absolute difference is 24848021 – 25560000 = -711979 ≈ -0.71 × 106 m3
100 × (24848021 – 25560000)/25560000 ≈ -2.8%
Average absolute coordinate error
The average absolute coordinate error is the mean of absolute values of the differences between the actual and calculated hydrograph at each time step.
- AACE is the average absolute coordinate error
- is the flow at each time increment for the actual hydrograph
- is the flow at each time increment for the calculated hydrograph
- There are time increments.
For the Werribee case study, the first time step is at zero and the final is at 28 so there are 29 increments in total.
The average absolute coordinate error =
The percentage error is the average absolute coordinate error divided by the average flow of the actual hydrograph.
For the Werribee case study, the average flow during the actual hydrograph is 122.4138 m3/s so the percentage average absolute coordinate error is:
Time to centroid
The time to the centroid of the hydrograph can be be calculated as sum of flow × time divided by the sum of the flow
Looking at the data in Figure 2, and correcting for the fact that the first time step is at zero, not 2 hours, the centroid for the actual hydrograph will be:
Time to centroid (actual) = 24.11662 hour
Time to centroid (calculated) = 23.44448 hour
Absolute error = 23.4 4- 24.11 = -0.672 ≈-0.7 hour
Percentage error in the time to centroid is the absolute error divided by the time to centroid of the actual hydrograph.
Lag (centre of mass to centre of mass)
The RORB manual (p101) defines the lag (c.m to c.m) as:
the time from the centroid of concentrated inputs to the channels upstream of the point concerned, including both sub-area inflows and concentrated channel inflow hydrographs, to the time of centroid of the hydrograph.
For the Werribee worked example, the concentrated input is the hydrograph at Melton Reservoir. This is listed in Figure 3 (the two lines under the green box):
0, 0, 66, 150, 253, …
The time to the centroid of this hydrograph can be calculated using the same procedure as before.
Time to centroid of the input hydrograph at Melton Reservoir is 19.90657 hour.
The difference in time between the centre of mass of the input hydrograph and that of the actual hydrograph is:
24.111662 – 19.90657 = 4.21 ≈ 4.2 hour
The difference in time between the centre of mass of the input hydrograph and that of the calculated hydrograph is:
23.44448 – 19.90657 = 3.53823 ≈ 3.5379 hour
The absolute difference between these values is:
3.5379- 4.2101 ≈ -0.672 ≈ -0.7 hour
The percentage difference is the absolute difference divided by the difference in time between the centre of mass of the input hydrograph and that of the actual hydrograph.
100 × -0.672/4.2101 = -15.965% ≈ -16%
So the calculated value is -0.7 hour (16%) smaller than the actual value. This means the calculated hydrograph is moving through the modelled reach of the Werribee River more quickly than the real hydrograph is moving through the real reach. A modeller could adjust kc to slow the modelled hydrograph (but this would also affect other aspects of the calculated hydrograph, such as the peak).
Lag to peak
The lag to peak is the time from the centroid of the concentrated input to channels upstream of the point concerned, including both sub-areas inflows and concentrated channel inflow hydrographs, to time of peak of the hydrograph.
For Werribee worked example, the time to the centroid of the input hydrograph (at Melton Reservoir) is 19.90657 hour and the peak of the actual output hydrograph is at 16 hours so the absolute difference is:
16 – 19.90657 ≈ -3.9 hours
The peak of the calculated hydrograph is also at 16 hours so the lag to peak of the calculated and actual hydrographs is the same i.e. the error is zero.
The range of error statistics in RORB provides a summary of model performance which can be used as a guide during calibration. Most modellers concentration on matching peaks and volumes but the other statistics also provide insight into aspects where the model can be improved.
Example calculations are available as an excel spreadsheet and a gist.