Category Archives: R

TQmean, a measure of the impact of urbanisation on flow

Urban development has a profound impact on flow and hydrologic indicators have been proposed to highlight the changes as suburbs spread over a catchment, increasing impermeable area.  A commonly used measure of impact is TQmean– the proportion of time that flow in each year is greater than mean flow for that year.  This decreases with urbanisation, and has been shown to be linked to ecological condition of a stream (Booth et al., 2004).
As an example, consider two neighbouring streams in eastern Melbourne: Brushy Creek which flows through the suburb of Croydon, with a catchment that is 28% impervious, and Olinda Creek with a catchment that is mainly forested and which is 5% impervious.
Often, the value of TQmean is calculated and then averaged for several years. Using this approach, The TQmean for Brushy Creek, is 0.21, with the less urbanised Olinda Creek, having a value of 0.37 (for the period 1988 to 2016). Using the relationships in Booth et al. (2004) this suggests Olinda Creek would be predicted to have ‘good’ biological condition, while Brushy Creek would be predicted to be ‘very poor’.

Calculating a single value of TQmean is instructive but the temporal distribution of the time of the year when flows exceed the mean is also altered by urbanisation. Using TQmean as the metric, urbanisation results in high flows occurring more often, but for shorter periods, that are dispersed throughout the year. Comparing the plots of Olinda Creek and Brushy Creek in  (figure), higher flows (flows above the mean), are clustered in the winter (June to August) for Olinda Creek. There is more winter runoff because the Olinda Creek catchment wets up owing to the higher rainfall, and reduced evaporation that occur seasonally in this area. For Brushy Creek, with the same climate, short bursts of high flow occur throughout the whole year. This can be attributed to runoff from impervious surfaces which will occur anytime there is rain.

TQmean_Olinda

TQmean_Brushy.png

Figure 1:  Periods of flow above and below the mean flow for (A) Olinda Ck and (B) Brushy Ck

A key issue revealed by this analysis is the changed seasonality of high flow which is a result of urbanisation. This is just one of the many changes in flow regime caused by urbanisation that leads to poor stream condition (Burns et al. 2014).

Code to plot these graphs is available as a gist.

 

Graphing a long flow series

A long series of flows can be challenging to show graphically without squeezing the data so much that all the useful information is lost (Figure 1).  Two approaches are shown here.  First, a ‘cut-and-stack’ plot, which takes a long graph and cuts it into segments equal to the width of a page.  These segments are stacked on top of each other, stretching out the x-axis (Figure 2).  The figure shows the flows for each decade of the ~ 50 years of data for the Broken River at Caseys Weir (Gauge 404216).

An alternative is a trellis or facet plot (Figure 3).  Here, the flow in each year is plotted as a separate graph.  If the y-axis scale is held constant across all years, the overall temporal variation is highlighted and the very dry years stand out (for example 2006-2009).

If the scales are varied for each year the seasonal flow patterns are emphasised (Figure 4).  The transfers from upstreams dams standout with the rectangular hydrographs in 1977, 1982, 1983 and the summer of 1983-84.

flow_Broken_Caseys.png

Figure 1: Broken River at Caseys Weir (404216) 31 March 1972 to 19 April 2017

flow_Broken_Caseys_C&S

Figure 2: Cut-and-stack plot of the mean daily flow for the Broken River at Caseys Weir (flow data is the same as shown in Figure 1)

flow_Broke_Caseys_facet_5yr

Figure 3: Facet plot of the mean daily flow for the Broken River at Caseys Weir.  Flow data is the same as shown in Figure 1; y-axis scaling is held contant

flow_Broken_Caseys_5yr_free_y

Figure 4: Same as Figure 3 except that y-axis scaling varies between years

OLYMPUS DIGITAL CAMERA

Figure 5: Broken River at Caseys Weir (20 Mar 2017)

Data for gauge 404216 was obtained from the Victorian Water Measurement Information System (WMIS).

R code to produce the graphs in this blog is available as gists (here for the cut-and-stack plot; and here for the facet plots).

Better line graphs for hydrologic data

Line graphs are commonly used for comparisons of hydrologic data but they can become difficult to interpret if there are too many lines; the derogatory term is spaghetti graphs.  In my work reviewing flood studies I’ve seen some spaghetti.

The example below highlights common issues with these types of figures. It is difficult to unambiguously determine how the lines relate to the legend because the colours are not distinct and the lines overlap. Also, in this case the x-axis scaling is not meaningful.  The same distance on the graph represents different time periods: 3 hours between the first 2 tick marks (9 to 12) and 24 hours between the final 2 tick marks (48 to 72).

Line graph-1

Figure 1: Line graph – spaghetti version

An improved version is shown in Figure 2. Colours are distinct, lines are labelled directly, (rather than using a legend), the x-axis scaling is consistent, using a thousands separator makes it easier to read the y-axis values. Both ends of a line can be labelled where this contributes to clarity. A grey background can help make it easier to see the data and allows the use of unintrusive pale gridlines.

Ling graph-2

Figure 2: Line graph – improved version

Some may prefer the minimalist version shown in Figure 3.  This attempts to maximise the information to ink ratio by reducing “chart junk” – those parts of a chart that don’t contribute to understanding (Tufte, 1983).

Line graph-4

Figure 3: Line graph – minimalist version

Code to reproduce these figures is available as a gist.

References

 

Cleveland, W. S. (1993) Visualizing Data.  Hobart Press.

Tufte, E. R. (1983) The visual display of quantitative information.  Graphics Press.  Cheshire, CT, USA.

Better flood frequency plots from Flike – II

I’ve previously written about improving the flood frequency plots from Flike.   This is an update to that earlier post.

Using Flike version 5.0.300.0 I’ve fitted a Log Pearson III distribution to the annual series for the Tyers River at Browns (226007) using data from 1963 to 2007.  The graph produced by Flike is shown in Figure 1.  This graph is fine to show the quality of the fit but it would be nice to polish it for incorporation into a report.  A csv file of the flow data input to Flike is is available here; the Flike .fld file is here.

Flikeplot_Tyers_Browns

Figure 1: Flike plot – flood frequency curve for the Tyers River at Browns (226007) (1963-2007).  Log Pearson III probability model

The data used to create the plot can be downloaded into a csv file by clicking the ‘Save’ button below the plot (the file associated with this graph is available here).  There are three parts to the resulting file:

  1. the data points – deviates and gauged values
  2. points specifying the expected parameter quantiles and confidence limits
  3. points specifying the expected probability quantiles.

These can be read into Excel, or a graphics program, and plotted.   An example is shown in Figure 2.

A key enhancement in this figure, compared to the standard Flike plot, is that the y-axis tick marks are labelled with the flow values rather than logs. The log transformation has been retained, just the labelling has been changed. The x-axis tick mark labels are similar. The deviate values are plotted but are labelled using the ‘1 in Y’ format.

Although it takes some time to construct and label a plot, much of the work can be repeatedly re-used in future reports. If you are doing a lot of flood frequency analysis, its worth setting up a template.

Tyers_Browns_FFA_plot

I’ve used the ggplot2 package in R to produce this plot.  Details are available via this gist.

R Packages for Hydrology

The Environmetrics Task View contains information about using R to analyse ecological and environmental data.  Importantly it includes a list of packages useful for hydrology.

I’ve extracted part of the task view here but please also check the original.  Acknowledgements to Gavin Simpson.

  • Package HydroMe estimates the parameters in infiltration and water retention models by curve-fitting method.
  • hydroTSM is a package for management, analysis, interpolation and plotting of time series used in hydrology and related environmental sciences.
  • hydroGOF is a package implementing both statistical and graphical goodness-of-fit measures between observed and simulated values, mainly oriented to be used during the calibration, validation, and application of hydrological/environmental models. Related packages are tiger, which allows temporally resolved groups of typical differences (errors) between two time series to be determined and visualized, and qualV which provides quantitative and qualitative criteria to compare models with data and to measure similarity of patterns
  • hydroPSO is a model-independent global optimization tool for calibration of environmental and other real-world models that need to be executed from the system console.hydroPSO implements a state-of-the-art PSO (SPSO-2011 and SPSO-2007 capable), with several fine-tuning options. The package is parallel-capable, to alleviate the computational burden of complex models.
  • EcoHydRology provides a flexible foundation for scientists, engineers, and policy makers to base teaching exercises as well as for more applied use to model complex eco-hydrological interactions.
  • topmodel is a set of hydrological functions including an R implementation of the hydrological model TOPMODEL, which is based on the 1995 FORTRAN version by Keith Beven. New functionality is being developed as part of the RHydro package on R-Forge.
  • dynatopmodel is a native R implementation and enhancement of the Dynamic TOPMODEL, Beven and Freers’ (2001) extension to the semi-distributed hydrological model TOPMODEL (Beven and Kirkby, 1979).
  • wasim provides tools for data processing and visualisation of results of the hydrological model WASIM-ETH
  • The nsRFA package provides collection of statistical tools for objective (non-supervised) applications of the Regional Frequency Analysis methods in hydrology.
  • The boussinesq package is a collection of functions implementing the one-dimensional Boussinesq Equation (ground-water).
  • rtop is a package for geostatistical interpolation of data with irregular spatial support such as runoff related data or data from administrative units.

 

Graphing a water balance

The water balance for a urban catchment equates the change in storage during a certain period, with the difference between water inputs (precipitation and mains water) and water outputs (evaporation, stormwater runoff and wastewater discharge).

\Delta s = (P+I) - (E_a + R_s + R_w)
where:

\Delta s change in catchment storage
P precipitation
I  imported water
E_a actual evaportranspiration
R_s stormwater runoff
R_w wastewater discharge

Mitchell et al., (2003) provides data on the water balance for Curtin, ACT for 1979 to 1996.  The water balance for the average, wettest and driest years are shown in the table below.

Water-balance_table.png

When presenting financial statements, a common approach is to use a waterfall chart which shows how the components of a financial balance contribute to an overall result.  Here I’ve used a waterfall chart to show the water balance for Curtin for the driest and wettest year as reported by Mitchell et al., (2003).

Water_balance_driest

Water_balance_wettest

Figure 1: Water balance for Curtin, ACT in (A) and driest and (B) the wettest years as estimated by Mitchell et al., (2003).

Does this approach to visualising a water balance help understanding?  A few things stand out:

  • In the driest year, more water was input from the mains than from rainfall
  • In the driest year, actual evapotranspiration was larger than rainfall and mains inputs.
  • Evapotranspiration and stormwater change with climate, with large variation between the wet and dry years.  Wastewater doesn’t change all that much.
  • Precipitation is highly variable, ranging from 247 mm to 914 mm.

There is a guide to making a waterfall chart in excel here.  The R code to produce the graphs shown in this blog is available as a gist, which draws on this blog.

References

Mitchell, V. G., T. A. McMahon and R. G. Mein (2003) Components of the Total Water Balance of an Urban Catchment. Environmental Management 32(6): 735-746. (link)

Munging rating tables

The Victorian water monitoring site includes rating tables for stream gauges but they are in a format that is not easy to work with.   An example is shown in Figure 1 below.

Rating table

Figure 1: Extract of rating table

The following steps can be used to extract and convert the data into a useable format.

1. Download and save rating table.  Click the button shown to get the rating table as a text file.

Rating_table_download

Figure 2: Save the rating table

2. Re-format the data to create columns of levels and flows.  You’ll need to use your favourite tool for this munging step.  An example using R is available as a gist.

3. Plot and compare with the online version

Rating_table_404216_warehouse

Figure 2: Rating plot (source: data.water.vic.gov.au/monitoring.htm?ppbm=404216&rs&1&rscf_org)

Rating_table_404216_R

Figure 3: Rating plot using data from rating table

4. Save as a csv file for further use.

R code is available here.

Related posts: