Mapping missing data

Missing values are a ubiquitous problem in streamflow data.  A graph showing the occurrence of missing values provides a quick way to see where there are problems.  The figure below maps missing flows for the site Bunyip River at Iona (gauge no. 228213) (lat = 38°07’39.5″S, long = 145°41’09.9″E).  Clearly there were lots of missing data in 2007 and 2008.

Missing flows, Bunyip River at Iona

Map of missing values of flow at the gauge, Bunyip River at Iona (228213)

R code to produce a map of missing values is as follows.

A function to produce a missing map is available as a gist.

###################################################################
#
# Mapping missing data
#
# 
# tony.ladson@moroka.com.au
# 22 Dec 2014
#
####################################################################

library(lubridate)
library(ggplot2)
library(grid)



# Generate some test data

xdate <- seq(from = as.Date('1/1/2000', format = '%d/%m/%Y'),
             to = as.Date('31/12/2010', format = '%d/%m/%Y'),
             by = '1 day')

# generate some flow data, usually this would come from a stream gauging record
flow <- rlnorm(n= length(xdate), meanlog = 1, sdlog=0.5)

# insert some missing data into the flow record
is.na(flow[100:120]) <- TRUE
is.na(flow[2000:2500]) <- TRUE
is.na(flow[3200:3600]) <- TRUE

# make a data frame
my.data <- data.frame(xdate = xdate, flow = flow)

# To use your own data, replace the my.data dataframe with your values.

# day numbers for the start of each month. This will be used in the plot
month.start <- yday(seq(as.Date('2010-01-01'),as.Date('2010-12-31'), by='months'))


missing.factor <- with(my.data, factor(is.na(flow), labels=c('not missing','missing')))

# make the plot

pl <- ggplot(my.data,aes(yday(xdate), factor(year(xdate))))
pl + geom_tile(aes(fill = missing.factor)) + 
  scale_fill_manual(values = c('gray','red'), name='' ) + 
  theme_bw() +
  theme(axis.title.x = element_text(colour="grey20", size=20, vjust=-2)) +
  theme(axis.text.x = element_text(colour="grey20",size=12)) +
  theme(axis.title.y = element_text(colour="grey20",size=20, vjust= 1)) +
  theme(axis.text.y = element_text(colour="grey20",size=12)) +
  theme(legend.title = element_text(colour="grey20",size=12)) +
  theme(plot.margin = unit(c(2.5, 2.5, 2.5, 2.5), "cm")) +
  ylab("Year") +
  scale_x_continuous(name="Month", breaks=month.start, labels=month.abb)



One thought on “Mapping missing data

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s