Exploration and Graphics for RivEr Trends

Statistics and graphics for streamflow history, water quality trends, and the statistical modeling algorithm: Weighted Regressions on Time, Discharge, and Season (WRTDS). The modeling method is introduced and discussed in Hirsch et al. (2010) , and expanded in Hirsch and De Cicco (2015) .


EGRET

travis CoverageStatus status CRANversion Research softwareimpact

Exploration and Graphics for RivEr Trends (EGRET): An R-package for the analysis of long-term changes in water quality and streamflow, including the water-quality method Weighted Regressions on Time, Discharge, and Season (WRTDS).

Please visit the wiki for more information: EGRET Wiki

The link for the official USGS publication user guide is here:

https://pubs.usgs.gov/tm/04/a10/

Package Installation

To install the EGRET package, you must be using R 3.0 or greater and run the following command:

install.packages("EGRET")

To get inter-CRAN release updates, use the command:

install.packages("EGRET",repos="https://owi.usgs.gov/R")

To get cutting-edge changes, install from GitHub using the devtools packages:

library(devtools)
install_github("USGS-R/EGRET")

Background:

Evaluating long-term changes in river conditions (water quality and discharge) is an important use of hydrologic data. To carry out such evaluations, the hydrologist needs tools to facilitate several key steps in the process: acquiring the data records from a variety of sources, structuring it in ways that facilitate the analysis, routines that will process the data to extract information about changes that may be happening, and graphical techniques that can display findings about change. The R package EGRET (Exploration and Graphics for RivEr Trends) was developed for carrying out each of these steps in an integrated manner. It is designed to accept easily data from three sources: U.S. Geological Survey hydrologic data, Water Quality Portal Data (currently including U.S. Environmental Protection Agency (EPA) STORET data, and USDA STEWARDS data), and user-supplied flat files. The EGRET package has components oriented towards the description of long-term changes in streamflow statistics (high flow, average flow, and low flow) as well as changes in water quality. For the water-quality analysis, it uses Weighted Regressions on Time, Discharge and Season (WRTDS) to describe long-term trends in both concentration and flux. EGRET also creates a wide range of graphical presentations of the water-quality data and of the WRTDS results. The following report serves as a user guide, providing detailed guidance on installation and use of the software, documentation of the analysis methods used, as well as guidance on some of the kinds of questions and approaches that the software can facilitate.

EGRET includes statistics and graphics for streamflow history, water quality trends, and the statistical modeling algorithm Weighted Regressions on Time, Discharge, and Season (WRTDS). Please see the official EGRET User Guide for more information on the EGRET package:

https://doi.org/10.3133/tm4A10 The best ways to learn about the WRTDS approach is to read the User Guide and two journal articles. These articles are available, for free, from the journals in which they were published. The first relates to nitrate and total phosphorus data for 9 rivers draining to Chesapeake Bay. The URL is:

https://onlinelibrary.wiley.com/doi/full/10.1111/j.1752-1688.2010.00482.x.

The second is an application to nitrate data for 8 monitoring sites on the Mississippi River or its major tributaries. The URL is:

<http://pubs.acs.org/doi/abs/10.1021/es201221s

For a thorough discussion of the generalized flow normalization method implemented in the EGRET enhancements, see the paper: "Tracking changes in nutrient delivery to western Lake Erie: Approaches to compensate for variability and trends in streamflow":

https://www.sciencedirect.com/science/article/pii/S0380133018302235

Sample Workflow

WRTDS on the Choptank River at Greensboro MD, for Nitrate:

library(EGRET)
# Gather discharge data:
siteID <- "01491000" #Choptank River at Greensboro, MD
startDate <- "" #Gets earliest date
endDate <- "2011-09-30"
# Gather sample data:
parameter_cd<-"00631" #5 digit USGS code
Sample <- readNWISSample(siteID,parameter_cd,startDate,endDate)
#Gets earliest date from Sample record:
#This is just one of many ways to assure the Daily record
#spans the Sample record
startDate <- min(as.character(Sample$Date)) 
# Gather discharge data:
Daily <- readNWISDaily(siteID,"00060",startDate,endDate)
# Gather site and parameter information:
 
# Here user must input some values for
# the default (interactive=TRUE)
INFO<- readNWISInfo(siteID,parameter_cd)
INFO$shortName <- "Choptank River at Greensboro, MD"
 
# Merge discharge with sample data:
eList <- mergeReport(INFO, Daily, Sample)
############################
library(EGRET)
# Sample data included in package:
eList <- Choptank_eList
 
############################
# Check sample data:
boxConcMonth(eList)

boxQTwice(eList)

plotConcTime(eList)

plotConcQ(eList)

multiPlotDataOverview(eList)
############################
 
############################
# Run WRTDS model:
eList <- modelEstimation(eList)

#> 
#>  first step running estCrossVal may take about 1 minute
#>  estCrossVal % complete:
#> 0    1   2   3   4   5   6   7   8   9   10  
#> 11   12  13  14  15  16  17  18  19  20  
#> 21   22  23  24  25  26  27  28  29  30  
#> 31   32  33  34  35  36  37  38  39  40  
#> 41   42  43  44  45  46  47  48  49  50  
#> 51   52  53  54  55  56  57  58  59  60  
#> 61   62  63  64  65  66  67  68  69  70  
#> 71   72  73  74  75  76  77  78  79  80  
#> 81   82  83  84  85  86  87  88  89  90  
#> 91   92  93  94  95  96  97  98  99  
#> Next step running  estSurfaces with survival regression:
#> Survival regression (% complete):
#> 0    1   2   3   4   5   6   7   8   9   10  
#> 11   12  13  14  15  16  17  18  19  20  
#> 21   22  23  24  25  26  27  28  29  30  
#> 31   32  33  34  35  36  37  38  39  40  
#> 41   42  43  44  45  46  47  48  49  50  
#> 51   52  53  54  55  56  57  58  59  60  
#> 61   62  63  64  65  66  67  68  69  70  
#> 71   72  73  74  75  76  77  78  79  80  
#> 81   82  83  84  85  86  87  88  89  90  
#> 91   92  93  94  95  96  97  98  99  
#> Survival regression: Done
############################

############################
#Check model results:

#eList:
plotConcTimeDaily(eList)

plotFluxTimeDaily(eList)

plotConcPred(eList)

plotFluxPred(eList)

plotResidPred(eList)

plotResidQ(eList)

plotResidTime(eList)

boxResidMonth(eList)

boxConcThree(eList)

 
#Require Daily + INFO:
plotConcHist(eList)

plotFluxHist(eList)

# Multi-line plots:
date1 <- "2000-09-01"
date2 <- "2005-09-01"
date3 <- "2009-09-01"
qBottom<-100
qTop<-5000
plotConcQSmooth(eList, date1, date2, date3, qBottom, qTop, 
                   concMax=2,qUnit=1)

q1 <- 10
q2 <- 25
q3 <- 75
centerDate <- "07-01"
yearEnd <- 2009
yearStart <- 2000
plotConcTimeSmooth(eList, q1, q2, q3, centerDate, yearStart, yearEnd)

 
# Multi-plots:
fluxBiasMulti(eList)

#Contour plots:
clevel<-seq(0,2,0.5)
maxDiff<-0.8
yearStart <- 2000
yearEnd <- 2010
 
plotContours(eList, yearStart,yearEnd,qBottom,qTop, 
             contourLevels = clevel,qUnit=1)

plotDiffContours(eList, yearStart,yearEnd,
                 qBottom,qTop,maxDiff,qUnit=1)

Sample workflow for a flowHistory application for the entire record

library(EGRET)
 
# Flow history analysis
############################
# Gather discharge data:
siteID <- "01491000" #Choptank River at Greensboro, MD
startDate <- "" # Get earliest date
endDate <- "" # Get latest date
Daily <- readNWISDaily(siteID,"00060",startDate,endDate)
#> There are 25856 data points, and 25856 days.
# Gather site and parameter information:
# Here user must input some values for
# the default (interactive=TRUE)
INFO<- readNWISInfo(siteID,"00060")
#> Your site for streamflow data is:
#>  01491000 .
#> Your site name is CHOPTANK RIVER NEAR GREENSBORO, MD 
#> but you can modify this to a short name in a style you prefer. 
#> This name will be used to label graphs and tables. 
#> If you want the program to use the name given above, just do a carriage return,
#> otherwise enter the preferred short name(no quotes):
#> 
#> The latitude and longitude of the site are:  38.99719 ,  -75.78581 (degrees north and west).
#> 
#> The drainage area at this site is  113 square miles
#>  which is being stored as 292.6687 square kilometers.
#> 
#> It is helpful to set up a station abbreviation when doing multi-site studies,
#> enter a unique id (three or four characters should work). It is case sensitive.  
#> Even if you don't feel you need an abbreviation for your site you need to enter something(no quotes):
#> 
#> Your water quality data are for parameter number:
#> 00060 
#> which has the name:' Discharge, cubic feet per second '.
#> Typically you will want a shorter name to be used in graphs and tables.
#> The suggested short name is:' Stream flow, mean. daily '.
#> If you would like to change the short name, enter it here, 
#> otherwise just hit enter (no quotes):
#> The units for the water quality data are:  ft3/s .
#> It is helpful to set up a constiuent abbreviation, enter a unique id 
#> three or four characters should work something like tn or tp or NO3).
#> Even if you don't feel you need an abbreviation you need to enter something (no quotes):
#> 
#> Required concentration units are mg/l. 
#> The INFO dataframe indicates: ft3/s 
#> Flux calculations will be wrong if units are not consistent.
INFO$shortName <- "Choptank River at Greensboro, MD"
eList <- as.egret(INFO, Daily, NA, NA)
############################
 
############################
# Check flow history data:
plotFlowSingle(eList, istat=7,qUnit="thousandCfs")

plotSDLogQ(eList)

plotQTimeDaily(eList, qLower=1,qUnit=3)

plotFour(eList, qUnit=3)

plotFourStats(eList, qUnit=3)

############################

Model Archive

When using the WRTDS model, it is important to be able to reproduce the results in the future. The following version of R and package dependencies were used most recently to pass the embedded tests within this package. There is no guarantee of reproducible results using future versions of R or updated versions of package dependencies; however, we will make diligent efforts to test and update future modeling environments.

devtools::session_info()
#> Session info -------------------------------------------------------------
#>  setting  value                       
#>  version  R version 3.5.1 (2018-07-02)
#>  system   x86_64, mingw32             
#>  ui       RTerm                       
#>  language (EN)                        
#>  collate  English_United States.1252  
#>  tz       America/Chicago             
#>  date     2018-10-16
#> Packages -----------------------------------------------------------------
#>  package       * version    date       source                          
#>  assertthat      0.2.0      2017-04-11 CRAN (R 3.5.1)                  
#>  backports       1.1.2      2017-12-13 CRAN (R 3.5.0)                  
#>  base          * 3.5.1      2018-07-02 local                           
#>  bindr           0.1.1      2018-03-13 CRAN (R 3.5.1)                  
#>  bindrcpp        0.2.2      2018-03-29 CRAN (R 3.5.1)                  
#>  compiler        3.5.1      2018-07-02 local                           
#>  crayon          1.3.4      2017-09-16 CRAN (R 3.5.1)                  
#>  curl            3.2        2018-03-28 CRAN (R 3.5.1)                  
#>  dataRetrieval   2.7.4      2018-05-09 local                           
#>  datasets      * 3.5.1      2018-07-02 local                           
#>  devtools        1.13.6     2018-06-27 CRAN (R 3.5.1)                  
#>  digest          0.6.18     2018-10-10 CRAN (R 3.5.1)                  
#>  dotCall64       1.0-0      2018-07-30 CRAN (R 3.5.1)                  
#>  dplyr           0.7.6      2018-06-29 CRAN (R 3.5.1)                  
#>  EGRET         * 3.0.1      2018-10-16 local                           
#>  evaluate        0.12       2018-10-09 CRAN (R 3.5.1)                  
#>  fields          9.6        2018-01-29 CRAN (R 3.5.1)                  
#>  glue            1.3.0      2018-07-17 CRAN (R 3.5.1)                  
#>  graphics      * 3.5.1      2018-07-02 local                           
#>  grDevices     * 3.5.1      2018-07-02 local                           
#>  grid            3.5.1      2018-07-02 local                           
#>  hms             0.4.2      2018-03-10 CRAN (R 3.5.1)                  
#>  htmltools       0.3.6      2017-04-28 CRAN (R 3.5.1)                  
#>  httr            1.3.1      2017-08-20 CRAN (R 3.5.1)                  
#>  jsonlite        1.5        2017-06-01 CRAN (R 3.5.1)                  
#>  knitr           1.20       2018-02-20 CRAN (R 3.5.1)                  
#>  lattice         0.20-35    2017-03-25 CRAN (R 3.5.1)                  
#>  lubridate       1.7.4      2018-04-11 CRAN (R 3.5.1)                  
#>  magrittr        1.5        2014-11-22 CRAN (R 3.5.1)                  
#>  maps            3.3.0      2018-04-03 CRAN (R 3.5.1)                  
#>  Matrix          1.2-14     2018-04-13 CRAN (R 3.5.1)                  
#>  memoise         1.1.0      2017-04-21 CRAN (R 3.5.1)                  
#>  methods       * 3.5.1      2018-07-02 local                           
#>  pillar          1.3.0      2018-07-14 CRAN (R 3.5.1)                  
#>  pkgconfig       2.0.2      2018-08-16 CRAN (R 3.5.1)                  
#>  plyr            1.8.4      2016-06-08 CRAN (R 3.5.1)                  
#>  purrr           0.2.5      2018-05-29 CRAN (R 3.5.1)                  
#>  R6              2.3.0      2018-10-04 CRAN (R 3.5.1)                  
#>  Rcpp            0.12.19    2018-10-01 CRAN (R 3.5.1)                  
#>  readr           1.1.1      2017-05-16 CRAN (R 3.5.1)                  
#>  reshape2        1.4.3      2017-12-11 CRAN (R 3.5.1)                  
#>  rlang           0.2.2.9001 2018-09-25 Github (tidyverse/[email protected])
#>  rmarkdown       1.10       2018-06-11 CRAN (R 3.5.1)                  
#>  rprojroot       1.3-2      2018-01-03 CRAN (R 3.5.1)                  
#>  spam            2.2-0      2018-06-19 CRAN (R 3.5.1)                  
#>  splines         3.5.1      2018-07-02 local                           
#>  stats         * 3.5.1      2018-07-02 local                           
#>  stringi         1.1.7      2018-03-12 CRAN (R 3.5.0)                  
#>  stringr         1.3.1      2018-05-10 CRAN (R 3.5.1)                  
#>  survival        2.42-3     2018-04-16 CRAN (R 3.5.1)                  
#>  tibble          1.4.2      2018-01-22 CRAN (R 3.5.1)                  
#>  tidyselect      0.2.5      2018-10-11 CRAN (R 3.5.1)                  
#>  tools           3.5.1      2018-07-02 local                           
#>  utils         * 3.5.1      2018-07-02 local                           
#>  withr           2.1.2      2018-03-15 CRAN (R 3.5.1)                  
#>  xml2            1.2.0      2018-01-24 CRAN (R 3.5.1)                  
#>  yaml            2.2.0      2018-07-25 CRAN (R 3.5.1)

Reporting bugs

Please consider reporting bugs and asking questions on the Issues page: https://github.com/USGS-R/EGRET/issues

Follow @USGS_R on Twitter for updates on USGS R packages:

TwitterFollow

Subscribe

Please email questions, comments, and feedback to: [email protected]

Additionally, to subscribe to an email list concerning updates to these R packages, please send a request to [email protected].

Code of Conduct

We want to encourage a warm, welcoming, and safe environment for contributing to this project. See the code of conduct for more information.

Package Support

The Water Mission Area of the USGS has supported the development and maintenance of the EGRET R-package. Further maintenance is expected to be stable through September 2019. Resources are available primarily for maintenance and responding to user questions. Priorities on the development of new features are determined by the EGRET development team.

USGS

Sunset date

Funding for EGRET currently expires summer 2019. Expectations are that maintenance and customer service will continue to be supported past that date.

How to cite EGRET:

citation(package = "EGRET")
#> 
#> To cite EGRET in publications, please use:
#> 
#>   Hirsch, R.M., and De Cicco, L.A., 2015, User guide to
#>   Exploration and Graphics for RivEr Trends (EGRET) and
#>   dataRetrieval: R packages for hydrologic data (version 2.0,
#>   February 2015): U.S. Geological Survey Techniques and Methods
#>   book 4, chap. A10, 93 p., doi:10.3133/tm4A10
#> 
#> A BibTeX entry for LaTeX users is
#> 
#>   @InBook{,
#>     author = {Robert M. Hirsch and Laura A. {De Cicco}},
#>     title = {User guide to Exploration and Graphics for RivEr Trends (EGRET) and dataRetrieval: R packages for hydrologic data},
#>     publisher = {U.S. Geological Survey},
#>     address = {Reston, VA},
#>     booktitle = {Techniques and Methods},
#>     institution = {U.S. Geological Survey},
#>     year = {2015},
#>     chapter = {A10},
#>     url = {https://pubs.usgs.gov/tm/04/a10/},
#>   }

Disclaimer

This software has been approved for release by the U.S. Geological Survey (USGS). Although the software has been subjected to rigorous review, the USGS reserves the right to update the software as needed pursuant to further analysis and review. No warranty, expressed or implied, is made by the USGS or the U.S. Government as to the functionality of the software and related material nor shall the fact of release constitute any such warranty. Furthermore, the software is released on condition that neither the USGS nor the U.S. Government shall be held liable for any damages resulting from its authorized or unauthorized use.

News

EGRET 3.0.0

  • Added enhancements for generalized flow normalization
  • Functions include runPairs, runSeries, and runGroups

EGRET 2.6.1

  • Added foreach dependency which allows parallelizing WRTDS.
  • Added the function flexFN for flexible flow normalization.

EGRET 2.6.0

  • start/end year argument names unified to yearStart and yearEnd (previously included startYear and endYear)

EGRET 2.5.3

  • Argument for randomized censored values names randomCensored

EGRET 2.5.0

  • Added an argument to the concentration and residual plotting functions to randomize the censored values for the plots.
  • Minor bug fixes

EGRET 2.3.1

  • Removed unnecessary disclaimer.

EGRET 2.2.0

  • Minor bug fixes
  • Added kg/year option for flux

EGRET 2.1.4

  • Updated CITATION file

EGRET 2.1.1

  • Exposed sysdata.rda constants
  • Fixed some inconsistancies in readUserInfo function
  • Added prettyDate argument to plotFluxTimeDaily, plotConcTimeDaily, and plotQTimeDaily. This overrides the 'pretty' axis so that the plot starts and ends exactly at the startYear/endYear arguments
  • Added testthat unit tests
  • Added kg/year flux constant

EGRET 2.1.0

  • Updated processQWData to be more inclusive of Water Quality Portal data
  • Moved some lookup files (month labels, flux labels) to sysdata.rda

EGRET 2.0.1

  • Dec. 2, 2014
  • Minor update to account for changes to dataRetrieval
  • Help file updates.

EGRET 2.0.0

  • November 13, 2014
  • Changed workflow so that most functions accept a single argument: eList, which is a named list with the Daily, Sample, and INFO dataframes (and surface matrix if available)

EGRET 1.3.0

  • July 31, 2014
  • Added alternative runSurvReg option, which deals with edge effects of the model.The modified method tends to reduce curvature near the start and end of record.

EGRET 1.2.5

  • March 10, 2014
  • Added period of analysis control to all graphs and tables so they can present outputs by season as well as the full year. The only graph or table functions without this capability are now: plotContours, plotDiffContours, plotConcQSmooth, and plotConcTimeSmooth.
  • Changed the way that period of analysis is now specified: by using the function setPA. It sets paLong and paStart and applies to all subsequent functions until setPA is used again.
  • For plots where time is on the x-axis and the length is 4-years or less, the axis labels now give more date information.
  • Added more flexibility in character sizes, colors, line weights and other features and graphics.
  • Changed default color palette for plotContours and plotDiffContours.
  • Updated vignette.
  • Added setupYears call within most graphic and table functions to generally eliminate the need to create an AnnualResults data frame.
  • Continued to improve documentation.

EGRET 1.2.4

  • September 25, 2013
  • Plotting functions updated to allow users better control.
  • Fixed small leap year bug.
  • Added dataframe returns to table functions.
  • plotLogXXX functions were removed, logScale argument was added to linear plots

EGRET 1.2.3

  • New estDailyFromSurfaces function utilizing "fields" package, written by Jeffrey Chanat. Greatly increases the speed of computations of the daily results. Two additional R packages are needed: fields and spam

  • Improved documentation, especially example functions.

  • runSurvReg modified to include the option of a very wide seasonal window (windowS), which has the effect of eliminating the influence of the seasonal weights.

  • Progress indicators modified to be more informative.

  • Fixed a calculation bug in function plotLogFluxPred.

EGRET 1.2.1

  • Adds four new functions to the flowHistory capability. The four new flow History functions are: plot15 (makes an array of 15 plots for a given site - a matrix of 3 flow statistics (7-day min, mean, and 1-day max) versus 5 periods of analysis (annual, fall, winter, spring, and summer), plotSDLogQ (makes a plot of the running standard deviation of the Log Discharge versus time), plotFour (makes a set of four graphs: 1-day max, mean, 7-day min, and running standard deviation of the logs), and plotQTimeDaily (a way to plot discharge versus time, setup particularly to show flows above some threshold discharge).

  • Bug fix in WRTDS made no changes to calculations, it just prevents a crash in a situation where the user has shortened the length of the daily record.

EGRET 1.1.3

  • A few small bug fixes and cosmetic changes in some graphics, improved saveResults function, and better formatting of help pages for the functions.

EGRET 1.0.0

  • Initial release.

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("EGRET")

3.0.2 by Laura DeCicco, 2 months ago


https://pubs.usgs.gov/tm/04/a10/, https://github.com/USGS-R/EGRET/wiki


Report a bug at https://github.com/USGS-R/EGRET/issues


Browse source code at https://github.com/cran/EGRET


Authors: Robert Hirsch [aut] , Laura DeCicco [aut, cre] , David Watkins [ctb] , Lindsay Carr [ctb] , Jennifer Murphy [ctb]


Documentation:   PDF Manual  


Task views: Hydrological Data and Modeling


CC0 license


Imports dataRetrieval, survival, fields, methods, utils, graphics, stats, grDevices, truncnorm, foreach

Suggests EGRETci, xtable, knitr, rmarkdown, extrafont, testthat, rkt, doParallel, parallel


Imported by EGRETci.


See at CRAN