Spatial Data Download and Utility Functions

A suite of conversion scripts to create internally standardized spatial polygons data frames. Utility scripts use these data sets to return values such as country, state, timezone, watershed, etc. associated with a set of longitude/latitude pairs. (They also make cool maps.)



title: "MazamaSpatialUtils" pagetitle: MazamaSpatialUtils

CRAN_Status_Badge Downloads Build Status

A suite of conversion scripts to create internally standardized spatial
polygons dataframes. Utility scripts use these datasets to return values such
as country, state, timezone, watershed, etc. associated with a set of 
longitude/latitude pairs. (They also make cool maps.)

Background

The MazamaSpatialUtils package was created by MazamaScience to regularize our work with spatial data. The sp, rgdal and maptools packages have made it much easier to work with spatial data found in shapefiles. Many sources of shapefile data are available and can be used to make beautiful maps in R. Unfortunately, the data attached to these datasets, even when fairly complete, often lacks standardized identifiers such as the ISO 3166-1 alpha-2 encodings for countries. Maddeningly, even when these ISO codes are used, the dataframe column in which they are stored does not have a standardized name. It may be called ISO or ISO2 or alpha or COUNTRY or any of a dozen other names we have seen.

While many mapping packages provide ‘natural’ naming of countries, those who wish to develop operational, GIS-like systems need something that is both standardized and language-independent. The ISO 3166-1 alpha-2 encodings have emerged as the defacto standard for this sort of work. In similar fashion, ISO 3166-2 alpha-2 encodings are available for the next administrative level down – state/province/oblast, etc.. For timezones, the defacto standard is the set of Olson timezones used in all UNIX systems.

The main goal of this package is to create an internally standardized set of spatial data that we can use in various projects. Along with three built-in datasets, this package provides ‘convert~’ functions for other spatial datasets that we currently use. These convert functions all follow the same recipe:

  • download spatial data in shapefile format into a standard directory
  • convert shapefile data into a sp SpatialPolygonsDataFrame
  • modify the dataframe in the @data slot so that it adheres to package internal standards

Other datasets can be added following the same procedure.

The ‘package internal standards’ are very simple. Every spatial dataset will have at least one of the following, consistently named columns of data:

  • polygonID – unique identifier associated with each polygon
  • countryCode – ISO 3166-1 alpha-2
  • stateCode – ISO 3166-2 alpha-2
  • timezone – Olson timezone

If another column contains this data, that column must be renamed or duplicated with the internally standardized name. This simple level of consistency makes it possible to generate maps for any data that is ISO encoded. It also makes it possible to create functions that return the country, state or timezone associated with a set of locations.

Installation

This package is designed to be used with R (>= 3.1.0) and RStudio so make sure you have those installed first.

Users can use the devtools package to install the latest version of the package which may have new features that are not yet available on CRAN:

devtools::install_github('mazamascience/MazamaSpatialUtils', build_vignettes=TRUE)

Spatial Datasets

Package Datasets

The package comes with the following simplified spatial spatial datasets:

 * 276K	data/SimpleCountries.RData
 * 2.1M	data/SimpleCountriesEEZ.RData
 * 1.1M	data/SimpleTimezones.RData

These datasets allow you to work with low-resolution country outlines and timezones.

Core Datasets

Additional datasets are available at http://mazamascience.com/RData/Spatial/ and can be loaded with the following commands:

# Create a location where large spatial datasets will be stored
dir.create('~/Data/Spatial', recursive = TRUE)

# Tell the package about this location
setSpatialDataDir('~/Data/Spatial')

# Install core spatial data
installSpatialData()

Datasets included in the core set include:

 * 2.1M EEZCountries.RData
 *  15M NaturalEarthAdm1.RData
 *  61M OSMTimezones.RData
 * 3.0M	OSMTimezones_05.RData
 * 3.6M TMWorldBorders.RData
 *  48MTerrestrialEcoregions.RData
 * 3.5M TerrestrialEcoregions_05.RData
 * 7.5M USCensus115thCongress.RData
 *  17M USCensusCounties.RData
 * 4.6M USCensusStates.RData
 * 1.2M USIndianLands.RData
 *  17M WorldTimezones.RData

Further details about each dataset are provided in the associated convert~() function. Datasets appearing with, e.g., _05 are simplified datasets whose polygons retain only 5% of the vertices of the original .

Additional Datasets

Mazama Science regularly generates new datasets that adhere to package standards. These can be download manually from http://mazamascience.com/RData/Spatial/. As of Jan 10, 2019, the full list of available datasets includes:

 * 24K	CA_AirBasins_01.RData
 * 44K	CA_AirBasins_02.RData
 * 100K	CA_AirBasins_05.RData
 * 2.1M	CA_AirBasins.RData
 * 2.2M	EEZCountries.RData
 * 404K	GACC_05.RData
 * 7.0M	GACC.RData
 * 15M	NaturalEarthAdm1.RData
 * 3.1M	OSMTimezones_05.RData
 * 62M	OSMTimezones.RData
 * 3.6M	TerrestrialEcoregions_05.RData
 * 49M	TerrestrialEcoregions.RData
 * 3.7M	TMWorldBorders.RData
 * 7.6M	USCensus115thCongress.RData
 * 564K	USCensusCBSA_01.RData
 * 944K	USCensusCBSA_02.RData
 * 2.0M	USCensusCBSA_05.RData
 * 34M	USCensusCBSA.RData
 * 2.3M	USCensusCounties.RData
 * 3.5M	USCensusStates.RData
 * 1.2M	USIndianLands.RData
 * 769M	WBDHU10.RData
 * 1.5G	WBDHU12.RData
 * 424K	WBDHU2_01.RData
 * 840K	WBDHU2_02.RData
 * 38M	WBDHU2.RData
 * 1.1M	WBDHU4_01.RData
 * 2.2M	WBDHU4_02.RData
 * 108M	WBDHU4.RData
 * 1.4M	WBDHU6_01.RData
 * 2.8M	WBDHU6_02.RData
 * 137M	WBDHU6.RData
 * 295M	WBDHU8.RData
 * 18M	WorldTimezones.RData

Examples

Vignettes

The package vignette 'Introduction to MazamaSpatialUtils' has numerous examples.

Demos

There are three demos associated with the package:

demo(package = 'MazamaSpatialUtils')

Shiny App

There is also an exampe R Shiny app which uses the WBDHU# datasets combines two large datasets:

  • location data from the National Bridge Inventory
  • shapefiles from the Watershed Boundary Dataset

The app allows you to aggregate point location data by watershed to create summary values associated with each watershed. It also demonstrates the need to enable caching in a shiny app when plots take a long time to generate.

library(MazamaSpatialUtils)
setSpatialDataDir('~/Data/Spatial')
runExample()

Mapshaper

Instructions for installing the javascript mapshaper utility and using it to simplify large shapefiles are found in the localMapshaper/ directory.


This project is supported by Mazama Science.

News

MazamaSpatialUtils 0.6.1

  • made simplify() function example donotrun so as to avoid a CRAN testing failure on fedora only -- Ugh.

MazamaSpatialUtils 0.6.0

  • changes associated with minor version bump

MazamaSpatialUtils 0.5.10

  • refactored shiny app from localShiny and added it to package in inst
  • added runExample() to run shiny examples
  • updated README.md

MazamaSpatialutils 0.5.9

  • New convert function for public health districts
  • New convert function for GACCs
  • more testing

MazamaSpatialUtils 0.5.8

  • New convert functions for weather zones
  • Argument consistency with getHUC()
  • added dissolve() wrapper for rmapshaper::ms_dissolve()

MazamaSpatialUtils 0.5.7

  • documentation with pkgdown
  • various cleanup: spell check, reformatted NEWS.md, etc.
  • added simplify() wrapper for rmapshaper::ms_simplify()

MazamaSpatialUtils 0.5.6

  • added CONUS and US_52 vectors of US state codes as package data
  • added US_stateCodes dataframe as package data

MazamaSpatialUtils 0.5.5

  • added convertStateLegislativeDistricts() function to download and convert US state level legislative districts on demand
  • now using countrycode package for all code/name conversions
  • ebolamap.Rmd converted to vignette, moved to vignettes/ and localNotebooks/ deleted
  • usages of cat() replaced with message(), warning(), or error()

MazamaSpatialUtils 0.5.4

  • corrected GADM URL in documentation for convertGADM()

MazamaSpatialUtils 0.5.3

  • added convertUSCensusCBSA() to convert Metropolitan-/Micropolitan Statistical Areas
  • removed verbose argument from getStateCode() function signature

MazamaSpatialUtils 0.5.2

  • fixed use of allStateCodes in WBD~ datasets
  • converted [email protected] data from factor to character
  • updated docker/Dockerfile
  • added ebolaMap vignette

MazamaSpatialUtils 0.5.1

  • tweaks to satisfy CRAN

MazamaSpatialUtils 0.5.0

  • package now Depends on sp package so that sp plotting is used by default
  • package includes higher resolution SimpleCountries and SimpleTimezones datasets
  • internal standard now requires unique polygonID column for every dataset
  • organizePolygons() now uniformly uses polygonID for rownames and [email protected] so you can say: plot(SimpleCountries['IT',])
  • fixed bug in processing of USCensusCounties dataset
  • changed codeToCode() to two functions: iso2ToIso3() and iso3ToIso2()
  • package now includes SimpleCountriesEEZ dataset which is used as the default dataset for getCountry(). SimpleCountriesEEZ includes a 200-mile offshore buffer for more efficient and accurate spatial searches
  • new datasets include: TerrestrialEcoregions, USCensus115thCongress, USCensusStates, USCensusIndianLands

MazamaSpatialUtils 0.4.9

  • new .tar.gz file available containing spatial datasets
  • new installSpatialData() installs all required datasets
  • loadSpatialData() no loads one or more datasets based on a pattern
  • removed initializeSpatialData()
  • convertGADM() updated to GADM version 2.8 (handles .rds files)
  • updated localVignettes/ebolaMap.Rmd
  • new docker/ directory for building docker images to run MazamaSpatialUtils
  • new app/ directory demonstrates dockerized web-service based on MazamaSpatialUtils

MazamaSpatialUtils 0.4.8

  • locations that do not intersect any polygon now return NA rather than generating warnings
  • various minor bug fixes.

MazamaSpatialUtils 0.4.5

  • convertHMSSmoke() now handles shapefiles with new Density information

MazamaSpatialUtils 0.4.4

  • Fixed bug/typo in findTimezones demo.
  • Fixed bugs/typos in convertWBDHUC() and convertHMSSmoke().

MazamaSpatialUtils 0.4.3

  • New convertHMSSmoke() function for smoke data from the NOAA Hazard Mapping Service.
  • Shapefiles with no projection information are assigned "+proj=longlat +ellps=GRS80 +datum=NAD83 +no_defs".
  • Added dependency on lubridate package.

MazamaSpatialUtils 0.4.2

  • Added encoding argument to converLayer().
  • Modified convertUSCensusCounties() to use encoding='latin1'.

MazamaSpatialUtils 0.4.1

  • Added useBuffering arguent to getState(), getCountry() and getTimezone().

MazamaSpatialUtils 0.3.2

  • getSpatialData() no longer fails on invliad/missing locations, now returns dataframe rows with all NA.

MazamaSpatialUtils 0.3.1 -- addition of buffered search and WorldEEZ polygons

  • Updated included datasets to use "+proj=longlat +ellps=GRS80 +datum=NAD83 +no_defs".
  • Addition of buffered search so that locations can find nearby polygons.
  • Addition of convertWorldEEZ() function.

MazamaSpatialUtils 0.2.4

  • Updated default projection from "+proj=longlat" to "+proj=longlat +ellps=GRS80 +datum=NAD83 +no_defs" to support libproj >= 4.9.1

MazamaSpatialUtils 0.2.3

  • Removed unneeded test that failed with sp version 1.1-0.

MazamaSpatialUtils 0.2.2 -- minor tweaks to 0.2.1

  • User specification of SpatialDataDir is now required.
  • Minor documentation improvements.

MazamaSpatialUtils 0.2.1 -- addition of GADM and USGS HUC8

  • Convert functions for GADM administrative boundaries and and USGS watershed datasets.
  • Addition of code-name, name-code and code-code conversion utilities.
  • Addition of organizePolygons() function.

MazamaSpatialUtils 0.1 -- initial release

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("MazamaSpatialUtils")

0.6.1 by Jonathan Callahan, 2 months ago


https://github.com/MazamaScience/MazamaSpatialUtils


Report a bug at https://github.com/MazamaScience/MazamaSpatialUtils/issues


Browse source code at https://github.com/cran/MazamaSpatialUtils


Authors: Jonathan Callahan [aut, cre] , Tom Bergamaschi [aut] , Ruby Fore [aut] , Will Leahy [aut] , Helen Miller [aut] , Henry Nguyen [aut] , Robin Winstanley [aut] , Alice Yang [aut]


Documentation:   PDF Manual  


GPL-2 license


Imports countrycode, dplyr, geojsonio, lubridate, rgdal, rgeos, rlang, rmapshaper, rvest, shiny, stringr, utils, xml2

Depends on sp

Suggests knitr, maps, markdown, readr, rmarkdown, testthat


Depended on by PWFSLSmoke.


See at CRAN