Allows automating the creation of time series of rasters derived from MODIS Satellite Land Products data. It performs several typical preprocessing steps such as download, mosaicking, reprojection and resize of data acquired on a specified time period. All processing parameters can be set using a user-friendly GUI. Users can select which layers of the original MODIS HDF files they want to process, which additional Quality Indicators should be extracted from aggregated MODIS Quality Assurance layers and, in the case of Surface Reflectance products , which Spectral Indexes should be computed from the original reflectance bands. For each output layer, outputs are saved as single-band raster files corresponding to each available acquisition date. Virtual files allowing access to the entire time series as a single file are also created. Command-line execution exploiting a previously saved processing options file is also possible, allowing to automatically update time series related to a MODIS product whenever a new image is available.
MODIStsp is a "R" package devoted to automatizing the creation of time series of rasters derived from MODIS Land Products data. MODIStsp allows to perform several preprocessing steps (e.g., download, mosaicing, reprojection and resize) on MODIS data available within a given time period. Users have the ability to select which specific layers of the original MODIS HDF files they want to process. They also can select which additional Quality Indicators should be extracted from the aggregated MODIS Quality Assurance layers and, in the case of Surface Reflectance products, which Spectral Indexes should be computed from the original reflectance bands. For each output layer, outputs are saved as single-band raster files corresponding to each available acquisition date. Virtual files allowing access to the entire time series as a single file can be also created. All processing parameters can be easily selected with a user-friendly GUI, although non-interactive execution exploiting a previously created Options File is possible. Stand-alone execution outside an "R" environment is also possible, allowing to use scheduled execution of MODIStsp to automatically update time series related to a MODIS product and extent whenever a new image is available.
MODIStsp is developed and maintained by L.Busetto and L.Ranghetti, from the Institute of Remote Sensing of Environment - National Research Council - Italy (CNR-IREA)
11/08/2017 - MODIStp 1.3.3 was released today. It provides improvements in processing speed, as well as the usual bug fixes (thanks to all the users that signalled problems !). Check the Release Notes for further details !
25/07/2017 - As of today, most of the content related to MODIStsp has been moved to our new website at lbusett.github.io/MODIStsp, which provides a much better user interface and ease of access to MODIStsp-related information. From now on, please consult the new website for detailed and updated information on the package.
Also our previous FAQ page on github containing info for solving common installation, downloading and processing problems and issues was discontinued and migrated at lbusett.github.io/MODIStsp/articles/faq.html.
MODIStsp requires R v >= 3.2.1 and GDAL (Geospatial Data Abstraction Library) v >= 1.11.1 with support for HDF4 raster format to be installed in your system. Brief instructions for installing R and GDAL can be found HERE.
You can install the stable version of
MODIStsp from CRAN:
, or the development version (containing the latest improvements and bug fixes) from github:
Note that if the
GTK+ library is not already installed on your system, installation may fail. In that case, please install and load the
gWidgetsRGtk2 library beforehand:
gWidgetsRGtk2, an error window will probably appear. This signals that library "GTK+" is not yet installed on your system or is not on your PATH. To install itpress "OK". A new window dialog window will appear, asking if you want to install "GTK+". Select "Install GTK" and then "OK" . Windows will download and install the GTK+ library. When it finishes, the RSession should be restarted and you should be ready to go !
In case RStudio doesn't automatically restart or continuously asks to install GTK+ again, kill it form "Task Manager" (or restart the R session from RStudio "Session" menu), reload RStudio and the try to reload
gWidgetsRGtk2. If it loads correctly, you should be ready to go.
If it still fails, try downloading the GTK+ bundle from:
http://ftp.gnome.org/pub/gnome/binaries/win64/gtk+/2.22/gtk+-bundle_2.22.1-20101229_win64.zip (OR http://ftp.gnome.org/pub/gnome/binaries/win32/gtk+/2.22/gtk+-bundle_2.22.1-20101227_win32.zip if on Win32)
, unzip the archive on a folder of your choice (e.g.,
C:\\Program Files\\GTK+), then add the path to its "bin" subfolder (e.g.,
C:\\Program Files\\GTK+\\bin\\ to your system PATH environment variable.
Restart your system and try loading again
gWidgetsRGtk2: if it loads ok, you should be ready to install
MODIStsp on Linux, you have to first install the following required dependencies:
GLib>= 2.8.0 (required by package
Curl(required by package
PROJ.4>= 4.4.9 (required by package
On Debian and Ubuntu-based systems, to install those packages open a terminal and type:
sudo apt-get install r-cran-cairodevice r-cran-rgtk2 libcairo2-dev libatk1.0-dev libpango1.0-devlibgtk2.0-dev libglib2.0-dev libcurl4-openssl-dev libgdal-dev libproj-dev
On rpm-base systems, to install packages open a terminal and type:
sudo yum install libcairo2-devel libatk1.0-devel libpango1.0-devel gtk2 gtk2-develglib2-devel libcurl-devel gdal-devel proj proj-devel proj-epsg proj-nad
Then, you can install the stable version of MODIStsp from CRAN:
, or the development version (containing the latest improvements and bug fixes) from github;
NOTE: The following installation notes were kindly provided by a MODIStsp user and should be valid for MODIStsp installation on R 3.4.0 and above with Mac OSX Sierra. Since we are not working on Mac we were not able yet to check this, but we hope it may be useful at least to put you in the right direction if you have installation problems)
Installing MODIStsp requires many dependencies:
For installation on MAC OSX sierra, there are three main issues: - As outlined here in the comment by tobybot11 (https://gist.github.com/sebkopf/9405675), Rgtk requires the x11 libraries/headers (gdk/gdkx.h specifically) and doesn't work with the quartz libraries/headers which now are the default for GTK - When installing the dependencies gWidgetsRGtk2 and cairoDevice from CRAN you need to choose the version "OS X Mavericks binaries" and not "Package source" - You have to be sure that gdal is installed with HDF4 support.
1 - Update to R > 3.4 if needed, then update all packages
update.packages()# install the development version of devtools:install.packages(c("devtools"))devtools::install_github("hadley/devtools")
2 - Now, install RGtk2 using Homebrew (https://gist.github.com/sebkopf/9405675). First, ensure you have cairo installed with "--with-x11". Open a terminal and run:
brew uninstall cairo --ignore-dependenciesbrew install --with-x11 cairo
next, edit the configure options for GTK to require x11 rather than Quartz:
brew edit gtk+
in the def install section, remove the reference to quartz and switch to:
brew install --build-from-source --verbose gtk+export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/usr/local/lib/pkgconfig/gtk+-2.0.pc:/opt/X11/lib/pkgconfig
3 - next, install
RGtk2. Since install.packages("Rgtk2") is not going to work, go here, download RGtk2_2.20.33.tar.gz and, from a terminal run:
R CMD INSTALL RGtk2_2.20.33.tar.gz
Now, open R and run:
RGTk2 will load without errors !
4 - Install packages
Very important !!!! from CRAN, download the "OS X Mavericks binaries" for gWidgetsRGtk2 and cairoDevice( not "Package source"). Save both to Frameworks/R.framework/3.4..., open R and run the code below (This will also install cairoDevice)
(This will work for R version 3.4, mac OS X Sierra)
5 - Install GDAL with HDF4 support
Check that gdal is installed with hdf4 support. From a terminal:
if gdal is installed, check what drivers are installed: the list should include hdf4.
If gdal is not yet installed or hdf4 is not supproted, install/reinstall it following these notes
> brew install hdf4# prefer hdf4 links over NetCDF> brew link --overwrite hdf4> brew install gdal --complete --enable-unsupported --with-hdf4# check what drivers are installed, list should now include hdf4:> gdal-config --formats
6 - Install rgdal
since rgdal is not usually looking in "/usr/local/lib"" you must include that with
install.packages('rgdal',type = "source",configure.args = c('--with-proj-include=/usr/local/include','--with-proj-lib=/usr/local/lib'))
7 - Finally, install MODIStsp from CRAN or GitHub:
library(devtools)install_github("lbusett/MODIStsp", ref = "master")MODIStsp()
This is mostly a maintenance release - fixing a bug in 1.3.3 submission related to a missing import in NAMESPACE
Improved organization of Virtual Raster files and RData files in the "Time_Series" output subfolder. Now virtual files and RData files are organized by sensor and layer to facilitate access.
v1.3.3 was released on 10/08/2017
Added functionality for unit testing using
testthat and codecov integration.
Added "testing mode" to allow users to test proper functioning. Now, runnning
MODIStsp(test = X) (with X in (0,6)) runs the processing using default processing
parameters (commit 0c00fc6 and others)
Fxed a bug preventing the "last" choice (or that present in the json file) from correctly showing in the GUI upon launch/restore of a saved json file (commit 633c2dd)
Fixed a bug affecting MODIS layers coded as Unsigned Integer - 32 bit (Thanks to Rob Critchlow for signaling this). The bug was due to improper handling of UInt32 data in
gdalbuildvrt, causing sometimes an incorrect translation from HDF to output formats (#72).
M*D09A1 - 500m Reflectance Band Quality (V005 and V006); M*DO9CMG - Coarse Resolution QA (V005 and V006); M*D09CMG - Coarse Resolution Number Mapping (V006); M*D09GA - 500m Reflectance Band Quality (V005 and V006); M*DOCGA - Band quality for MODIS bands 8-15 (V006); M*D11C3 - Days with clear-sky conditions and validated LSTs; Nights with clear-sky conditions and validated LSTs (V005 and V006); MCD43A2 - BRDF_Albedo_Band_Quality (V005 and V006).
Fixed a bug affecting creation of time series files (RData and virtual rasters) on all MCD products (#77)
Fixed a bug a error on creation of "burn_date" layers for MCD45A1 product (#77)
Fixed bugs on specifying spatial extent files on non-interactive execution (#75)
MODIStsp was recently accepted on CRAN. From now on, you can install it simply using
You'll however still be able to install the
development version from github,
containing the last improvements and bug fixing using:
install_github("lbusett/MODIStsp", ref = "master")
v1.3.2 was released on 22/03/2017
MODIS hdf datasets are always stored as integer values, with scales factor and/or offsets to apply in order to convert to the indicated measure units reported in the products' documentation.
Starting from v1.3.2:
Some product names and output layer names were modified to reduce the length of output file names, homogenize the names of the outputs and correct some errors.
For compatibility with already created output files (versions up to 1.3.1), the old "XML" file specifying output files format is still available in
inst/ExtData/MODIStsp_ProdOpts_old_v1.3.1.xml. To use the old file naming conventions, users have to:
Timeouts of httr/ftp requests were increased to prevent problems on download on slow connections
v1.3.1 was released on 13/02/2017
Added functionality for processing of Snow Cover datasets: MOD10A1, MOD10A2, MOD10C1, MOD10C2, MOD10CM (Issue #55) on devel
Added functionality for downloading "partial" years(Issue #54) on devel
Added functionality for computing vegetation indexes on MCD43A4 (v5-v6), MCD43B4 (v5), MCD43C4 (v5-v6) (Issue #59) on master/devel
Added functionality for accelerating download using aria2c (Issue #55) on devel
Fixed bug on download with aria, throwing an error on partial download on http downlaod with aria (6fbc875)
Fixed bug on M*D15A2 processing (Issue #60) on devel/master
Fixed bug on MCD12Q1 processing (Issue #58) on devel/master
Fixed bug on MOD13C2 processing (Issue #52) on devel/master
Fixed bug on insertion of custom projection (Issue #57) on devel/master
Fixed bug on selection of custom index (Issue #53) on devel/master
v1.3.0 was released on 11/05/2016
Added functionality for downloading and preprocessing MODIS collection 006 datasets. For products with both 005 and 006 collections, the user can select the version using a new droplist in the GUI.
Added functionality for off-line processing. This allows both to i) reprocessing already downloaded data (for example, to create time series for an additional layer) without the need to connect to NASA servers, and ii) process HDF files downloaded outside MODIStsp (e.g., directly from NASA ftp) and stored on the user's PC, without the need of an active internet connection.
Improved the way in which options are saved. Much more readable .JSON files are now used instead than .RData. User options are no longer saved alongside products characteristics. This will allow to re-use an "old" options file even if changes are made on the XML file descriving the products.
Improved the GUI inteface for specifying additional Spectral Indexes. Hints are now showed to the user, and multiple indexes can be added in the same session.
General improvements in the GUI inteface. Products are now grouped by categories, to allow easier identification and selection.
Improvements in the README file and vignettes, providing more instructions on package use.
Improved functionality for checking for "complete" download, by comparing the size of the downloaded files with that of files on the server.
Added "configure" file for Linux installation.
Temporary files necessary for processing (e.g., vrt files) are now created (and destroyed) within the "R" temporary folder.
v1.2.1 was released on 11/05/2016
Modified format of "R" output time series from rts objects to RasterStack objects with temporal information added in the "z" attribute via setZ()
Major changes/improvements in MODIStsp_extract function:
Removed possibility to use "complex" resampling methods when reprojecting (e.g., bilinear, cubic, etc.) to avoid incorrect resampling on categorical variables and "contamination" of good pixels data.
v1.2.0 was released on 29/07/2015
First stable release of advanced implementation of MODIStsp ! We know it should be 1.0.0, but thats'it !