Read Hierarchical Fixed Width Files

Read hierarchical fixed width files like those commonly used by many census data providers. Also allows for reading of data in chunks, and reading 'gzipped' files without storing the full file in memory.

ProjectStatus:Active CRAN status Travis build status AppVeyor build status

hipread (hierarchical IPUMS reader) is a fork from tidyverse readr that allows for reading hierarchical fixed width text files, like those created by the CSPro software and commonly used by census data providers.

Compared to readr it is:

  • Able to natively read the "hierarchical" fixed width file format that IPUMS and some other census data providers use. These files can have multiple types of observations in them, each with their own specification of variables.

  • Better at reading gzipped data. It does not require loading the full file into a raw vector, which takes a large amount of memory, and prevents reading gigantic files altogether (because R can only store raw vectors of a certain size).

  • Less flexible. It only works on fixed width files, only accepts data of types character, double and integer, and is less detailed about the information it gives about parsing failures. This makes it easier for me to maintain.

I do not expect that this will be directly useful for too many people, so the documentation is a little bit light. Instead I expect most users will use this package through the ipumsr package. But, if you are interested and find something confusing, please let me know!


Install the development version from GitHub with:

# install.packages("devtools")


hipread 0.2.0

  • Added yield functions (hipread_long_yield() and hipread_list_yield()) which are another take on reading data in chunks that allow for more flexibility.

  • Progress bar now ends at 100% instead of looking like the read was incomplete (#6)

  • Several performance improvements

hipread 0.1.1

  • Fixes for platform-specific bugs revealed by CRAN checks (Solaris, UBSAN, Fedora)

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.


0.2.1 by Derek Burk, 7 days ago

Browse source code at

Authors: Greg Freedman Ellis [aut] , Derek Burk [aut, cre] , Joe Grover [ctb] , Mark Padgham [ctb] , Hadley Wickham [ctb] (Code adapted from readr) , Jim Hester [ctb] (Code adapted from readr) , Romain Francois [ctb] (Code adapted from readr) , R Core Team [ctb] (Code adapted from readr) , RStudio [cph, fnd] (Code adapted from readr) , Jukka Jylänki [ctb, cph] (Code adapted from readr) , Mikkel Jørgensen [ctb, cph] (Code adapted from readr) , University of Minnesota [cph]

Documentation:   PDF Manual  

GPL (>= 2) | file LICENSE license

Imports Rcpp, R6, rlang, tibble

Suggests dplyr, readr, testthat

Linking to Rcpp, BH

System requirements: C++11

Imported by ipumsr.

See at CRAN