A cross-platform interface to file system operations, built on top of the 'libuv' C library.
fs provides a cross-platform, uniform interface to file system operations. It shares the same back-end component as nodejs, the libuv C library, which brings the benefit of extensive real-world use and rigorous cross-platform testing. The name, and some of the interface, is partially inspired by Rust’s fs module.
You can install the released version of fs from CRAN with:
install.packages("fs")
And the development version from GitHub with:
# install.packages("devtools")devtools::install_github("r-lib/fs")
fs functions smooth over some of the idiosyncrasies of file handling with base R functions:
Vectorization. All fs functions are vectorized, accepting multiple paths as input. Base functions are inconsistently vectorized.
Predictable return values that always convey a path. All fs functions return a character vector of paths, a named integer or a logical vector, where the names give the paths. Base return values are more varied: they are often logical or contain error codes which require downstream processing.
Explicit failure. If fs operations fail, they throw an error. Base functions tend to generate a warning and a system dependent error code. This makes it easy to miss a failure.
UTF-8 all the things. fs functions always convert input paths to UTF-8 and return results as UTF-8. This gives you path encoding consistency across OSes. Base functions rely on the native system encoding.
Naming convention. fs functions use a consistent naming
convention. Because base R’s functions were gradually added over
time there are a number of different conventions used (e.g.
path.expand()
vs normalizePath()
; Sys.chmod()
vs
file.access()
).
fs functions always return ‘tidy’ paths. Tidy paths
/
to delimit directories/
or trailing /
Tidy paths are also coloured (if your terminal supports it) based on the
file permissions and file type. This colouring can be customized or
extended by setting the LS_COLORS
environment variable, in the same
output format as GNU
dircolors.
fs functions are divided into four main categories:
path_
for manipulating and constructing pathsfile_
for filesdir_
for directorieslink_
for linksDirectories and links are special types of files, so file_
functions
will generally also work when applied to a directory or link.
library(fs)# Construct a path to a file with `path()`path("foo", "bar", letters[1:3], ext = "txt")#> foo/bar/a.txt foo/bar/b.txt foo/bar/c.txt# list files in the current directorydir_ls()#> DESCRIPTION LICENSE.md NAMESPACE NEWS.md#> R README.Rmd README.md _pkgdown.yml#> a appveyor.yml b c#> codecov.yml cran-comments.md docs fs.Rproj#> fs_1.2.7.tar.gz inst man man-roxygen#> revdep script.R src tests#> top1 top2 vignettes# create a new directorytmp <- dir_create(file_temp())tmp#> /var/folders/dt/r5s12t392tb5sk181j3gs4zw0000gn/T/RtmpQMfPEs/file952a698992af# create new files in that directoryfile_create(path(tmp, "my-file.txt"))dir_ls(tmp)#> /var/folders/dt/r5s12t392tb5sk181j3gs4zw0000gn/T/RtmpQMfPEs/file952a698992af/my-file.txt# remove files from the directoryfile_delete(path(tmp, "my-file.txt"))dir_ls(tmp)#> character(0)# remove the directorydir_delete(tmp)
fs is designed to work well with the pipe, though because it is a minimal-dependency infrastructure package it doesn’t provide the pipe itself. You will need to attach magrittr or similar.
library(magrittr)#>#> Attaching package: 'magrittr'#> The following objects are masked from 'package:testthat':#>#> equals, is_less_than, notpaths <- file_temp() %>%dir_create() %>%path(letters[1:5]) %>%file_create()paths#> /var/folders/dt/r5s12t392tb5sk181j3gs4zw0000gn/T/RtmpQMfPEs/file952a21ea91af/a#> /var/folders/dt/r5s12t392tb5sk181j3gs4zw0000gn/T/RtmpQMfPEs/file952a21ea91af/b#> /var/folders/dt/r5s12t392tb5sk181j3gs4zw0000gn/T/RtmpQMfPEs/file952a21ea91af/c#> /var/folders/dt/r5s12t392tb5sk181j3gs4zw0000gn/T/RtmpQMfPEs/file952a21ea91af/d#> /var/folders/dt/r5s12t392tb5sk181j3gs4zw0000gn/T/RtmpQMfPEs/file952a21ea91af/epaths %>% file_delete()
fs functions also work well in conjunction with other tidyverse packages, like dplyr and purrr.
Some examples…
suppressMessages(library(tidyverse))
Filter files by type, permission and size
dir_info("src", recursive = FALSE) %>%filter(type == "file", permissions == "u+r", size > "10KB") %>%arrange(desc(size)) %>%select(path, permissions, size, modification_time)#> Warning: `recursive` is deprecated, please use `recurse` instead#> # A tibble: 11 x 4#> path permissions size modification_time#> <fs::path> <fs::perms> <fs::bytes> <dttm>#> 1 src/RcppExports.o rw-r--r-- 874.8K 2019-04-02 14:24:08#> 2 src/dir.o rw-r--r-- 493.9K 2019-04-02 14:24:08#> 3 src/id.o rw-r--r-- 388.8K 2019-03-18 09:33:55#> 4 src/fs.so rwxr-xr-x 367.3K 2019-04-02 14:24:08#> 5 src/file.o rw-r--r-- 358.7K 2019-03-18 10:10:35#> 6 src/path.o rw-r--r-- 279.9K 2019-03-18 09:33:55#> 7 src/link.o rw-r--r-- 227.7K 2019-03-18 09:33:55#> 8 src/utils.o rw-r--r-- 127.2K 2019-03-18 11:29:09#> 9 src/error.o rw-r--r-- 17.6K 2019-03-18 10:02:15#> 10 src/RcppExports.cpp rw-r--r-- 12.2K 2019-04-02 14:18:40#> 11 src/file.cc rw-r--r-- 10.1K 2019-03-18 10:08:54
Tabulate and display folder size.
dir_info("src", recursive = TRUE) %>%group_by(directory = path_dir(path)) %>%tally(wt = size, sort = TRUE)#> Warning: `recursive` is deprecated, please use `recurse` instead#> # A tibble: 53 x 2#> directory n#> <chr> <fs::bytes>#> 1 src 3.11M#> 2 src/libuv 2.43M#> 3 src/libuv/src/unix 1.1M#> 4 src/libuv/test 865.35K#> 5 src/libuv/src/win 683.14K#> 6 src/libuv/docs/src/static 328.3K#> 7 src/libuv/m4 319.95K#> 8 src/libuv/include 192.33K#> 9 src/libuv/docs/src/static/diagrams.key 184.02K#> 10 src/libuv/src 181.85K#> # … with 43 more rows
Read a collection of files into one data frame.
dir_ls()
returns a named vector, so it can be used directly with
purrr::map_df(.id)
.
# Create separate files for each speciesiris %>%split(.$Species) %>%map(select, -Species) %>%iwalk(~ write_tsv(.x, paste0(.y, ".tsv")))# Show the filesiris_files <- dir_ls(glob = "*.tsv")iris_files#> setosa.tsv versicolor.tsv virginica.tsv# Read the data into a single table, including the filenamesiris_files %>%map_df(read_tsv, .id = "file", col_types = cols(), n_max = 2)#> # A tibble: 6 x 5#> file Sepal.Length Sepal.Width Petal.Length Petal.Width#> <chr> <dbl> <dbl> <dbl> <dbl>#> 1 setosa.tsv 5.1 3.5 1.4 0.2#> 2 setosa.tsv 4.9 3 1.4 0.2#> 3 versicolor.tsv 7 3.2 4.7 1.4#> 4 versicolor.tsv 6.4 3.2 4.5 1.5#> 5 virginica.tsv 6.3 3.3 6 2.5#> 6 virginica.tsv 5.8 2.7 5.1 1.9file_delete(iris_files)
We hope fs is a useful tool for both analysis scripts and packages. Please open GitHub issues for any feature requests or bugs.
In particular, we have found non-ASCII filenames in non-English locales on Windows to be especially tricky to reproduce and handle correctly. Feedback from users who use commonly have this situation is greatly appreciated.
Fix missed test with UTF-8 characters, which now passes on a strict Latin-1 locale.
Fix undefined behavior when casting -1 to size_t
.
dir_ls()
, dir_map()
, dir_walk()
, dir_info()
and dir_tree()
gain a
recurse
argument, which can be either a TRUE
or FALSE
(as was supported
previously) or a number of levels to recurse. The previous argument
recursive
has been deprecated.dir_copy()
gains a overwrite
argument, to overwrite a given directory
(@pasipasi123, #193)dir_create()
now throws a more accurate error message when you try to
create a directory in a non-writeable location (#196).
fs_path
objects now always show 10 characters by default when printed in
tibbles (#191).
path_file()
, path_dir()
and path_ext()
now return normal character
vectors rather than tidy paths (#194).
path_package()
now works with paths in development packages automatically
(#175).
tests now pass successfully when run in strict Latin-1 locale
file_size()
function added as a helper for file_info("file")$size
(#171)
is_file_empty()
function added to test for empty files` (#171)
dir_tree()
function added to print a command line representation of a
directory tree, analogous to the unix tree
program (#82).
Add a comparison vignette to quickly compare base R, fs and shell alternatives (@xvrdm, #168).
path_ext_set()
and file_temp()
now treat extensions with a leading .
and those without equally. e.g. path_ext_set("foo", ext = "bar")
and
path_ext_set("foo", ext = ".bar")
both result in "foo.bar"
Tidy paths are now always returned with uppercase drive letters on Windows (#174).
format.bench_bytes()
now works with str()
in R 3.5.1+ (#155).
path_ext()
, path_ext_remove()
, and path_ext_set()
now work on paths
with no extension, and file_temp()
now prepends a .
to the file extension
(#153).
Link with -pthread by default and fix on BSD systems (#128, #145, #146).
file_chown()
can now take a group_id
parameter as character (@cderv, #162).
Parameter browser
in file_show()
now works as described in the documentation (@GegznaV, #154).
path_real()
now works even if the file does not exist, but there are
symlinks further up the path hierarchy (#144).
colourise_fs_path()
now returns paths uncolored if the colors argument /
LS_COLORS
is malformed (#135).
This is a small bugfix only release.
file_move()
now fall back to copying, then removing files when moving files
between devices (which would otherwise fail) (#131, https://github.com/r-lib/usethis/issues/438).
Fix for a double free when using warn = TRUE
(#132)
New path_wd()
generates paths from the current working directory (#122).
New path_has_parent()
determines if a path has a given parent (#116).
New file_touch()
used to change access and modification times for a file (#98).
dir_ls()
, dir_map()
, dir_walk()
, dir_info()
and file_info()
gain a
fail
parameter, to signal warnings rather than errors if they are called on
a path which is unavailable due to permissions or locked resources (#105).
path_tidy()
now always includes a trailing slash for the windows root
directory, e.g. C:/
(#124).
path_ext()
, path_ext_set()
and path_ext_remove()
now handle paths with
non-ASCII characters (#120).
fs_path
objects now print (without colors) even if the user does not have
permission to stat them (#121).
compatibility with upcoming gcc 8 based Windows toolchain (#119)
Experimental support for /
and +
methods for fs_path
objects (#110).
file_create()
and dir_create()
now take ...
, which is passed to
path()
to make construction a little nicer (#80).
path_ext()
, path_ext_set()
and path_ext_remove()
now handle paths with
directories including hidden files without extensions (#92).
file_copy()
now copies files into the directory if the target is a
directory (#102).
fs_path
fs_bytes
and fs_perm
objects now use methods::setOldClass()
so that S4 dispatch to their base classes works as intended (#91).
Fix allocation bug in path_exists()
using delete []
when we should have
used free()
.
path_abs()
gains a start
argument, to specify where the absolute path
should be calculated from (#87).path_ext()
now returns character()
if given 0 length inputs (#89)
Fix for a memory issue reported by ASAN and valgrind.
path_expand()
and path_home()
now use USERPROFILE
or
HOMEDRIVE
/HOMEPATH
as the user home directory on Windows. This differs
from the definition used in path.expand()
but is consistent with
definitions from other programming environments such as python and rust. This
is also more compatible with external tools such as git and ssh, both of
which put user-level files in USERPROFILE
by default. To mimic R's (and
previous) behavior there are functions path_expand_r()
and path_home_r()
.
Handling missing values are more consistent. In general is_*
functions
always return FALSE
for missing values, path_*
functions always propagate
NA values (NA inputs become NA outputs) and dir_*
file_*
and link_*
functions error with NA inputs.
fs functions now preserve tildes in their outputs. Previously paths were
always returned with tildes expanded. Users can use path_expand()
to expand
tildes if desired.
path_norm()
(#63).file_chmod()
is now vectorized over both of its arguments (#71).link_create()
now fails silently if an identical link already exists (#77).path_package()
function created as an analog to system.file()
which
always fails if the package or file does not exist (#75)Tidy paths no longer expand ~
.
Filesystem modification functions now error for NA inputs. (#48)
path()
now returns 0 length output if given any 0 length inputs (#54).
dir_delete()
now correctly expands paths (#47).
dir_delete()
now correctly deletes hidden files and directories (#46).
link_path()
now checks for an error before trying to make a string,
avoiding a crash (#43).
libuv return paths now marked as UTF-8 strings in C code, fixing encoding issues on windows. (#42)
dir_copy()
now copies the directory inside the target if the target is a
directory (#51).
dir_copy()
now works correctly with absolute paths and no longer removes
files when overwrite = TRUE
.
NEWS.md
file to track changes to the package.