A Test Environment for HTTP Requests

Testing and documenting code that communicates with remote servers can be painful. Dealing with authentication, server state, and other complications can make testing seem too costly to bother with. But it doesn't need to be that hard. This package enables one to test all of the logic on the R sides of the API in your package without requiring access to the remote service. Importantly, it provides three contexts that mock the network connection in different ways, as well as testing functions to assert that HTTP requests were---or were not---made. It also allows one to safely record real API responses to use as test fixtures. The ability to save responses and load them offline also enables one to write vignettes and other dynamic documents that can be distributed without access to a live server.


Build Status Build status codecov cran CII Best Practices

httptest makes it easy to write tests for code and packages that wrap web APIs. Testing code that communicates with remote servers can otherwise be painful: things like authentication, server state, and network flakiness can make testing seem too costly to bother with. The httptest package enables you to test all of the logic on the R sides of the API in your package without requiring access to the remote service.

Importantly, it provides multiple contexts that mock the network connection and tools for recording real requests for future offline use as fixtures, both in tests and in vignettes. The package also includes additional expectations to assert that HTTP requests were---or were not---made.

Using these tools, you can test that code is making the intended requests and that it handles the expected responses correctly, all without depending on a connection to a remote API. The ability to save responses and load them offline also enables you to write package vignettes and other dynamic documents that can be distributed without access to a live server.

This package bridges the gap between two others: (1) testthat, which provides a useful (and fun) framework for unit testing in R but doesn't come with tools for testing across web APIs; and (2) httr, which makes working with HTTP in R easy but doesn't make it simple to test the code that uses it. httptest brings the fun and simplicity together.

Installing

httptest can be installed from CRAN with

install.packages("httptest")

The pre-release version of the package can be pulled from GitHub using the remotes package (formerly part of and now a dependency of devtools):

# install.packages("remotes")
remotes::install_github("nealrichardson/httptest")

Using

To start using httptest with your package, run use_httptest() in the root of your package directory. This will

  • add httptest to "Suggests" in the DESCRIPTION file
  • add library(httptest) to tests/testthat/helper.R, which testthat loads before running tests

Then, you're ready to start using the tools that httptest provides. Here's an overview of how to get started. For a longer discussion and examples, see vignette("httptest"), and see also the package reference for a list of all of the test contexts and expectations provided in the package.

In your test suite

The package includes several contexts, which you wrap around test code that would otherwise make network requests through httr. They intercept the requests and prevent actual network traffic from occurring.

with_mock_api() maps requests---URLs along with request bodies and query parameters---to file paths. If the file exists, its contents are returned as the response object, as if the API server had returned it. This allows you to test complex R code that makes requests and does something with the response, simulating how the API should respond to specific requests.

Requests that do not have a corresponding fixture file raise errors that print the request method, URL, and body payload, if provided. expect_GET(), expect_POST(), and the rest of the HTTP-request-method expectations look for those errors and check that the requests match the expectations. These are useful for asserting that a function call would make a correctly-formed HTTP request without the need to generate a mock, as well as for asserting that a function does not make a request (because if it did, it would raise an error in this context).

Adding with_mock_api() to your tests is straightforward. Given a very basic test that makes network requests:

test_that("Requests happen", {
    expect_is(GET("http://httpbin.org/get"), "response")
    expect_is(
        GET("http://httpbin.org/response-headers",
            query=list(`Content-Type`="application/json")),
        "response"
    )
})

if we wrap the code in with_mock_api(), actual requests won't happen.

with_mock_api({
    test_that("Requests happen", {
        expect_is(GET("http://httpbin.org/get"), "response")
        expect_is(
            GET("http://httpbin.org/response-headers",
                query=list(`Content-Type`="application/json")),
            "response"
        )
    })
})

Those requests will now raise errors unless we either (1) wrap them in expect_GET() and assert that we expect those requests to happen, or (2) supply mocks in the file paths that match those requests. We might get those mocks from the documentation for the API we're using, or we could record them ourselves---and httptest provides tools for recording.

Another context, capture_requests(), collects the responses from requests you make and stores them as mock files. This enables you to perform a series of requests against a live server once and then build your test suite using those mocks, running your tests in with_mock_api.

In our example, running this once:

capture_requests({
    GET("http://httpbin.org/get")
    GET("http://httpbin.org/response-headers",
        query=list(`Content-Type`="application/json"))
})

would make the actual requests over the network and store the responses where with_mock_api() will find them.

For convenience, you may find it easier in an interactive session to call start_capturing(), make requests, and then stop_capturing() when you're done, as in:

start_capturing()
GET("http://httpbin.org/get")
GET("http://httpbin.org/response-headers",
    query=list(`Content-Type`="application/json"))
stop_capturing()

Mocks stored by capture_requests are written out as plain-text files. By storing fixtures as human-readable files, you can more easily confirm that your mocks look correct, and you can more easily maintain them if the API changes subtly without having to re-record them (though it is easy enough to delete and recapture). Text files also play well with version control systems, such as git.

When recording requests, httptest redacts the standard ways that auth credentials are passed, so you won't accidentally publish your personal tokens. The redacting behavior is fully customizable: you can programmatically sanitize or alter other parts of the request and response. See vignette("redacting") for details.

In your vignettes

Package vignettes are a valuable way to show how to use your code, but when communicating with a remote API, it has been difficult to write useful vignettes. With httptest, however, by adding as little as one line of code to your vignette, you can safely record API responses from a live session, using your secret credentials. These API responses are scrubbed of sensitive personal information and stored in a subfolder in your vignettes directory. Subsequent vignette builds, including on continuous-integration services, CRAN, and your package users' computers, use these recorded responses, allowing the document to regenerate without a network connection or API credentials. To record fresh API responses, delete the subfolder of cached responses and re-run.

To use httptest in your vignettes, add a code chunk with start_vignette() at the beginning, and for many use cases, that's the only thing you need. If you need to handle changes of server state, as when you make an API request that creates a record on the server, add a call to change_state(). See vignette("vignettes") for more discussion and links to examples.

FAQ

Where are my mocks recorded?

By default, the destination path for capture_requests() is relative to the current working directory of the process. This matches the behavior of with_mock_api(), which looks for files relative to its directory, which typically is tests/testthat/.

If you're running capture_requests within a test suite in an installed package, or if you're running interactively from a different directory, the working directory may not be the same as your code repository. If you aren't sure where the files are going, set options(httptest.verbose=TRUE), and it will message the absolute path of the files as it writes them.

To change where files are being written or read from, use .mockPaths() (like base::.libPaths()) to specify a different directory.

How do I fix "non-portable file paths"?

If you see this error in R CMD build or R CMD check, it means that there are file paths are longer than 100 characters, which can sometimes happen when you record requests. httptest preserves the URL structure of mocks in file paths to improve the readability and maintainability of your tests, as well as to make visible the properties of your API. Indeed, the file-system tree view of the mock files gives a visual representation of your API. This value comes with a tradeoff: sometimes URLs can be long, and R doesn't like that.

Depending on how long your URLs are, there are a few ways to save on characters without compromising readability of your code and tests.

A big way to cut long file paths is by using a request preprocessor: a function that alters the content of your 'httr' request before mapping it to a mock file. For example, if all of your API endpoints sit beneath https://language.googleapis.com/v1/, you could set a request preprocessor like:

set_requester(function (request) {
    gsub_request(request, "https\\://language.googleapis.com/v1/", "api/")
})

and then all mocked requests would look for a path starting with "api/" rather than "language.googleapis.com/v1/", saving you (in this case) 23 characters.

You can also provide this function in inst/httptest/request.R, and any time your package is loaded (as when you run tests or build vignettes), this function will be called automatically. See vignette("redacting") for more.

You may also be able to economize on other parts of the file paths. If you've recorded requests and your file paths contain long ids like "1495480537a3c1bf58486b7e544ce83d", depending on how you access the API in your code, you may be able to simply replace that id with something shorter, like "1". The mocks are just files, disconnected from a real server and API, so you can rename them and munge them as needed.

Finally, if you have your tests inside a tests/testthat/ directory, and your fixture files inside that, you can save 9 characters by moving the fixtures up to tests/ and setting .mockPaths("../").

How do I switch between mocking and real requests?

Q. I'd like to run my mocked tests sometimes against the real API, perhaps to turn them into integration tests, or perhaps to use the same test code to record the mocks that I'll later use. How can I do this without copying the contents of the tests inside the with_mock_api() blocks?

A. One way to do this is to set with_mock_api() to another function in your test file (or in helper.R if you want it to run for all test files). So

with_mock_api({
    a <- GET("https://httpbin.org/get")
    print(a)
})

looks for the mock file, but

with_mock_api <- force
with_mock_api({
    a <- GET("https://httpbin.org/get")
    print(a)
})

just evaluates the code with no mocking and makes the request, and

with_mock_api <- capture_requests
with_mock_api({
    a <- GET("https://httpbin.org/get")
    print(a)
})

would make the request and record the response as a mock file. You could control this behavior with environment variables by adding something like

if (Sys.getenv("MOCK_BYPASS") == "true") {
    with_mock_api <- force
} else if (Sys.getenv("MOCK_BYPASS") == "capture") {
    with_mock_api <- capture_requests
}

to your helper.R or setup.R.

You could also experiment with using start_vignette(), which switches behavior based on the existence of the specified mock directory.

Contributing

Suggestions and pull requests are more than welcome!

For developers

The repository includes a Makefile to facilitate some common tasks from the command line, if you're into that sort of thing.

Running tests

$ make test. You can also specify a specific test file or files to run by adding a "file=" argument, like $ make test file=offline. test_package will do a regular-expression pattern match within the file names. See its documentation in the testthat package.

Updating documentation

$ make doc. Requires the roxygen2 package.

News

httptest 3.2.2

  • Patch for compatibility with the upcoming 1.4.0 release of httr.

httptest 3.2.0

  • use_httptest() for convenience when setting up a new package
  • Warn when capturing requests if the httr request function errors and no response file is written (#16)
  • Support recording with capture_requests() when directly calling GET et al. interactively with the httr package attached (#17)
  • Support regular expression matching of URLs and request bodies in expect_GET() et al. (#19)

httptest 3.1.0

Better, more efficient response recording

  • capture_requests() no longer includes the "request" object inside the recorded response when writing .R verbose responses. As of 3.0.0, with_mock_api() inserts the current request when loading mock files, so it was being overwritten anyway. This eliminates some (though not all) of the need for redacting responses. As a result, the redacting functions redact_oauth() and redact_http_auth() have been removed because they only acted on the response$request, which is now dropped entirely.
  • capture_requests() will record simplified response bodies for a range of Content-Types when simplify=TRUE (the default). Previously, only .json (Content-Type: application/json) was recorded as a simple text files; now, .html, .xml, .txt, .csv, and .tsv are supported.
  • When recording with simplify=TRUE, HTTP responses with 204 No Content status are now written as empty files with .204 extension. This saves around 2K of disk space per file.
  • with_mock_api() now can also load these newly supported file types.
  • Bare JSON files written by capture_requests() are now "prettified" (i.e. multiline, nice indentation).
  • capture_requests() now records responses from httr::RETRY() (#13)

Vignette setup and teardown

  • Store package-level vignette setup and teardown code, called inside start_vignette() and end_vignette(), in inst/httptest/start-vignette.R and inst/httptest/end-vignette.R, respectively. Like with the package redactors and request preprocessors, these are automatically executed whenever your package is loaded and start/end_vignette is called. This makes it easy to write multiple vignettes without having to copy and paste as much setup code. See vignette("vignettes") for details.

Other enhancements and options

  • gsub_response() now applies over the URL in a Location header, if found.
  • Add options(httptest.max.print) to allow you the ability to specify a length to which to truncate the request body printed in the error message for requests in with_mock_api(). Useful for debugging mock files not found when there are large request bodies.
  • Add options(httptest.debug), which if TRUE prints more details about which functions are being traced (by base::trace()) and when they're called.
  • Deprecate the "verbose" argument to capture_requests(): use options(httptest.verbose) instead.

httptest 3.0.0

Major features

  • Write vignettes and other R Markdown documents that communicate with a remote API using httptest. Add a code chunk at the beginning of the document including start_vignette(). The first time you run the document, the real API responses are recorded to a subfolder in your vignettes directory. Subsequent vignette builds use these recorded responses, allowing the document to regenerate without a network connection or API credentials. If your document needs to handle changes of server state, as when you make an API request that creates a record on the server, add a call to change_state(). See vignette("vignettes") for more discussion and links to examples.
  • Packages can now have a default redacting function, such that whenever the package is loaded, capture_requests() will apply that function to any responses it records. This ensures that you never forget to sanitize your API responses if you need to use a custom function. To take advantage of this feature, put a function (response) {...} in a file at inst/httptest/redact.R in your package. See the updated vignette("redacting", package="httptest") for more.
  • You can also now provide a function to preprocess mock requests. This can be particularly for shortening URLs---and thus the mock file paths---because of CRAN-mandated constraints on file path lengths ("non-portable file paths"). This machinery works very similar to redacting responses when recording them, except it operates on request objects inside of with_mock_api(). To use it, either pass a function (request) {...} to set_requester() in your R session, or to define one for the package, put a function (request) {...} in a file at inst/httptest/request.R. gsub_request() is particularly useful here. vignette("redacting", package="httptest") has further details.

Other big changes and enhancements

  • Standardize exported functions on snake_case rather than camelCase to better align with httr and testthat (except for .mockPaths(), which follows base::.libPaths()). Exported functions that have been renamed have retained their old aliases in this release, but they are to be deprecated.
  • use_mock_api() and block_requests() enable the request altering behavior of with_mock_api() and without_internet(), respectively, without the enclosing context. (use_mock_api is called inside start_vignette().) To turn off mocking, call stop_mocking().
  • Internal change: mocking contexts no longer use testthat::with_mock() and instead use trace().
  • capture_requests()/start_capturing() now allow you to call .mockPaths() while actively recording so that you can record server state changes to a different mock "layer". Previously, the recording path was fixed when the context was initialized.
  • The redact argument to capture_requests()/start_capturing() is deprecated in favor of set_redactor(). This function can take a function (response) {...}; a formula as shorthand for an anonymous function with . as the "response" argument, as in the purrr package; a list of functions that will be chained together; or NULL to disable the default redact_auth().
  • redact_headers() and within_body_text() no longer return redacting functions. Instead, they take response as their first argument. This makes them more natural to use and chain together in custom redacting functions. To instead return a function as before, see as.redactor().
  • gsub_response() is a new redactor that does regular-expression replacement (via base::gsub()) within a response's body text and URL.
  • .mockPaths() only keeps unique path values, consistent with base::.libPaths().
  • Option "httptest.verbose" to govern some extra debug messaging (automatically turned off by start_vignette())
  • Fix a bug where write_disk responses that were recorded in one location and moved to another directory could not be loaded

httptest 2.3.4

  • Ensure forward compatibility with a change in deparse() in the development version of R (r73699).

httptest 2.3.2

  • Add redact_oauth() to purge httr::Token() objects from requests (#9). redact_oauth() is built in to redact_auth(), the default redactor, so no action is required to start using it.

httptest 2.3.0

  • Remove support for mocking utils::download.file(), as testthat no longer permits it. Use httr::GET(url, config=write_disk(filename)) instead, which httptest now more robustly supports in capture_requests().

httptest 2.2.0

  • Add redacting functions (redact_auth(), redact_cookies(), redact_http_auth(), redact_headers(), within_body_text()) that can be specified in capture_requests() so that sensitive information like tokens and ids can be purged from recorded response files. The default redacting function is redact_auth(), which wraps several of them. See vignette("redacting", package="httptest") for more.
  • When loading a JSON mock response, the current "request" object is now included in the response returned, as is the case with real responses.
  • Remove the file size limitation for mock files loaded in with_mock_api()
  • skip_if_disconnected() now also wraps testthat::skip_on_cran() so that tests that require a real network connection don't cause a flaky test failure on CRAN

httptest 2.1.2

  • Fix for compatibility with upcoming release of httr that affected non-GET requests that did not contain any request body.

httptest 2.1.0

  • with_mock_api() and without_internet() handle multipart and urlencoded form data in mocked HTTP requests.
  • buildMockURL() escapes URL characters that are not valid in file names on all R platforms (which R CMD check would warn about).
  • capture_requests() now has a verbose argument, which, if TRUE, prints a message with the file path where each captured request is written.
  • capture_requests() takes the first element in .mockPaths() as its default "path" argument. The default is unchanged since .mockPaths() by default returns the current working directory, just as the "path" default was, but if you set a different mock path for reading mocks, capture_requests() will write there as well.

httptest 2.0.0

  • capture_requests() now writes non-JSON-content-type and non-200-status responses as full "response" objects in .R files. with_mock_api() now looks for .R mocks if a .json mock isn't found. This allows all requests and all responses, not just JSON content, to be mocked.
  • New .mockPaths() function, in the model of .libPaths(), which allows you to specify alternate directories in which to search for mock API responses.
  • Documentation enriched and vignette("httptest") added.

httptest 1.3.0

  • New context capture_requests() to collect the responses from real requests and store them as mock files
  • with_trace() convenience wrapper around trace/untrace
  • mockDownload() now processes request URLs as mock_request() does

httptest 1.2.0

  • Add support in with_mock_api() for loading request fixtures for all HTTP verbs, not only GET (#4). Include request body in the mock file path hashing.
  • buildMockURL() can accept either a 'request' object or a character URL
  • Bump mock payload max size up to 128K

httptest 1.1.2

  • Support full URLs, not just file paths, in with_mock_api() (#1)

httptest 1.1.0

  • expect_header() to assert that a HTTP request has a header
  • Always prune the leading ":///" that appears in with_mock_api() if the URL has a querystring

httptest 1.0.0

  • Initial addition of functions and tests, largely pulled from httpcache and crunch.

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.