Interface to Bold Systems API

A programmatic interface to the Web Service methods provided by Bold Systems (< http://www.boldsystems.org/>) for genetic 'barcode' data. Functions include methods for searching by sequences by taxonomic names, ids, collectors, and institutions; as well as a function for searching for specimens, and downloading trace files.


Project Status: Active – The project has reached a stable, usable state and is being actively developed. cran version

bold accesses BOLD barcode data.

The Barcode of Life Data Systems (BOLD) is designed to support the generation and application of DNA barcode data. The platform consists of four main modules: a data portal, a database of barcode clusters, an educational portal, and a data collection workbench.

This package retrieves data from the BOLD database of barcode clusters, and allows for searching of over 1.7M public records using multiple search criteria including sequence data, specimen data, specimen plus sequence data, as well as trace files.

Documentation for the BOLD API.

See also the taxize book for more options for taxonomic workflows with BOLD: https://ropensci.github.io/taxize-book/

cran checks Build Status codecov.io rstudio mirror downloads

Installation instructions

Stable Version

install.packages("bold")

Development Version

Install sangerseqR first

source("http://bioconductor.org/biocLite.R")
biocLite("sangerseqR")

Then bold

devtools::install_github("ropensci/bold")

Usage

library("bold")

Search for sequence data only

Default is to get a list back

bold_seq(taxon='Coelioxys')[[1]]
#> $id
#> [1] "ABEE013-17"
#> 
#> $name
#> [1] "Coelioxys afra"
#> 
#> $gene
#> [1] "ABEE013-17"
#> 
#> $sequence
#> [1] "AATATTATATATAATTTTTGCAATTTGATCAGGTATAATTGGATCTTCATTAAGAATAATTATTCGAATAGAATTAAGAACTCCAGGAAGATGAATCAACAACGATCAAATTTATAATTCTTTTATTACAGCTCATGCATTTTTAATAATTTTTTTTTTAGTAATACCATTTTTAATTGGAGGATTTGGAAATTGATTAGTACCTTTAATACTAGGAGCCCCCGATATAGCTTTTCCACGAATAAATAATGTAAGATTTTGACTATTACCTCCCTCAATTTTCTTATTATTATCAAGAACCCTAATTAACCCAAGAGCTGGTACTGGATGAACTGTATATCCTCCTTTATCCTTATATACATTTCATGCCTCACCTTCCGTTGATTTAGCAATTTTTTCACTTCATTTATCAGGAATTTCATCAATTATTGGATCAATAAATTTTATTGTTACAATCTTAATAATAAAAAATTTTTCTTTAAATTATAGACAAATACCATTATTTTCATGATCAGTTTTAATTACTACAATTTTACTTTTATTATCACTACCAATTTTAGCTGGAGCAATTACTATACTCCTATTTGATCGAAATTTAAATACCTCATTCTTTGACCCAATAGGAGGAGGAGATCCAATTTTATATCAACATTTATTT-----------------"

You can optionally get back the crul response object

res <- bold_seq(taxon='Coelioxys', response=TRUE)
res$response_headers
#> $status
#> [1] "HTTP/1.1 200 OK"
#> 
#> $date
#> [1] "Wed, 14 Nov 2018 00:49:40 GMT"
#> 
#> $server
#> [1] "Apache/2.2.15 (Red Hat)"
#> 
#> $`x-powered-by`
#> [1] "PHP/5.3.15"
#> 
#> $`content-disposition`
#> [1] "attachment; filename=fasta.fas"
#> 
#> $connection
#> [1] "close"
#> 
#> $`transfer-encoding`
#> [1] "chunked"
#> 
#> $`content-type`
#> [1] "application/x-download"

Search for specimen data only

By default you download tsv format data, which is given back to you as a data.frame

res <- bold_specimens(taxon='Osmia')
head(res[,1:8])
#>     processid      sampleid recordID    catalognum
#> 1  ABEE151-17  NHMW-HYM 773  8362250              
#> 2  ABEE184-17 NHMW-HYM 2142  8362283 NHMW-HYM 2142
#> 3  ABEE185-17 NHMW-HYM 2144  8362284 NHMW-HYM 2144
#> 4  ABEE188-17 NHMW-HYM 2242  8362287 NHMW-HYM 2242
#> 5  ABEE190-17 NHMW-HYM 2259  8362289 NHMW-HYM 2259
#> 6 GBAH3878-08      EU726629   856409      EU726629
#>                                                       fieldnum
#> 1                                                             
#> 2 NBH2 Zimmermann 2017.04.03 Oesterreich WienAUTWienZimmermann
#> 3 NBH2 Zimmermann 2017.04.03 Oesterreich WienAUTWienZimmermann
#> 4       NHB3 Schoder 2017.05.30 Oesterreich WienAUTWienSchoder
#> 5       NBH9 Schoder 2017.06.15 Oesterreich WienAUTWienSchoder
#> 6                                                             
#>             institution_storing collection_code      bin_uri
#> 1 Naturhistorisches Museum Wien              NA BOLD:AAE5409
#> 2 Naturhistorisches Museum Wien              NA BOLD:AAE5409
#> 3 Naturhistorisches Museum Wien              NA BOLD:ADJ1069
#> 4 Naturhistorisches Museum Wien              NA BOLD:AAF2155
#> 5 Naturhistorisches Museum Wien              NA BOLD:AAD0313
#> 6      Mined from GenBank, NCBI              NA BOLD:AAA4494

Search for specimen plus sequence data

By default you download tsv format data, which is given back to you as a data.frame

res <- bold_seqspec(taxon='Osmia', sepfasta=TRUE)
res$fasta[1:2]
#> $`ABEE151-17`
#> [1] "----------------------------------------------------------TTTTTGCTATATGATCAGGTACAGTAGGTTCAGCTATAAGAATTATTATTCGAATAGAACTTAGAGTTCCAGGATCATGAATTTCTAATGACCAAATTTATAATACTTTAGTAACTGCTCATGCTTTTTTAATAATTTTCTTTCTTGTAATACCATTTCTAATTGGAGGATTTGGAAATTGATTAATTCCTTTAATATTAGGAATTCCAGATATAGCCTTTCCACGAATAAATAATATTAGATTTTGACTTTTACCACCTTCTTTAATATTATTAATATTAAGAAATTTTATAAATCCAAGTCCAGGAACTGGATGAACTGTTTATCCTCCTCTTTCATCTTATATATTTCATTCTTCCCCATCAGTAGATTTAGCAATTTTTTCATTACATATTTCCGGATTATCCTCTATTATAGGTTCATTAAATTTTATTGTCACAATTATTATAATAAAAAATATTTCATTAAAACATACTCAATTACCCTTATTTTCTTGATCTGTATTTATTACTACTATTTTATTACTTTTCTCTCTCCCAGTTTTAGCTGGAGCTATTACTATACTTTTATTTGATCGAAATTTTAACACCTCATTTTTTGACCCGACGGGAGGTGGAGATCCAATTTTATACCAACATTTATTTTGATTTTTTGGACAT-----------------------"
#> 
#> $`ABEE184-17`
#> [1] "-------CTCACTATAGGGATTCAACCAATCATAAAGATATTGGAATTCTTTATATAATTTTTGCTATATGATCAGGTACAGTAGGTTCAGCTATAAGAATTATTATTCGAATAGAACTTAGAGTTCCAGGATCATGAATTTCTAATGACCAAATTTATAATACTTTAGTAACTGCTCATGCTTTTTTAATAATTTTCTTTCTTGTAATACCATTTCTAATTGGAGGATTTGGAAATTGATTAATTCCTTTAATATTAGGAATTCCAGATATAGCCTTTCCACGAATAAATAATATTAGATTTTGACTTTTACCACCTTCTTTAATATTATTAATATTAAGAAATTTTATAAATCCAAGTCCAGGAACTGGATGAACTGTTTATCCTCCTCTTTCATCTTATATATTTCATTCTTCCCCATCAGTAGATTTAGCAATTTTTTCATTACATATTTCCGGATTATCCTCTATTATAGGTTCATTAAATTTTATTGTCACAATTATTATAATAAAAAATATTTCATTAAAACATACTCAATTACCCTTATTTTCTTGATCTGTATTTATTACTACTATTTTATTACTTTTCTCTCTCCCAGTTTTAGCTGGAGCTATTACTATACTTTTATTTGATCGAAATTTTAACACCTCATTTTTTGACC-------------------------------------------------------------------------------"

Or you can index to a specific sequence like

res$fasta['GBAH0293-06']
#> $`GBAH0293-06`
#> [1] "------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------TTAATGTTAGGGATTCCAGATATAGCTTTTCCACGAATAAATAATATTAGATTTTGACTGTTACCTCCATCTTTAATATTATTACTTTTAAGAAATTTTTTAAATCCAAGTCCTGGAACAGGATGAACAGTTTATCCTCCTTTATCATCAAATTTATTTCATTCTTCTCCTTCAGTTGATTTAGCAATTTTTTCTTTACATATTTCAGGTTTATCTTCTATTATAGGTTCATTAAATTTTATTGTTACAATTATTATAATAAAAAATATTTCTTTAAAATATATTCAATTACCTTTATTTTCTTGATCTGTATTTATTACTACTATTCTTTTATTATTTTCTTTACCTGTATTAGCTGGAGCTATTACTATATTATTATTTGATCGAAATTTTAATACATCTTTTTTTGATCCAACAGGAGGGGGAGATCCAATTCTTTATCAACATTTATTTTGATTTTTTGGTCATCCTGAAGTTTATATTTTAATTTTACCTGGATTTGGATTAATTTCTCAAATTATTTCTAATGAAAGAGGAAAAAAAGAAACTTTTGGAAATATTGGTATAATTTATGCTATATTAAGAATTGGACTTTTAGGTTTTATTGTT---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------"

Get trace files

This function downloads files to your machine - it does not load them into your R session - but prints out where the files are for your information.

x <- bold_trace(ids = 'ACRJP618-11', progress = FALSE)
read_trace(x$ab1)
#> Number of datapoints: 8877
#> Number of basecalls: 685
#> 
#> Primary Basecalls: NNNNNNNNNNNNNNNNNNGNNNTTGAGCAGGNATAGTAGGANCTTCTCTTAGTCTTATTATTCGAACAGAATTAGGAAATCCAGGATTTTTAATTGGAGATGATCAAATCTACAATACTATTGTTACGGCTCATGCTTTTATTATAATTTTTTTTATAGTTATACCTATTATAATTGGAGGATTTGGTAATTGATTAGTTCCCCTTATACTAGGAGCCCCAGATATAGCTTTCCCTCGAATAAACAATATAAGTTTTTGGCTTCTTCCCCCTTCACTATTACTTTTAATTTCCAGAAGAATTGTTGAAAATGGAGCTGGAACTGGATGAACAGTTTATCCCCCACTGTCATCTAATATTGCCCATAGAGGTACATCAGTAGATTTAGCTATTTTTTCTTTACATTTAGCAGGTATTTCCTCTATTTTAGGAGCGATTAATTTTATTACTACAATTATTAATATACGAATTAACAGTATAAATTATGATCAAATACCACTATTTGTGTGATCAGTAGGAATTACTGCTTTACTCTTATTACTTTCTCTTCCAGTATTAGCAGGTGCTATCACTATATTATTAACGGATCGAAATTTAAATACATCATTTTTTGATCCTGCAGGAGGAGGAGATCCAATTTTATATCAACATTTATTTTGATTTTTTGGACNTCNNNNAAGTTTAAN
#> 
#> Secondary Basecalls:

Large data

Sometimes with bold_seq() you request a lot of data, which can cause problems due to BOLD's servers.

An example is the taxonomic name Arthropoda. When you send a request like bold_seq(taxon = "Arthropoda") BOLD attempts to give you back sequences for all records under Arthropoda. This, as you can imagine, is a lot of sequences.

library("taxize")

Using taxize::downstream get children of Arthropoda

x <- downstream("Arthropoda", db = "ncbi", downto = "class")
nms <- x$Arthropoda$childtaxa_name

Optionally, check that the name exists in BOLD's data. Any that are not in BOLD will give back a row of NAs

checks <- bold_tax_name(nms)
# all is good
checks[,1:5]
#>     taxid         taxon tax_rank tax_division parentid
#> 1   26059   Pycnogonida    class      Animals       20
#> 2      63     Arachnida    class      Animals       20
#> 3      74   Merostomata    class      Animals       20
#> 4  493944     Pauropoda    class      Animals       20
#> 5   80390      Symphyla    class      Animals       20
#> 6      85     Diplopoda    class      Animals       20
#> 7      75     Chilopoda    class      Animals       20
#> 8      82       Insecta    class      Animals       20
#> 9     372    Collembola    class      Animals       20
#> 10 734357       Protura    class      Animals       20
#> 11     84     Remipedia    class      Animals       20
#> 12     73 Cephalocarida    class      Animals       20
#> 13     68  Branchiopoda    class      Animals       20
#> 14 765970   Hexanauplia    class      Animals       20
#> 15     69  Malacostraca    class      Animals       20
#> 16 889450 Ichthyostraca    class      Animals       20
#> 17     80     Ostracoda    class      Animals       20

Then pass those names to bold_seq(). You could pass all names in at once, but we're trying to avoid the large data request problem here, so run each one separately with lapply or a for loop like request.

out <- lapply(nms, bold_seq)

Citation

Get citation information for bold in R by running: citation(package = 'bold')

Meta

  • Please report any issues or bugs
  • License: MIT
  • Get citation information for bold in R doing citation(package = 'bold')
  • Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.

ropensci_footer

News

bold 0.8.0

MINOR IMPROVEMENTS

  • link to taxize bookdown book in readme and vignette (#51)
  • bold_identify_parents() gains many new parameters (taxid, taxon, tax_rank, tax_division, parentid, parentname, taxonrep, specimenrecords) to filter parents based on any of a number of fields - should solve problem where multiple parents found for a single taxon, often in different kingdoms (#50)
  • add note in docs of bold_identify() that the function uses lapply internally, so queries with lots of sequences can take a long time

BUG FIXES

  • fix bold_specimens(): use rawToChar() on raw bytes instead of parse() from crul (#47)

bold 0.5.0

NEW FEATURES

  • Now using BOLD's v4 API throughout the package. This was essentially just a change of the BASE URL for each request (#30)
  • Now using crul for HTTP requests. Only really affects users in that specifying curl options works slightly differenlty (#42)

BUG FIXES

  • marker parameter in bold_seqspec was and maybe still is not working, in the sense that using the parameter doesn't always limit results to the marker you specify. Not really fixed - watch out for it, and filter after you get results back to get markers you want. (#25)
  • Fixed bug in bold_identify_parents - was failing when no match for a parent name. (#41) thx @VascoElbrecht
  • tsv results were erroring in bold_specimens and other fxns (#46) - fixed by switching to new BOLD v4 API (#30)

MINOR IMPROVEMENTS

  • Namespace calls to base pkgs for stats and utils - replaced is with inherits (#39)

bold 0.4.0

NEW FEATURES

  • New function bold_identify_parents() to add taxonomic information to the output of bold_identif(). We take the taxon names from bold_identify output, and use bold_tax_name to get the taxonomic ID, passing it to bold_tax_id to get the parent names, then attaches those to the input data. There are two options given what you put for the wide parameter. If TRUE you get data.frames of the same dimensions with parent rank name and ID as new columns (for each name going up the hierarchy) - while if FALSE you get a long data.frame. thanks @dougwyu for inspiring this (#36)

MINOR IMPROVEMENTS

  • replace xml2::xml_find_one with xml2::xml_find_first (#33)
  • Fix description of db options in bold_identify man file - COX1 and COX1_SPECIES were switched (#37) thanks for pointing that out @dougwyu

BUG FIXES

  • Fix to bold_tax_id for when some elements returned from the BOLD API were empty/NULL (#32) thanks @fmichonneau !!

bold 0.3.5

MINOR IMPROVEMENTS

  • Added more tests to the test suite (#28)

BUG FIXES

  • Fixed a bug in an internal data parser (#27)

bold 0.3.4

NEW FEATURES

  • Added a code of conduct

MINOR IMPROVEMENTS

  • Switched to xml2 from XML as the XML parser for this package (#26)
  • Fixes to bold_trace() to create dir and tar file when it doesn't already exist

BUG FIXES

  • Fixed odd problem where sometimes resulting data from HTTP request was garbled on content(x, "text"), so now using rawToChar(content(x)), which works (#24)

bold 0.3.0

MINOR IMPROVEMENTS

  • Explicitly import non-base R functions (#22)
  • Better package level manual file

bold 0.2.6

MINOR IMPROVEMENTS

  • sangerseqR package now in Suggests for reading trace files, and is only used in bold_trace() function.
  • General code tidying, reduction of code duplication.
  • bold_trace() gains two new parameters: overwrite to choose whether to overwrite an existing file of the same name or not, progress to show a progress bar for downloading or not.
  • bold_trace() gains a print method to show a tidy summary of the trace file downloaded.

BUG FIXES

  • Fixed similar bugs in bold_tax_name() (#17) and bold_tax_id() (#18) in which species that were missing from the BOLD database returned empty arrays but 200 status codes. Parsing those as failed attempts now. Also fixes problem in taxize in bold_search() that use these two functions.

bold 0.2.0

NEW FEATURES

  • Package gains two new functions for working with the BOLD taxonomy APIs: bold_tax_name() and bold_tax_id(), which search for taxonomic data from BOLD using either names or BOLD identifiers, respectively. (#11)
  • Two new packages in Imports: jsonlite and reshape.

MINOR IMPROVEMENTS

  • Added new taxonomy API functions to the vignette (#14)
  • Added reference URLS to all function doc files to allow easy reference for the appropriate API docs.
  • callopts parameter changed to ... throughout the package, so that passing on options to httr::GET is done via named parameters, e.g., config=verbose(). (#13)
  • Added examples of doing curl debugging throughout man pages.

bold 0.1.2

MINOR IMPROVEMENTS

  • Improved the vignette (#8)
  • Added small function to print helpful message when user inputs no parameters or zero length parameter values.

BUG FIXES

  • Fixed some broken tests with the new httr (v0.4) (#9), and added a few more tests (#7)

bold 0.1.0

NEW FEATURES

  • released to CRAN

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("bold")

0.8.6 by Scott Chamberlain, 2 months ago


https://github.com/ropensci/bold


Report a bug at https://github.com/ropensci/bold/issues


Browse source code at https://github.com/cran/bold


Authors: Scott Chamberlain [aut, cre]


Documentation:   PDF Manual  


MIT + file LICENSE license


Imports xml2, crul, stringr, jsonlite, reshape, plyr, data.table, tibble

Suggests roxygen2, sangerseqR, knitr, testthat, vcr


Imported by funbarRF, taxize.


See at CRAN