usdarnass provides an alternative for downloading various USDA data from https://quickstats.nass.usda.gov/ through R. You must sign up for an API key from the mentioned website in order for this package to work.
The USDA’s documentation on Quick Stats can be found throughout https://www.nass.usda.gov/Quick_Stats/index.php. A short description of what the data entail can be summarised from the Quick Stats description on data.gov:
Quick Stats is the National Agricultural Statistics Service’s (NASS) online, self-service tool to access complete results from the 1997, 2002, 2007, and 2012 Censuses of Agriculture as well as the best source of NASS survey published estimates. The census collects data on all commodities produced on U.S. farms and ranches, as well as detailed information on expenses, income, and operator characteristics. The surveys that NASS conducts collect information on virtually every facet of U.S. agricultural production.
There are two main USDA sources within Quick Stats: censuses and surveys. The census values in Quick Stats start in 1997 while the survey values can range all the way back to 1850 and then annually since 1866. Although Agricultural Censuses occur once every 5 years (1997, 2002, 2007, 2012, and eventually 2017), USDA will administer other censuses which explains the extra years available. At this time, there is no support for documenting the various kinds of data which can be extracted from Quick Stats and this package only serves as an R interface for downloading Quick Stats data and it is up to the user to understand the source of the data they download.
Development version (needs devtools installed):
Load package:
If a query works on the https://quickstats.nass.usda.gov/ interface, then it will work with the usdarnass package. Keep in mind that there is a 50,000 observation limit for both the web interface and data queries with this package.
There are three main functions for this package with the first as the workhorse:
nass_data() this will return a data.frame to the specifications of the query from all of the arguments set in the function call. This mimics the simple “GET DATA” command off of https://quickstats.nass.usda.gov/ and requires an API key. There is a 50,000 limit for each call.nass_param() returns all of the possible values for a parameter in a query. Helpful to understand how to subset a query if it runs into the 50,000 limit.nass_count() returns the number of records for a query. Very useful in conjunction with nass_param() to determine what queries can return data with a nass_data() call.All of these functions require an API Key for each query, which can be set with another function.
nass_set_key("YOUR_KEY_IN_QUOTATIONS")
# First time, reload your enviornment so you can use the key without restarting R.
readRenviron("~/.Renviron")
# You can check it with:
Sys.getenv("NASS_KEY")The above script will add a line to your .Renviron file to be re-used whenever you are using the package. If you are not comfortable with that, you can add the following line to your .Renviron file manually to produce the same result.
NASS_KEY = 'YOUR_KEY_IN_QUOTATIONS'
If you are not comfortable with either of these options, then to use the package you need to ensure that the token parameter is set to your API key in each of your function calls that queries USDA Quick Stats.
As previously stated, the workhorse function is the nass_data() function which will make query calls and return a data.frame as long as the query will return 50,000 or fewer observations. By way of example, we can put in a query to Quick Stats for the value of agricultural land (and buildings) in Wake County North Carolina for 2012:
nass_data(year = 2012,
short_desc = "AG LAND, INCL BUILDINGS - ASSET VALUE, MEASURED IN $",
county_name = "WAKE",
state_name = "NORTH CAROLINA")
#> source_desc sector_desc group_desc commodity_desc
#> 1 CENSUS ECONOMICS FARMS & LAND & ASSETS AG LAND
#> class_desc prodn_practice_desc util_practice_desc
#> 1 INCL BUILDINGS ALL PRODUCTION PRACTICES ALL UTILIZATION PRACTICES
#> statisticcat_desc unit_desc
#> 1 ASSET VALUE $
#> short_desc domain_desc
#> 1 AG LAND, INCL BUILDINGS - ASSET VALUE, MEASURED IN $ TOTAL
#> domaincat_desc agg_level_desc state_ansi state_fips_code state_alpha
#> 1 NOT SPECIFIED COUNTY 37 37 NC
#> state_name asd_code asd_desc county_ansi county_code
#> 1 NORTH CAROLINA 50 CENTRAL PIEDMONT 183 183
#> county_name region_desc zip_5 watershed_code watershed_desc
#> 1 WAKE <NA> <NA> 00000000 <NA>
#> congr_district_code country_code country_name
#> 1 <NA> 9000 UNITED STATES
#> location_desc year freq_desc begin_code
#> 1 NORTH CAROLINA, CENTRAL PIEDMONT, WAKE 2012 POINT IN TIME 12
#> end_code reference_period_desc week_ending load_time
#> 1 12 END OF DEC <NA> 2012-12-31 00:00:00
#> Value CV (%)
#> 1 806,464,000 7.9The output of this query has a lot to digest, but the main focus is on the Value variable in the resulting data.frame. Please note that the returned Value for the query is of the class character. The parameter numeric_vals can be set to TRUE to have this return a numeric value, however the default is to be a character type as there are some suppressed values which will be coerced to NA when numeric_vals is set to TRUE.
Aside from the output, the particular query used four parameters for its output: year, short_desc, county_name, and state_name. Each of these parameters have a particular set of values which can generate of query, which can be figured out using the nass_param() function. In querying the Quick Stats API, you usually do not need to subset many parameters to get to the 50,000 limitation but if you do not get within the 50,000 limitation the call will produce an error.
There are 18 parameters for each query, although most of those will be NULL values and not required to specify an output.
There are 18 parameters which can be included in each query to Quick Stats however some of them are concatenations of other parameter values. If there is a name in quotations following a particular parameter, then that is the drop down menu via the Quick Stats web interface that gives you a value. Not all parameters will have a drop down menu though:
source_desc “Program” - Source of data (“CENSUS” or “SURVEY”). Census program includes the Census of Ag as well as follow up projects. Survey program includes national, state, and county surveys.sector_desc “Sector” - Five high level, broad categories useful to narrow down choices. (“ANIMALS & PRODUCTS”, “CROPS”, “DEMOGRAPHICS”, “ECONOMICS”, or “ENVIRONMENTAL”)group_desc “Group” - Subsets within sector (e.g., under sector_desc = “CROPS”, the groups are “FIELD CROPS”, “FRUIT & TREE NUTS”, “HORTICULTURE”, and “VEGETABLES”).commodity_desc “Commodity” - The primary subject of interest (e.g., “CORN”, “CATTLE”, “LABOR”, “TRACTORS”, “OPERATORS”).short_desc “Data Item” - A concatenation of six columns: commodity_desc, class_desc, prodn_practice_desc, util_practice_desc, statisticcat_desc, and unit_desc.domain_desc “Domain” - Generally another characteristic of operations that produce a particular commodity (e.g., “ECONOMIC CLASS”, “AREA OPERATED”, “NAICS CLASSIFICATION”, “SALES”). For chemical usage data, the domain describes the type of chemical applied to the commodity. The domain_desc = “TOTAL” will have no further breakouts; i.e., the data value pertains completely to the short_desc.domaincat_desc “Domain Category” - Categories or partitions within a domain (e.g., under domain_desc = “SALES”, domain categories include $1,000 TO $9,999, $10,000 TO $19,999, etc).agg_level_desc “Geographic Level” - Aggregation level or geographic granularity of the data. (“AGRICULTURAL DISTRICT”, “COUNTY”, “INTERNATIONAL”, “NATIONAL”, “REGION : MULTI-STATE”, “REGION : SUB-STATE”, “STATE”, “WATERSHED”, or “ZIP CODE”)statisticcat_desc “Category” - The aspect of a commodity being measured (e.g., “AREA HARVESTED”, “PRICE RECEIVED”, “INVENTORY”, “SALES”).state_name “State” - State full name.asd_desc “Ag District” - Ag statistics district name.county_name “County” - County name.region_desc “Region” - NASS defined geographic entities not readily defined by other standard geographic levels. A region can be a less than a state (SUB-STATE) or a group of states (MULTI-STATE), and may be specific to a commodity.zip_5 “Zip Code” - US Postal Service 5-digit zip code.watershed_desc “Watershed” - Name assigned to the HUC.year “Year” - The numeric year of the data and can be either a character or numeric vector. Conditional values are also possible, for example a character vector of “>=1999” of “1999<=” will give years greater than or equal to 1999. Right now the logical values can either be greater/less than or equal to with the logical at either the beginning or end of a string with the year.freq_desc “Period Type” - Length of time covered (“ANNUAL”, “SEASON”, “MONTHLY”, “WEEKLY”, “POINT IN TIME”). “MONTHLY” often covers more than one month. “POINT IN TIME” is as of a particular day.reference_period_desc “Period” - The specific time frame, within a freq_desc.The descriptions of parameters here are mostly thought of as minimal because much of the actual parameters have many values. This is where the nass_param() function comes into play in order to give the full set of values for each parameter.
The nass_param() function will return a vector of all the possible values for a parameter conditional on the other parameter subsets given above. By way of example, we can see that there are only two sources of datasets for the Quick Stats queries by asking what the values for the source_desc parameter are:
The first argument in nass_param() is the parameter of interest, which can take on any of the 18 values from the parameters section. This argument must be passed in a character format, so make sure to use quotations in your calls. The characters are also not case sensitive in the calls. The year parameter is the only parameter that does not need to be a character vector but can be numeric instead.
This function is most helpful in determining what variables are available for a certain subset. For example, if I were interested in what county level variables in Ohio are available in 2000 I might start by determining what “Group” is available at that level:
nass_param("group_desc",
state_name = "OHIO",
agg_level_desc = "COUNTY",
year = 2000)
#> [1] "DAIRY" "FARMS & LAND & ASSETS" "FIELD CROPS"
#> [4] "LIVESTOCK" "VEGETABLES"Now, if I want to further figure out what commodities are available for the “DAIRY” subset of this data but only after 2000, I would make a call of :
nass_param("commodity_desc",
group_desc = "dairy",
state_name = "OHIO",
agg_level_desc = "COUNTY",
year = ">2000")
#> [1] "MILK"While the year parameter does not need to be a character vector, it does accept relational operators which can modify the subsets even further for queries.
Every query with nass_data() has a 50,000 limit of observations. In order to determine the number of observations in a query, the nass_count() function will accept all the same parameters as nass_data() but its output is a numeric of the observations in a query:
Here we see how many observations are currently in the Quick Stats as of 2019-06-20. Clearly, the entire data can not be downloaded with a query of their API. If you are concerned about downloading all of the Quick Stats data, then it would be more efficient to use their ftp site ftp://ftp.nass.usda.gov/quickstats/.
By way of another example, we can look at how many observations are available related to agricultural land at the county level:
This particular query would not be able to be run for nass_data() because the number of observations greatly exceeds 50,000 and indeed that query returns an error:
nass_data(commodity_desc = "AG LAND",
agg_level_desc = "COUNTY")
#> Error in nass_data(commodity_desc = "AG LAND", agg_level_desc = "COUNTY"): Query returns 2,053,425 records. The limit is 50,000. Subset the query to fit within limit. See nass_count()At this point in time a bit of understanding of the data and the user’s goals are needed. If there is only one state of interest for the study, then subsetting the data further to a state is likely the best strategy. However, it is more likely that the user wants the all county level data related to agricultural land. My strategy would be to look at the number of observations for each year of interest I might have:
years <- 2000:2017
sapply(years, function(x) nass_count(year = x,
commodity_desc = "AG LAND",
agg_level_desc = "COUNTY"))
#> [1] 0 0 418908 0 0 0 0 436633 0 0
#> [11] 0 0 422050 0 0 0 0 417663The observations here are only related to 2002, 2007, and 2012 which are agricultural census years and it is highly likely that there are a lot of variables in the category what would likely not be useful. It is then best to look at the descriptions of the variables to figure out what data would be most useful:
agland_params <- nass_param("short_desc",
commodity_desc = "AG LAND",
agg_level_desc = "COUNTY")
agland_params
#> [1] "AG LAND - ACRES"
#> [2] "AG LAND - NUMBER OF OPERATIONS"
#> [3] "AG LAND - OPERATIONS WITH TREATED"
#> [4] "AG LAND - TREATED, MEASURED IN ACRES"
#> [5] "AG LAND, (EXCL CROPLAND & PASTURELAND & WOODLAND) - ACRES"
#> [6] "AG LAND, (EXCL CROPLAND & PASTURELAND & WOODLAND) - NUMBER OF OPERATIONS"
#> [7] "AG LAND, (EXCL HARVESTED CROPLAND), IRRIGATED - ACRES"
#> [8] "AG LAND, (EXCL HARVESTED CROPLAND), IRRIGATED - NUMBER OF OPERATIONS"
#> [9] "AG LAND, CROP INSURANCE - ACRES"
#> [10] "AG LAND, CROP INSURANCE - NUMBER OF OPERATIONS"
#> [11] "AG LAND, CROPLAND - ACRES"
#> [12] "AG LAND, CROPLAND - NUMBER OF OPERATIONS"
#> [13] "AG LAND, CROPLAND, (EXCL HARVESTED & PASTURED) - ACRES"
#> [14] "AG LAND, CROPLAND, (EXCL HARVESTED & PASTURED) - NUMBER OF OPERATIONS"
#> [15] "AG LAND, CROPLAND, (EXCL HARVESTED & PASTURED), ALL CROPS FAILED - ACRES"
#> [16] "AG LAND, CROPLAND, (EXCL HARVESTED & PASTURED), ALL CROPS FAILED - NUMBER OF OPERATIONS"
#> [17] "AG LAND, CROPLAND, (EXCL HARVESTED & PASTURED), CULTIVATED SUMMER FALLOW - ACRES"
#> [18] "AG LAND, CROPLAND, (EXCL HARVESTED & PASTURED), CULTIVATED SUMMER FALLOW - NUMBER OF OPERATIONS"
#> [19] "AG LAND, CROPLAND, (EXCL HARVESTED & PASTURED), IDLE - ACRES"
#> [20] "AG LAND, CROPLAND, (EXCL HARVESTED & PASTURED), IDLE - NUMBER OF OPERATIONS"
#> [21] "AG LAND, CROPLAND, (EXCL PASTURED) - OPERATIONS WITH TREATED"
#> [22] "AG LAND, CROPLAND, (EXCL PASTURED) - TREATED, MEASURED IN ACRES"
#> [23] "AG LAND, CROPLAND, HARVESTED - ACRES"
#> [24] "AG LAND, CROPLAND, HARVESTED - NUMBER OF OPERATIONS"
#> [25] "AG LAND, CROPLAND, HARVESTED, IRRIGATED - ACRES"
#> [26] "AG LAND, CROPLAND, HARVESTED, IRRIGATED - NUMBER OF OPERATIONS"
#> [27] "AG LAND, CROPLAND, HARVESTED, ORGANIC - ACRES"
#> [28] "AG LAND, CROPLAND, HARVESTED, ORGANIC - NUMBER OF OPERATIONS"
#> [29] "AG LAND, CROPLAND, ORGANIC - ACRES"
#> [30] "AG LAND, CROPLAND, ORGANIC - NUMBER OF OPERATIONS"
#> [31] "AG LAND, CROPLAND, PASTURED ONLY - ACRES"
#> [32] "AG LAND, CROPLAND, PASTURED ONLY - NUMBER OF OPERATIONS"
#> [33] "AG LAND, INCL BUILDINGS - ASSET VALUE, MEASURED IN $"
#> [34] "AG LAND, INCL BUILDINGS - ASSET VALUE, MEASURED IN $ / ACRE"
#> [35] "AG LAND, INCL BUILDINGS - ASSET VALUE, MEASURED IN $ / OPERATION"
#> [36] "AG LAND, INCL BUILDINGS - OPERATIONS WITH ASSET VALUE"
#> [37] "AG LAND, IRRIGATED - ACRES"
#> [38] "AG LAND, IRRIGATED - NUMBER OF OPERATIONS"
#> [39] "AG LAND, ORGANIC - ACRES"
#> [40] "AG LAND, ORGANIC - NUMBER OF OPERATIONS"
#> [41] "AG LAND, ORGANIC, TRANSITIONING - ACRES"
#> [42] "AG LAND, ORGANIC, TRANSITIONING - NUMBER OF OPERATIONS"
#> [43] "AG LAND, OWNED, IN FARMS - ACRES"
#> [44] "AG LAND, PASTURELAND - ACRES"
#> [45] "AG LAND, PASTURELAND - NUMBER OF OPERATIONS"
#> [46] "AG LAND, PASTURELAND - OPERATIONS WITH TREATED"
#> [47] "AG LAND, PASTURELAND - TREATED, MEASURED IN ACRES"
#> [48] "AG LAND, PASTURELAND, (EXCL CROPLAND & WOODLAND) - ACRES"
#> [49] "AG LAND, PASTURELAND, (EXCL CROPLAND & WOODLAND) - NUMBER OF OPERATIONS"
#> [50] "AG LAND, PASTURELAND, (EXCL WOODLAND) - ACRES"
#> [51] "AG LAND, PASTURELAND, (EXCL WOODLAND) - NUMBER OF OPERATIONS"
#> [52] "AG LAND, PASTURELAND, ORGANIC - ACRES"
#> [53] "AG LAND, PASTURELAND, ORGANIC - NUMBER OF OPERATIONS"
#> [54] "AG LAND, RENTED FROM OTHERS, IN FARMS - ACRES"
#> [55] "AG LAND, WOODLAND - ACRES"
#> [56] "AG LAND, WOODLAND - NUMBER OF OPERATIONS"
#> [57] "AG LAND, WOODLAND, (EXCL PASTURED) - ACRES"
#> [58] "AG LAND, WOODLAND, (EXCL PASTURED) - NUMBER OF OPERATIONS"
#> [59] "AG LAND, WOODLAND, PASTURED - ACRES"
#> [60] "AG LAND, WOODLAND, PASTURED - NUMBER OF OPERATIONS"We can use the output for the parameter values to see the number of observations within each of these categories:
sapply(agland_params, function(x) nass_count(short_desc = x,
commodity_desc = "AG LAND",
agg_level_desc = "COUNTY"))
#> AG LAND - ACRES
#> 15199
#> AG LAND - NUMBER OF OPERATIONS
#> 15199
#> AG LAND - OPERATIONS WITH TREATED
#> 98699
#> AG LAND - TREATED, MEASURED IN ACRES
#> 98699
#> AG LAND, (EXCL CROPLAND & PASTURELAND & WOODLAND) - ACRES
#> 15370
#> AG LAND, (EXCL CROPLAND & PASTURELAND & WOODLAND) - NUMBER OF OPERATIONS
#> 15370
#> AG LAND, (EXCL HARVESTED CROPLAND), IRRIGATED - ACRES
#> 10250
#> AG LAND, (EXCL HARVESTED CROPLAND), IRRIGATED - NUMBER OF OPERATIONS
#> 10250
#> AG LAND, CROP INSURANCE - ACRES
#> 11884
#> AG LAND, CROP INSURANCE - NUMBER OF OPERATIONS
#> 11884
#> AG LAND, CROPLAND - ACRES
#> 15374
#> AG LAND, CROPLAND - NUMBER OF OPERATIONS
#> 15374
#> AG LAND, CROPLAND, (EXCL HARVESTED & PASTURED) - ACRES
#> 29422
#> AG LAND, CROPLAND, (EXCL HARVESTED & PASTURED) - NUMBER OF OPERATIONS
#> 29422
#> AG LAND, CROPLAND, (EXCL HARVESTED & PASTURED), ALL CROPS FAILED - ACRES
#> 14903
#> AG LAND, CROPLAND, (EXCL HARVESTED & PASTURED), ALL CROPS FAILED - NUMBER OF OPERATIONS
#> 14903
#> AG LAND, CROPLAND, (EXCL HARVESTED & PASTURED), CULTIVATED SUMMER FALLOW - ACRES
#> 13778
#> AG LAND, CROPLAND, (EXCL HARVESTED & PASTURED), CULTIVATED SUMMER FALLOW - NUMBER OF OPERATIONS
#> 13778
#> AG LAND, CROPLAND, (EXCL HARVESTED & PASTURED), IDLE - ACRES
#> 12224
#> AG LAND, CROPLAND, (EXCL HARVESTED & PASTURED), IDLE - NUMBER OF OPERATIONS
#> 12224
#> AG LAND, CROPLAND, (EXCL PASTURED) - OPERATIONS WITH TREATED
#> 15326
#> AG LAND, CROPLAND, (EXCL PASTURED) - TREATED, MEASURED IN ACRES
#> 15326
#> AG LAND, CROPLAND, HARVESTED - ACRES
#> 410798
#> AG LAND, CROPLAND, HARVESTED - NUMBER OF OPERATIONS
#> 388699
#> AG LAND, CROPLAND, HARVESTED, IRRIGATED - ACRES
#> 15170
#> AG LAND, CROPLAND, HARVESTED, IRRIGATED - NUMBER OF OPERATIONS
#> 15170
#> AG LAND, CROPLAND, HARVESTED, ORGANIC - ACRES
#> 1895
#> AG LAND, CROPLAND, HARVESTED, ORGANIC - NUMBER OF OPERATIONS
#> 1895
#> AG LAND, CROPLAND, ORGANIC - ACRES
#> 1467
#> AG LAND, CROPLAND, ORGANIC - NUMBER OF OPERATIONS
#> 1467
#> AG LAND, CROPLAND, PASTURED ONLY - ACRES
#> 15240
#> AG LAND, CROPLAND, PASTURED ONLY - NUMBER OF OPERATIONS
#> 15240
#> AG LAND, INCL BUILDINGS - ASSET VALUE, MEASURED IN $
#> 15382
#> AG LAND, INCL BUILDINGS - ASSET VALUE, MEASURED IN $ / ACRE
#> 15382
#> AG LAND, INCL BUILDINGS - ASSET VALUE, MEASURED IN $ / OPERATION
#> 15382
#> AG LAND, INCL BUILDINGS - OPERATIONS WITH ASSET VALUE
#> 119569
#> AG LAND, IRRIGATED - ACRES
#> 131817
#> AG LAND, IRRIGATED - NUMBER OF OPERATIONS
#> 128786
#> AG LAND, ORGANIC - ACRES
#> 2122
#> AG LAND, ORGANIC - NUMBER OF OPERATIONS
#> 2122
#> AG LAND, ORGANIC, TRANSITIONING - ACRES
#> 2084
#> AG LAND, ORGANIC, TRANSITIONING - NUMBER OF OPERATIONS
#> 4048
#> AG LAND, OWNED, IN FARMS - ACRES
#> 15300
#> AG LAND, PASTURELAND - ACRES
#> 15350
#> AG LAND, PASTURELAND - NUMBER OF OPERATIONS
#> 15350
#> AG LAND, PASTURELAND - OPERATIONS WITH TREATED
#> 15040
#> AG LAND, PASTURELAND - TREATED, MEASURED IN ACRES
#> 15040
#> AG LAND, PASTURELAND, (EXCL CROPLAND & WOODLAND) - ACRES
#> 15335
#> AG LAND, PASTURELAND, (EXCL CROPLAND & WOODLAND) - NUMBER OF OPERATIONS
#> 15335
#> AG LAND, PASTURELAND, (EXCL WOODLAND) - ACRES
#> 14515
#> AG LAND, PASTURELAND, (EXCL WOODLAND) - NUMBER OF OPERATIONS
#> 14515
#> AG LAND, PASTURELAND, ORGANIC - ACRES
#> 1459
#> AG LAND, PASTURELAND, ORGANIC - NUMBER OF OPERATIONS
#> 1459
#> AG LAND, RENTED FROM OTHERS, IN FARMS - ACRES
#> 15300
#> AG LAND, WOODLAND - ACRES
#> 15255
#> AG LAND, WOODLAND - NUMBER OF OPERATIONS
#> 15255
#> AG LAND, WOODLAND, (EXCL PASTURED) - ACRES
#> 15138
#> AG LAND, WOODLAND, (EXCL PASTURED) - NUMBER OF OPERATIONS
#> 15138
#> AG LAND, WOODLAND, PASTURED - ACRES
#> 15024
#> AG LAND, WOODLAND, PASTURED - NUMBER OF OPERATIONS
#> 15024While most of these data items fit within the 50,000 limit, not all do. Take for example the “AG LAND - TREATED, MEASURED IN ACRES” category exceeds the limit and would not be able to be downloaded. This is because the treated category actually has multiple domains which can be seen by combining the nass_param() and nass_count():
agland_domain <- nass_param("domain_desc",
short_desc = "AG LAND - TREATED, MEASURED IN ACRES",
commodity_desc = "AG LAND",
agg_level_desc = "COUNTY")
sapply(agland_domain, function(x) nass_count(domain_desc = x,
short_desc = "AG LAND - TREATED, MEASURED IN ACRES",
commodity_desc = "AG LAND",
agg_level_desc = "COUNTY"))
#> CHEMICAL, FUNGICIDE CHEMICAL, HERBICIDE CHEMICAL, INSECTICIDE
#> 13998 15296 27098
#> CHEMICAL, OTHER FERTILIZER
#> 12092 30215