OPEN DATA RESOURCES
OPEN DATA RESOURCES
From government entities to tech companies, organizations are sharing data on economic indicators with increased transparency amidst pandemic-induced uncertainty and rapid change. Operators of small and mid-sized businesses may find these data sets useful in thinking about the short- and mid-term outlook for their customer, industry, and geographic region.
Below we will continually update sources of information considered relevant to operators, including how the data is collected, how often it is being updated, and the source. As the data compiles, we may also highlight trends and/or comparisons in short briefs.
Please share freely. And if you know of a useful open data source we should add, please send it our way.
CONSUMER ACTIVITY
Tracked Data:
Consumer Foot Traffic by Industry
Foot Traffic at Top Brands (McDonalds, Starbucks, Target, Walmart, Home Depot, Costco)
Foot Traffic at Commercial Locations by Region
Foot Traffic at Restaurants by Cuisine Type
Reporting Frequency: Weekly
How Data Is Collected: Based on “a panel of opt-in, anonymized smartphone devices, and is well balanced across USA demographics and geographies,” at 6 million points of interest across the U.S.
Source: SafeGraph
Tracked Data: By brand and state
Change in foot traffic year over year
Change in number of venues year over year
Total number of COVID-19 cases by geographic region
Reporting Frequency: Daily
How Data Is Collected: Based on data collected by Placer.ai from over 20 million devices.
Source: Placer.ai
Tracked Data: By Country
Driving
Walking
Transit
Reporting Frequency: Daily
How Data Is Collected: Based on “counting the number of requests made to Apple Maps for directions in select countries/regions, sub-regions, and cities.”
Source: Apple
Tracked Data:
Congestion in Top 8 Cities compared to 2019
Daily Commuter Traffic Trends
Morning Rush Hour Traffic Trends
Evening Rush Hour Traffic Trends
Reporting Frequency: Weekly
How Data Is Collected: Based on TomTom data.
Source: Deloitte
Tracked Data:
Card Spending By Category
Card Spending By Generation
Card Spending By Card Presence
Card Spending By State
Card Spending By Stay-At-Home Order Status
Reporting Frequency: Daily, Tuesday through Friday
How Data Is Collected: Based on Chase consumer credit card data.
Source: JPMorganChase
Tracked Data:
Daily Spend Compared To 2019
Spend By Industry
Spend At Amazon & Walmart
Reporting Frequency: Daily
How Data Is Collected: Based on “raw financial transaction data from legacy technologies.”
Source: Facteus
DAILY AVERAGE REVENUE AT RETAIL
Tracked Data: Revenue by County/State at:
Restaurants & Bars – Daily Average Revenue and YoY Variance
Grocery Stores & Supermarkets – Daily Average Revenue and YoY Variance
Sporting Goods, Guns & Ammo – Daily Average Revenue and YoY Variance
Hotels, Motels & Lodging – Daily Average Revenue and YoY Variance
Pet Stores – Daily Average Revenue and YoY Variance
Reporting Frequency: Daily
How Data Is Collected: Based on credit card processing data as provided in partnership with “the nation’s largest credit card processors.”
Source: Womply
Tracked Data: Daily Global Order Values by Industry (With YoY Comparison):
Apparel & Accessories
Automotive
Electronics
Food & Beverage
Jewelry
Home & Garden
Hardware & Home Improvement
Health & Beauty
Sporting Goods
Toys & Hobbies
Office Supplies
Specialty
Mass Merchant
Reporting Frequency: Daily
How Data Is Collected: Based on “Klaviyo’s global data.”
Source: Klaviyo
Tracked Data: Foot traffic by County/State/Country at
Retail & Recreation (e.g. restaurants, museums, theme parks, shopping centers)
Grocery & Pharmacy (e.g. grocery stores, farmers markets, pharmacies)
Parks (e.g. national parks, public beaches, marinas, public gardens)
Transit Stations (e.g. subway, bus, train stations)
Workplaces
Residential
Reporting Frequency: Last updated May 16, 2020
How Data Is Collected: Based on “users who have opted-in to Location History for their Google Account.”
Source: Google
Tracked Data: Foot traffic by DMA/State at
Airports
Auto Dealerships
Big Box Stores
Casual Dining Restaurants
Grocery Stores
Malls & Department Stores
Movie Theaters
Quick Service Restaurants
Reporting Frequency: Weekly
How Data Is Collected: Based on collected mobility data from “more than 90M U.S. consumers seen in almost 8B locations per day.”
Source: NinthDecimal
Tracked Data:
Monthly Sales for Retail Trade and Food Services
Quarterly Retail eCommerce Sales Report
Reporting Frequency: Monthly/Quarterly
How Data Is Collected: Based on data collected by regulators.
Source: U.S. Census Bureau
Tracked Data: Seated Diners at Restaurants
Reporting Frequency: Daily
How Data Is Collected: Represents data from the approximately 20,000 restaurants on the OpenTable network across all channels: online reservations, phone reservations, and walk-ins. “Only states or cities with 50+ restaurants in the sample are included.”
Source: OpenTable
Tracked Data: YTD and Quarterly Sales by:
Manufacturer
Brand
Model
Reporting Frequency: Monthly/Quarterly
How Data Is Collected: Based on data collected from manufacturers.
Source: Good Car Bad Car
Tracked Data:
Towables
Motorhomes
Park Model RVs
Reporting Frequency: Monthly
How Data Is Collected: Based on data collected from manufacturers.
Source: RV Industry Association
Tracked Data: Sales By:
State
OEM
Date Period
New vs. Used
Reporting Frequency: Daily
How Data Is Collected: Based on data “from 18,712 new car franchise dealer websites nightly.”
Source: Foureyes
Tracked Data: Sales Trends In:
Apparel
Baby
Beauty
Crafts & Games
Electronics
Food
Health
Jewelry
& More
Reporting Frequency: Daily
How Data Is Collected: Based on data from a “subset of ShipHero’s e-commerce customers and includes over 1,000 merchants and over $2 billion in annual orders primarily selling on Shopify and Amazon Seller Fulfilled Prime or Amazon Fulfilled by Merchant.”
Source: Shiphero
Tracked Data: Projects by:
Top 30 Regions
Industry
Top 25 Categories
Reporting Frequency: Data set for March 1 through April 30, 2020. Likely to update again based on the display through May 15th shown here.
How Data Is Collected: Based on booked Thumbtack projects.
Source: Thumbtack
Tracked Data: Year-over-year trends in:
e-Commerce Revenue
Ad Spend
Conversion Rate
Reporting Frequency:
How Data Is Collected: Based on “our clients’ first-party data in U.S. e-commerce. We manage over $500MM in media spend for fast-growing VC-backed startups and Fortune 100 companies. Our partners collectively drive over $5B in e-commerce revenue.”
Source: Within
FOOD
Tracked Data:
Livestock and Domestic Meat
Trade
Production
Slaughter
Cold Storage Stocks
Wholesale Prices
Fruit and Tree Nuts
Retail Prices (selective)
Imports and Exports
Production
Vegetables and Pulses
Shipment Volume
Farm Prices
Retail Prices (selective)
Imports and Exports
Reporting Frequency: Quarterly
How Data Is Collected: Based on data collected by regulators.
Source: USDA
Tracked Data:
Cereals and Bakery Products
Meats, Poultry, Fish, and Eggs
Dairy
Fruits and Vegetables
Nonalcoholic Beverages
Food Away From Home
Reporting Frequency: Monthly
How Data Is Collected: Based on data collected by regulators.
Source: U.S. Bureau of Labor Statistics
TRAVEL
Tracked Data:
Total Flights per Day
Commercial Flights per Day
Reporting Frequency: Daily
How Data Is Collected: Based on air traffic control reporting.
Source: Flightradar24
Tracked Data: Number of U.S. Commercial Airline Passengers Screened
Reporting Frequency: Daily
How Data Is Collected: Represents TSA Checkpoint reporting.
Tracked Data:
Cancellations
U.S. Bookings by Lead Time
Paid Occupancy
Reporting Frequency: Rolling Updates
How Data Is Collected: Based on 170,000 direct listings and “millions of listings from online travel agencies.”
Source: KeyData Dashboard
SHORT-TERM VACATION RENTALS II
Tracked Data:
Cancellations
Supply
Average Rates
Timeline of travel-related changes
Total monthly online visits to Expedia, Booking.com, VRBO, and AirBnb
Reporting Frequency: Rolling Updates
How Data Is Collected: Based on industry data collected by Transparent.
Source: Transparent
Tracked Data: Covering 416 cities across 57 countries:
Live Congestion
Congestion In Past 48 Hours
Congestion in Past 7 Days
Congestion in Past Years
Reporting Frequency: Real-Time
How Data Is Collected: See details here.
Source: TomTom
Tracked Data: Daily Volume Change Compared To Same Day Of Week In Same Month Of Most Recent Year
Reporting Frequency: Daily
How Data Is Collected: Based on data from “either MS2’s Autopolling Extension every day or transmitted by the agencies nearly every day.”
Source: MS2
CONSTRUCTION
Tracked Data:
Design Contracts
Inquiries
Billings
Reporting Frequency: Monthly (next release June 24th)
How Data Is Collected: Based on data “from AIA’s Work-on-the-Boards survey, which has gathered data on shifts in billings from architectural firm leaders for over 20 years.”
Tracked Data:
Private Family Home Starts
Multi-Unit Home Starts
Built-For-Sale Home Starts
Built-For-Rent Home Starts
Reporting Frequency: Monthly/Quarterly
How Data Is Collected: Based on economic data tracked by FRED.
Tracked Data:
Lumber Prices by Type
Timber Prices (U.S. & Canada)
Framing Lumber Prices
Pulp Prices
Paper Prices
Reporting Frequency: Weekly
How Data Is Collected: Based on data collected from industry producers.
Source: Global Wood, Madison’s Lumber Reporter, U.S. Census Bureau
Tracked Data:
Total Employment
Average Hourly Wage & Weekly Hours
Job Openings
Hires & Separations
Reporting Frequency: Monthly
How Data Is Collected: Based on data collected by regulators.
Source: U.S. Bureau of Labor Statistics
HOUSING
Tracked Data: Ratio of Houses for Sale to Houses Sold
Reporting Frequency: Monthly
How Data Is Collected: Based on economic data tracked by FRED.
Homeownership & Rental Vacancies
Tracked Data:
Home Ownership (Region/State/75 Top Metros)
Rental Vacancies (Region/State/75 Top Metros)
Reporting Frequency: Quarterly
How Data Is Collected: Based on U.S. Census Bureau primary surveys and estimating.
Source: U.S. Census Bureau
Tracked Data:
Zillow Home Value Index by State/County/Zip/City/Neighborhood
Zillow Observed Rent Index by State/County/Zip/City/Neighborhood
For Sale Inventory
New For Sale Inventory
Home Sales
Median Sale Price
Median Price Cut
Days Listed
Reporting Frequency: Daily
How Data Is Collected: Based on listings tracked by Zillow, alongside detailed methodology outlined here.
Source: Zillow Research
Also see their recent presentation: The Housing Market Before & After The Coronavirus Crisis.
Tracked Data: By market:
Delistings
New Listing Median Price
Age of Inventory
Days to Close
Pending Sales
Reporting Frequency: Weekly
How Data Is Collected: Based on “direct access to data from local multiple listing services, as well as insight from our real estate agents across the country.”
Source: Redfin
Tracked Data: Weekly Mortgage Applications Index
Reporting Frequency: Weekly (Direct data is only available to subscribers, but basic numbers are released in weekly commentary.)
How Data Is Collected: Based on survey of mortgage bankers, commercial banks and thrifts, covering “over 75 percent of all U.S. retail residential mortgage applications.”
Source: Mortgage Bankers Association
PRODUCTION
MANUFACTUING (PMI)/NON-MANUFACTURING (NMI) REPORTS
Tracked Data: By Sector:
Business Activity Index
New Order Index
Employment Index
Supplier Delivery Index
Inventory
Backlog of Orders
Exports/Imports
Reporting Frequency: Monthly
How Data Is Collected: Based on monthly data collected from the “Business Survey Committee, a group made up of more than 800 purchasing and supply executives from across the country.”
Source: Institute for Supply Management
Tracked Data:
Final Demand Goods
Final Demand Services
Intermediate Demand Goods
Intermediate Demand Services
Select Goods and Services
Reporting Frequency: Monthly
How Data Is Collected: Based on economic data gathered by regulators.
Source: U.S. Bureau of Labor Statistics
WAREHOUSING, LOGISTICS, & FREIGHT ACTIVITY
Tracked Data: By Value and Weight:
Commodity Trade by Port
Commodity Imports and Exports by State
Imports and Exports from/to State by Port
Mode of Transport
Reporting Frequency: Monthly
How Data Is Collected: Based on data collected by regulators.
Tracked Data: Share of value and tonnage by
Airport
Seaport
Reporting Frequency: Monthly
How Data Is Collected: Based on data from the U.S. Census Bureau.
Source: WorldCity
Tracked Data:
Fuel Prices
Petroleum Production
Petroleum Imports
Reporting Frequency: Weekly
How Is Data Collected: Based on industry data collected by regulators.
Tracked Data: Aggregates news stories and disclosures on:
Factory Output/Production
Trucking/Rail
Ocean & Ports
Airlines
PPE Relief Efforts
Reporting Frequency: Weekly
How Is Data Collected: Based on “a variety of sources based on the information we have to date.”
Source: FlexPort
Tracked Data:
Carloads
Intermodal Units
Carload Categories (e.g. Coal, Grain, Cars, Metals)
Reporting Frequency: Weekly
How Is Data Collected: Based on data collected from “the major freight railroads of the U.S., Canada and Mexico, as well as Amtrak.”
ONLINE ADVERTISING
Tracked Data: Online Ad Rates Per Day Compared To Highest Recorded Day
Reporting Frequency: Daily
How Data Is Collected: Based on “strategically sampling across our dataset of online ad impressions and consolidates the data into a single number on a 1 to 100 scale.”
Source: ezoic
CONSUMER DIGITAL MARKETING PERFORMANCE
Tracked Data: All in comparison to Baseline (pre-COVID-19):
Daily Spend
Cost Per Click (CPC)
Cost Per 1,000 Impressions (CPM)
Click-through Rate (CTR)
Reporting Frequency: Daily (lag)
How Data Is Collected: Based on “compiled data from 20 accounts at a ~$50M annual spend rate.”
Source: Common Thread Collective
JOBS
Tracked Data: For hourly employees (“primarily consist of restaurant, food & beverage, retail and services”):
Employees Working
Hours Worked
Locations Opened
Reporting Frequency: Daily
How Data Is Collected: Based on “Homebase data for over 60,000 businesses and 1 million hourly employees active in the US” in comparison to data collected January 4-31, 2020.
Source: Homebase
Tracked Data: Job postings by (and in comparison to 2019 averages):
Geographic Region
Industry
Title
Skills
Reporting Frequency: Weekly
How Data Is Collected: Based on tracked unique postings during the designated time period.
Source: EMSI
HISTORICAL UNEMPLOYMENT FILINGS
Tracked Data:
Initial Claims by Month
Initial Claims Average by Year
Claims by State by Month
Daily Claims by Year by State
Rates by County
Rates by Age Range
Reporting Frequency: Last Updated March 2020
How Data Is Collected: Based on unemployment data.
Source: Recovery Decision Science
Tracked Data:
Employment
Unemployment Claims
College Enrollment
Reporting Frequency: Weekly
How Data Is Collected: Based on economic data gathered by regulators.
Source: U.S. Bureau of Labor Statistics
GENERAL
Tracked Data: Frequency of search inquiries over time.
Reporting Frequency: Real-time
How Data Is Collected: Based on Google search inquiries.
Source: Google
Tracked Data: Volume and Price by Country/Region of
Trade
Imports
Exports
Reporting Frequency: Monthly (lag)
How Data Is Collected: Based on data collected and published by the European Commission (details here).
Tracked Data: By National Average, State or NAICS Industry Code:
Change in Revenues
Supply Chain Disruption
Change in Staffing
Temporary Closings
Missed Loan Payments
Cash on Hand
Reporting Frequency: Weekly (reported on Thursdays, through July 2nd, 2020)
How Data Is Collected: Based on surveys of businesses that as of 2017 had 1 to 499 employees (details here).
Source: U.S. Census Bureau
Tracked Data: 2020 Closed Transactions Compared to 2019 Closed Transactions By Industry
Reporting Frequency: Weekly
How Data Is Collected: Based on data from S&P Capital IQ (private source).
Source: PCE
COVID-19 HEALTH
Tracked Data: COVID-19 Tests Administered
Reporting Frequency: Daily
How Data Is Collected: Based on testing data from The Covid Tracking Project and U.S. Census Data.
Source: Sam Priddy
Tracked Data:
Cases
Deaths
Recoveries
Reporting Frequency: Daily
How Data Is Collected: Based on data reported by public health agencies.
Tracked Data: Race and Ethnicity of Positive COVID-19 Cases by State
Reporting Frequency: 2X Per Week
How Data Is Collected: Based on state reporting.
Source: The Covid Tracking Project
Tracked Data: Excess Deaths, Including Those Attributed to COVID-19, Among Select Countries and Cities
Reporting Frequency: Weekly
How Data Is Collected: Based on “data from EuroMOMO, a network of epidemiologists who collect weekly reports on deaths from all causes in 24 European countries, covering 350m people,” and similar data sources in other countries.
Source: The Economist