Design a Reproducible Carrier-Research Sample

To audit a trucking-company list with limited time, freeze the eligible list, define what one sampled unit represents, record how you select it and keep deliberately chosen problem records separate from the general review sample. Preserve unfinished checks in the results. A reproducible diagnostic sample can expose a defect and guide the next investigation; it does not automatically establish the accuracy of the entire list.

This guide helps a research manager choose which carrier records to inspect and what the resulting evidence can support. The output is a sampling plan and a saved selection manifest. It is an editorial workflow for your existing files, not a USDOTData sampling feature or a statistical certification.

Decide whether you need diagnosis or an estimate

Write the decision before selecting attractive or suspicious rows. A useful diagnostic question is: “Can researchers recover the source date and entity match for this export, including its less complete segments?” A different question is: “What percentage of every eligible record meets our defined rule?” The second requires a population-estimation design, not just more spot checks.

Statistics Canada distinguishes probability methods, where random selection and calculable inclusion probabilities support statistical inference, from subjective selection. Its probability-sampling guidance explains simple random, systematic and stratified approaches. Picking records because they look interesting is a different design choice.

Choose the smallest claim that answers the immediate business question. Finding a reproducible import problem in six selected records may justify investigating that transformation. It does not require claiming that a particular share of all trucking companies has bad data. If a purchasing contract, external quality claim or budget decision needs a population estimate, arrange suitable statistical design before collecting the evidence.

The export-evaluation checklist defines what to check in a record. This page defines how to select the records without disguising selection decisions.

Freeze the frame and choose the counting unit

The sampling frame is the actual set from which selection can occur. Describe the source file, extraction time, available coverage and eligibility rule. “Our October export of entities meeting the saved filters” is a defensible frame description. “The trucking industry” usually is not.

Choose one unit for the main review: a distinct carrier entity, one entity's dated observation, a contact row or a service location. These answer different questions. If the audit concerns carrier-level identity, repeated contacts for one carrier must not give that carrier several chances to enter an apparently entity-level sample.

Before selection:

1. Preserve the original file and create a working copy.
2. Count original rows and explain the row grain.
3. Resolve exact duplicates and conflicting identifiers under a documented rule.
4. Record exclusions and their reasons; unresolved records are not automatically invalid businesses.
5. Assign stable review IDs and freeze the eligible version.

FMCSA's Open Data Program provides dataset-specific documentation and distinguishes different record types. Keep the relevant dataset and dictionary attached to your frame. A current download and a recent carrier filing are separate facts.

If missing identifiers are the defect you want to investigate, do not silently exclude every row without an identifier and then report excellent identity completeness. Keep an unresolved-row diagnostic group, using local review IDs, and disclose that those rows have not been reconciled into distinct entities. Do not mix its row counts into entity totals.

Use two named selection streams

A practical diagnostic review can have a general selection stream and a targeted exception stream. These are our working labels. Keep each record's selection reason visible.

For the general stream, use a documented random selection method if available in your existing tools. Save the full eligible frame, chosen identifiers and the method's settings. If using temporary random values in a spreadsheet, freeze the values before sorting or reopening the workbook; otherwise the selection can move. Save the resulting selection order even if a seed is unavailable. A saved result makes the actual review reproducible without pretending that its random draw can necessarily be regenerated.

For the targeted stream, deliberately include issues that deserve investigation: conflicting names for one identifier, missing source dates or an unexplained field conversion. Record the condition, eligible exception pool and how records were chosen from it. This stream is valuable precisely because it concentrates on a concern. Its defect share must not be presented as the general list's rate.

Statistics Canada's non-probability guidance discusses the selection bias that can arise from convenience and judgment methods. More records do not, by themselves, remove that bias.

If a record qualifies for both streams, retain both reasons but review it once. State whether your workload target counts unique units or selection memberships. Where you deliberately select additional exceptions outside the already selected general group, label them supplementary cases. They do not become another random draw from the original frame.

Make segment coverage visible

You may want to inspect different reported fleet-size bands, source-date states or geographic groups because their research problems could differ. Define each group before selection, including an explicit unknown category. Record its frame count, selected count and selection method.

For mutually exclusive groups, every eligible unit should belong to exactly one group. If a carrier can appear in several cargo categories, those categories are not automatically suitable as disjoint sampling strata. Preserve overlapping tags separately or document a single assignment rule for the sampling design.

Equal review counts do not mean the underlying groups are equally common. Reviewing 15 records from a group of 360 and 15 from a group of 60 gives the smaller group much more representation in the reviewed set. That may be sensible for diagnosis. An unweighted pooled percentage would answer a different question from the whole-frame percentage.

Do not improvise a weighting adjustment after seeing favorable results. Population estimation needs the selection probabilities, an appropriate estimator and treatment of missing outcomes. This guide's example intentionally reports diagnostic counts only.

Worked example: 60 distinct entities across two streams

All figures below are fictional. They demonstrate an audit plan, not USDOTData quality results, customer outcomes or a recommended universal sample size.

A service seller has reconciled an export into a frame of 600 distinct carrier entities. It wants to test whether its research process can recover a usable source date and resolve the entity match. Three mutually exclusive segments contain 360, 180 and 60 entities. Those counts describe this saved frame; they are not industry population estimates.

The manager sets a workload of 60 unique entities:

• General stream: select 15 entities randomly without replacement within each segment, for 45 total. Save the group definitions, frame, method and selected IDs.
• Targeted stream: select 15 additional entities with documented date or identity concerns from outside those 45. Preserve the deliberate selection reasons.
• Total workload: 45 + 15 = 60 distinct entities. No record is silently replaced for being difficult.

The general stream gives each segment one-third of the selected records, while the frame composition is 60%, 30% and 10%. That mismatch is disclosed from the start. The review is intended to find workflow defects across segments, not to publish an overall estimated accuracy rate.

At cutoff, the general stream has 33 completed matches with usable dates, five completed checks with date evidence unavailable, four completed checks with unresolved identity disagreement and three checks not completed. The counts reconcile to 45. In the targeted stream, the same four categories contain five, six, three and one entities, totaling 15.

The team has completed 42 general-stream checks and 14 targeted checks: 56 of the 60 planned reviews. Four remain unfinished. “Check completed, evidence unavailable” differs from “check not completed.” Neither establishes an inaccurate source value without further evidence.

The handoff should say: “The general selection found five unavailable-date cases and four unresolved identity disagreements; the supplementary exception review found six and three. Investigate the source-date mapping and reconcile the identity cases. Four reviews remain open.” It should not say “30% of all carriers have bad data” by combining 18 unresolved outcomes across 60 selected entities. That would confuse the outcome definition, missing work and two selection methods.

Suppose all six targeted unavailable-date cases came from the same transformed column. The next action is a focused check of that mapping across the relevant source records. Preserve this as a second investigation with its own scope. Do not rewrite the first sample's membership to make the defect appear larger or smaller.

Keep missing outcomes and replacements auditable

Choose a review cutoff and a consistent check procedure. Record whether each check was completed, unavailable because the source could not be accessed, unresolved because evidence conflicted or unfinished because the reviewer ran out of time. For research involving a permitted business response, record nonresponse separately from refusal and from an explicit answer.

Do not replace a difficult selected unit with an easier one and erase the original. If operationally necessary, add a separately labeled supplemental unit and preserve the original unresolved state. A larger completed set may improve diagnosis, but it is a changed observed set whose selection must remain visible.

For a reporting percentage, state the unit, numerator, denominator and exact condition. “56 of 60 planned reviews completed” measures execution. It does not measure accurate carriers, qualified leads or buyers. Our cohort-outcome guide covers subsequent fit, need and business outcomes.

Copyable sampling-plan and selection checklist

Use a batch card plus a short selection manifest in your authorized existing files:

Question and decision this review supports:
Permitted claim: diagnostic findings / designed estimate:
Source dataset, dictionary version and extraction time:
Counting unit and stable identifier rule:
Frame version, original rows and eligible unit count:
Exclusions, unresolved-row group and reasons:
Segment definitions and counts, including unknowns:
General method, tool/version and saved seed if available:
Saved selection order, random values or chosen-ID manifest:
Targeted conditions, selection reasons and overlap rule:
Required checks, outcome definitions and cutoff/time zone:
Selected / completed / unresolved / unfinished counts:
Reviewer, exception owner and next investigation:

Before review begins, confirm that a colleague can locate the unchanged frame, reconstruct its eligibility rule and identify every selected unit. After review, confirm that each unit still has its original selection label and that completion counts reconcile. A sampling method cannot repair an undefined field: use the carrier-data dictionary where meaning or dates are unclear.

Frequently asked questions

Is selecting every tenth row a random sample?

Not simply because the interval is regular. A probability-based systematic design uses a random start and must consider the list's ordering. Repeating patterns can distort the result. Record the interval, start and order; do not label a convenient fixed-start selection random. Statistics Canada's probability guidance explains this distinction.

Should we deliberately inspect suspicious records?

Yes, when the purpose is to understand a documented exception. Preserve a targeted label and reason, keep overlaps visible and avoid using that stream's defect share as the whole list's rate. Do not label a carrier suspicious merely because a field is missing.

Can a 60-record diagnostic review support an accuracy guarantee?

The count alone cannot. You need a defined outcome, frame coverage, valid selection design and appropriate analysis. Statistics Canada's survey-review questions connect sampling-error claims to probability sampling. This article supplies no confidence interval or guarantee.

What if the segments have very different sizes?

Keep their frame and selected counts beside one another. Equal-sized review groups can help investigate small segments but should not be pooled into a whole-frame estimate without suitable design and analysis. Report each group's observed diagnostic counts when estimation is outside scope.

Should an unanswered research question be counted as a failure?

Record the actual state: unfinished review, inaccessible source, conflicting evidence or no business response. A negative finding requires an explicit failed criterion with evidence. Retain unresolved selected units in the selection record instead of substituting a more convenient answer.

When do we need specialist statistical design?

Before using a sample to support a contractual accuracy statement, a margin of error or a costly population-level decision. The specialist needs the actual frame, unit, selection method, likely missing outcomes and intended estimate. A diagnostic pilot can help define that work without claiming it has already been done.

Start with a claim your sample can support

Write one narrow research question, freeze one existing export and save its selection manifest before examining results. If you need a new research set, begin with the actual available filters in the USDOTData trucking company database. Treat the resulting records as research inputs. A source match is an observed fact; possible service fit is a hypothesis; buying need requires separate business confirmation.

By John Hauler, USDOTData's editorial persona. Sources reviewed October 8, 2026. USDOTData is an independent information service, not FMCSA or a statistical agency. The workflow and fictional example are editorial recommendations, not a population-estimation service.