Confirmation Set v1 · Source QA

The prior-period window is usable only after explicit controls.

The 502,938-row strict historical window passes core schema, date, type, monthly coverage, and revenue reconciliation checks. Missing descriptions, optional customer IDs, exact duplicates, nonpositive prices, and cancellation encoding still require frozen analytical controls.

Strict rows502,938
Months present12 / 12
Missing CustomerID19.92%
Duplicate extra rows1.30%
Controlled net revenueGBP 9,266,060.76
Quality decision: conditionally suitable for aggregate business analysis after documented controls.
The dataset gate is still pending. This profile does not prove historical non-overlap, 36-context feasibility, split independence, or confirmation performance.

Dataset and grain

A complete twelve-month transaction window, without a stable source primary key.

The intended analytical grain is one normalized source invoice-line row before exact-row deduplication. InvoiceNo + StockCode is not unique; the local-only lineage key is source_sheet + source_row_number.

Invoices27,728
Stock codes4,621
Descriptions4,624
Customers observed4,336
Countries40

Completeness

Core transaction fields are complete; two analytical fields are not.

Canonical fieldNullBlankMissing totalMissing rate
InvoiceNo0000.00%
StockCode0000.00%
Description2,82102,8210.56%
Quantity0000.00%
InvoiceDate0000.00%
UnitPrice0000.00%
CustomerID100,2070100,20719.92%
Country0000.00%
Control: CustomerID remains optional for month, country, product, and cancellation questions. Description is mandatory for product-name and valid-net evidence.

Duplicates and grain

Only normalized exact rows are removed before aggregation.

Candidate grainExtra rowsExtra rateAffected rowsGroupsMax group
Raw exact row6,5441.30%12,6666,1226
Normalized exact row6,5441.30%12,6666,1226
InvoiceNo + StockCode candidate key12,7402.53%24,28011,54010

InvoiceNo plus StockCode is not unique. Preserve source-row grain locally and remove only repeated normalized exact rows before aggregation.

Cancellation and value controls

Negative quantity is broader than the invoice-prefix signal.

Cancel-prefix rows9,877
Negative-quantity rows11,933
Negative without C2,057
C without negative qty1
Return/cancel rows after dedup11,910
MetricValue
Gross positive revenueGBP 9,834,505.21
Negative revenueGBP -568,444.45
Net revenueGBP 9,266,060.76
Gross + negative - netGBP 0.00
Merchandise net revenueGBP 9,157,800.84

Frozen analytical policy

Cleaning rules are explicit and auditable.

LayerRule
Exact Duplicatekeep first normalized exact row across the eight canonical business fields
Positive Salesdeduplicated, no cancel prefix, Quantity > 0, UnitPrice > 0, Description present
Valid Net Revenuededuplicated, UnitPrice > 0, Description present; negative quantities and cancel invoices retained
Merchandise Net Revenuevalid net revenue plus the existing no-digit and special stock-code exclusion
Tail policy: IQR and percentile flags are diagnostic only. Large quantities, prices, or adjustments are not deleted automatically because that would introduce outcome-dependent filtering.

Monthly coverage

Every frozen month is present and reconciled.

MonthRowsInvoicesCustomersStock codesCountriesNegative qtyDuplicate extrasNet revenue
2009-1245,2282,3301,0453,141241,105506GBP 796,648.50
2010-0131,5551,6337862,80919814321GBP 622,479.50
2010-0229,3881,9698072,971191,023330GBP 531,265.37
2010-0341,5112,3671,1112,928231,034526GBP 763,247.24
2010-0434,0571,8929982,63523701410GBP 641,521.05
2010-0535,3232,4181,0622,790201,352415GBP 613,270.72
2010-0639,9832,2161,0952,78424933514GBP 677,073.87
2010-0733,3832,0179882,64821822391GBP 617,365.48
2010-0833,3061,8779642,72723666367GBP 654,774.39
2010-0942,0912,3751,2022,75528905484GBP 851,105.96
2010-1059,0982,9651,5772,903271,159849GBP 1,080,611.48
2010-1178,0153,6691,6833,137291,4191,431GBP 1,416,697.20

Findings and controls

Known data issues are documented rather than hidden.

mediumhigh confidence

Optional Customer Id Missingness

Risk: Customer-level questions and customer-count comparisons may be biased if missing IDs are treated as observed customers.

Control: Keep CustomerID optional for aggregate month, country, product, and cancellation analyses; prohibit customer-level confirmation questions until a separate coverage rule is frozen.

mediumhigh confidence

Missing Product Descriptions

Risk: Rows without descriptions cannot support auditable product-name claims.

Control: Exclude missing-description rows from product and valid-net evidence while retaining their counts in the quality report.

mediumhigh confidence

Exact Duplicate Source Rows

Risk: Uncontrolled duplicates would overstate quantities, gross revenue, cancellation impact, and ranking values.

Control: Use first-occurrence exact-row deduplication on the eight normalized canonical fields before revenue aggregation.

mediumhigh confidence

Cancellation Encoding Is Not Prefix Complete

Risk: Using only InvoiceNo prefix C would miss negative-quantity adjustments or returns.

Control: Retain negative quantities in net revenue and define cancellation/return review rows as cancel-prefix OR negative-quantity after exact deduplication.

mediumhigh confidence

Nonpositive Prices Require Control

Risk: Nonpositive prices do not represent ordinary merchandise revenue and can distort value-based questions.

Control: Require UnitPrice > 0 for positive-sales and valid-net evidence; retain excluded-row counts for auditability.

lowhigh confidence

Long Tailed Numeric Values

Risk: Large legitimate wholesale orders and manual adjustments can dominate rankings; automatic outlier deletion would create outcome-dependent filtering.

Control: Do not remove tail values automatically. Review only evidence contexts selected by frozen, outcome-blind rules.

Research boundary

Quality profiling does not authorize model execution.