Methods, Checks
and Remaining Risk

Validation memo

Object: Nine-sector-ETF momentum rebalancing, 2001-2025. Date: 9 October 2026. Purpose: assess a frequency claim and the reliability of its retrospective evidence. Judgment: calculation reconciled; general frequency superiority unsupported; remaining model and data risks disclosed.

Scope

Compare monthly selection/reweighting with six semiannual calendars, four costs and two signals. Add six autonomous sleeves as a calendar-diversified comparison. The main study is 12-1 at 5 bps per side; 2026 YTD and A/B/C mechanisms are secondary. This is a financial research case, not a bank-approved model, live strategy or trading recommendation.

Data lineage

Preserved vendor responses -> common-session adjusted open/close input -> fixed portfolio accounts -> independent accounting replay -> tranche sums -> monthly risk and paired intervals -> public derived CSVs and summary. Every capture, protocol and generated account has a SHA-256 manifest. Official French monthly RF is captured separately and joined to exactly 300 months. Hashes establish file identity, not independent authenticity of vendor observations.

Independent implementation checks

The legacy verifier uses standard-library code and an analytic fee solution rather than the engine's bisection. It reconciles 60 paths. The mechanism verifier reconciles 108 paths and 726,732 daily states; maximum relative NAV error is 1.78e-14. A separate extension verifier imports no project calculation code and checks eight tranches, 53,832 daily states and 34,800 monthly rows, including Sharpe, tracking error, beta and bootstrap percentiles. Maximum relative tranche NAV error is 2.22e-16.

Independence boundary: the same AI agent authored the implementations and reviewed the outputs. Different calculations can expose implementation errors; they cannot exclude a shared misunderstanding of the specification. No external reviewer acceptance is claimed.

Findings

• At 5 bps, changing only the semiannual calendar changes the 12-1 comparison's sign: -0.92 to +0.53 pp/year.
• Monthly minus March/September is -0.49 pp/year; paired 95% CI [-3.03, +2.11]. Monthly minus tranche is -0.22; CI [-2.44, +2.33]. Both span zero.
• Tranche CAGR is 8.24%, but its daily maximum drawdown is -40.91%; calendar pooling does not guarantee lower risk.

Limitations and residual risk

Historical results were inspected before this extension was specified. Adjusted units approximate total return and next-open execution, rather than a raw-share corporate-action ledger. Cash earns zero; costs are fixed and there is no spread, impact, capacity or tax model. ETF sector definitions changed. Bootstrap stationarity and block choice do not remove regime or design-selection risk. Source-price accuracy and real execution remain unverified. Written result and judgment reviews have been accepted; exact signal-window understanding and an observed oral presentation remain pending. Owner mastery is not certified.

Disposition

Accept as a reproducible, bounded historical validation case. Do not use it to select an optimal schedule, assert causal timing attribution, certify executable returns or establish personal technical mastery. Publish all tested comparisons and keep computation completion, owner readiness and deployment status separate.

Result summary · Extension receipt · Reproduction

Method protocol

Data and timing

Nine original sector ETFs: XLB, XLE, XLF, XLI, XLK, XLP, XLU, XLV and XLY; SPY is context. Preserved Yahoo chart responses cover their common sessions from 22 December 1998 through 2 October 2026. Signals and accounts start at the 31 December 1999 close; first execution is 3 January 2000. The primary anchor is 29 December 2000, the last common session of 2000, and the primary endpoint is 31 December 2025. No liquidation or capital reset occurs at reporting boundaries.

The signal for lookback L is (P[t-21] / P[t-L] - 1) / (sample_sd(daily_returns_in_that_window) * sqrt(252)). The primary recipe uses L=252. At t=1000, its prices are P[748] and P[979], and its volatility contains 231 simple daily returns r[749] through r[979], where r[k]=P[k]/P[k-1]-1. The most recent 21 returns r[980] through r[1000] enter neither numerator nor denominator. The mixed recipe weights the 252/126/63-session components 0.5/0.3/0.2. Scores must be positive; select up to three ranked ETFs. Each selected name receives one-third of post-fee NAV; unused slots stay as zero-interest cash.

Month-end closing decisions execute at the next exchange-session opening proxy. A monthly portfolio refreshes selection and weights monthly; each semiannual portfolio refreshes in its two designated signal months. Six phases cover Jan/Jul through Jun/Dec, with March/September retained as reference. Costs are 0/5/10/25 bps per side, each a separate self-financing account. Execution solves post-fee targets together with fees. Vendor adjusted closes and Open * AdjClose / Close are proxy units; no separate dividend cash is added.

Six-sleeve tranche

For each signal and cost, six semiannual accounts each represent one-sixth of initial capital. Pooled NAV, cash, asset units, gross traded notional and fees are their scaled sums. The 2000 burn-in can create unequal sleeve capital before reporting starts; equalizing at the reporting anchor would change this protocol. No subsequent sleeve transfers or order netting are allowed. Pooled turnover on a trade date divides summed gross notional by pooled opening pretrade NAV, including non-trading sleeves marked at that opening.

Inference and risk

Use exactly 300 aligned calendar-month net simple returns for 2001-2025. Primary differences are monthly minus March/September and monthly minus tranche, under 12-1 and 5 bps. Paired stationary bootstrap samples common monthly rows, with a geometric restart probability of 1/block_length and circular continuation. Seed 20261009; 10,000 draws; mean block lengths 12 (primary), 6 and 24. Calculate each sampled CAGR as exp(12 * mean(log(1+r))) - 1, then take percentile bounds of the paired CAGR differences. The calculation samples return paths, not prices or selection histories. No primary p-value claims or equivalence test are made.

Sharpe is sqrt(12) * mean(R - RF) / sample_sd(R - RF). Tracking error is sqrt(12) * sample_sd(R - R_SPY). Beta includes an intercept and regresses R-RF on R_SPY-RF. RF is the decimal monthly series from the captured Kenneth French factor archive; a revised historical series is not a point-in-time vintage. Daily volatility and maximum drawdown remain separate. Undefined zero-variance ratios are unavailable, never silently set to zero.

Mechanism appendix

The already computed A/B/C extension is retained. A=semiannual selection/reset; B=semiannual selection/monthly reset with the complete target vector and cash slots frozen; C=monthly selection/reset. Same input and costs, one next-open fill at coinciding events. Conditional differences B-A and C-B telescope in log growth but do not identify unique causal mechanisms. Mean monthly log differences use Bartlett HAC12 with small-sample correction, normal intervals and Holm correction for three contrasts. Stationary bootstrap sensitivity remains attached to the appendix results.

Registration and interpretation

The original history and calendar results were already inspected. The validation extension's rules were recorded and hashed before its tranche and interval calculation; this is retrospective freezing, not prospective registration or an untouched holdout. Freeze records and previous manifests remain preserved. No observed winning phase is promoted. Market-state, event, adaptive-rule and multi-factor studies are deferred.

Machine-readable extension protocol · Freeze record · Legacy ETF rules · Mechanism rules

Design decisions and questions for the owner

These explanations document the implemented design and its tradeoffs. They are AI-assisted technical notes, not reconstructed statements about what the owner thought when the original choices were made. The owner can endorse, revise or reject each rationale after review; actual responses are recorded separately.

1. Why skip the most recent 21 sessions?

Rule: return is P[t-21]/P[t-252]-1. At t=1000, use P[979]/P[748]-1; the far endpoint is measured from t, not from t-21. Rationale: measure a medium-horizon trend without including the latest month; this follows a common momentum convention and makes the research recipe explicit. Cost: a sudden trend reversal can be ignored for a month. Alternative: no skip or a shorter skip, specified as a separate test rather than selected from this sample. Owner prompt: show the two endpoint prices and explain exactly which returns are excluded.

2. Why divide by volatility?

Rule: divide each horizon's price return by sample daily-return volatility in the same endpoints, annualized by sqrt(252). For L=252, use the 231 returns from t-251 through t-21, with ddof=1; the latest 21 returns are excluded. Rationale: compare momentum relative to recent variability rather than rank only raw price gains. Cost: ranking depends on an estimated denominator and its window; low volatility can magnify a score. Alternative: raw momentum or an independently specified volatility window. This ranking adjustment does not make the portfolio volatility-targeted: holdings still use one-third slots. Owner prompt: explain how two ETFs with the same gain can receive different scores.

3. Why closing signals and next-open execution?

Rule: compute with the decision close and trade at the next exchange-session adjusted opening proxy. Rationale: the completed closing observation is available before the modeled fill. Cost: overnight gaps and opening frictions matter; the proxy is not an executable quote. Alternative: a later execution or a genuinely available intraday signal, with a different data contract. Owner prompt: identify a signal/fill date pair and explain what same-close execution would assume.

4. What can adjusted prices represent?

Rule: adjusted close for signals; Open times AdjClose/Close for execution. Rationale: a consistent vendor-adjusted total-return proxy without adding dividend cash twice. Cost: vendor revisions, dividend reinvestment conventions, distributions and spin-offs may diverge from a raw-share ledger; a retrospectively adjusted opening value is not an actual opening fill. Alternative: raw prices, share changes, ex/pay dates and a corporate-action ledger. Owner prompt: distinguish a proxy unit from a real ETF share and name one risk the replay cannot resolve.

5. Why is turnover buys plus sells, not divided by two?

Rule: sum absolute executed buy and sell notional, divide by pretrade NAV; aggregate over the period, then divide by 25 for annual activity. Rationale: the cost model charges each side, so both purchases and sales generate fees. Cost: the reported number differs from one-way industry turnover conventions. Alternative: also report one-way turnover under an explicit definition, without changing fee accounting. Owner prompt: selling $100 and buying $100 generates $200 gross traded notional and $0.10 fee at 5 bps per side, not $0.05.

Further decision: what is diversified by tranching?

One-sixth allocation at formation spreads calendar exposure across six autonomous sleeves. Sleeve wealth subsequently drifts. It diversifies a scheduling choice, not necessarily common holdings, market risk or drawdowns. The observed -40.91% maximum drawdown stays in the result even though it weakens a favorable mitigation story.

Personal contribution record

Confirmed in the conversation: the owner selected model-validation/AI-evaluation readers, the financial-project-only scope, calendar sensitivity as the principal story, and three short review checkpoints. Implementation, numerical checks and narrative drafts were AI-assisted. The owner personally reconciled one historical trade and submitted accepted written explanations of calendar sensitivity, paired uncertainty, pooled drawdown and the price-source/fill-evidence boundary. The exact signal-window explanation remains unresolved; observed oral timing and owner notebook execution are absent. The dated public progress record distinguishes these contributions from AI implementation. Historical rationales above are not attributed to the owner without an actual endorsement.

Methods · Validation memo

Reproduction and evidence guide

Public-input tier: no credentials or vendor download

Create a virtual environment and install requirements-research.txt. Use its Python for every command:

python scripts/validate_design.py
python scripts/validate_pilot_artifacts.py
python scripts/validate_stock_pilot_artifacts.py
python scripts/validate_validation_artifacts.py
python -m unittest discover -s tests -p 'test_*.py' -v
python -m pilot.validation_figures --summary site/data/validation-study-summary.json --output-dir site/assets
python scripts/build_validation_documents.py
python scripts/build_validation_pdfs.py
python scripts/build_validation_notebook.py
python -m http.server 8000 --directory site

The notebook includes editable plotting and statistical calculations. It loads public derived daily portfolio NAV and 34,800 monthly return rows, including captured RF; recomputes CAGR, risk metrics and primary bootstrap intervals; and checks those outputs against the validated summary. It does not reconstruct holdings from public asset prices. The notebook is executed and saved with outputs; HTML is supplied for reading without a kernel.

The summary identifies the protocol, freeze, original source manifests, generator code and replay receipts. Public data are portfolio aggregates, not raw vendor observations. RF comes from the official French factor archive. Current factor histories can be revised; capture identity is documented. FF definitions describe the RF series. The stationary bootstrap paper is the methodological reference.

Private-snapshot tier: exact engine and accounting replay

The preserved raw Yahoo responses, normalized input, source accounts and complete ledgers are excluded from Git. Exact replay requires the locally retained snapshots with matching manifest hashes. A new vendor acquisition can contain revised values and is a new run, not an exact reproduction. No paid API is required by the ETF extension.

python scripts/run_etf_mechanisms.py --input-run-dir results/generated/etf-pilot/20261009T021722928426Z-9e9cb830b3cb --output-dir results/generated/etf-mechanisms
python scripts/run_validation_study.py --run-dir results/generated/etf-mechanisms/20261009T050633543130Z-d96bbf03d531 --factor-dir data/raw/french-factors/20261009T050942196828Z-abb85a14 --validation results/generated/etf-mechanisms-validation/20261009T050633543130Z-d96bbf03d531.json --output-dir results/generated/validation-study/NEW-UNIQUE-RUN
python scripts/validate_validation_study_independent.py --source-dir results/generated/etf-mechanisms/20261009T050633543130Z-d96bbf03d531 --extension-dir results/generated/validation-study/20261009-v1 --factor-dir data/raw/french-factors/20261009T050942196828Z-abb85a14 --output results/generated/validation-study/REPLAY-RECEIPT.json

The first command creates a new timestamped mechanism directory. Its independent verifier must be run before using that directory as input, and the new hashes must be recorded in a newly versioned extension protocol. The second command above reproduces the existing frozen source run into a new output directory. It refuses overwritten output or stale source manifests. See the archived mechanism method for the source-account replay commands.

Artifact generation and validation

The extension's private manifest is immutable. Local public preparation copies only permitted derived inputs and binds their hashes to the public summary. The extension receipt binds both the private and public summary identities. Runtime plot/report generation uses the validated summary; a completion record separately states whether outputs and owner checkpoints are complete.

The independent extension verifier imports no project calculation code. It recomputes sleeve sums, fees, opening NAV, monthly returns, risk ratios and bootstrap percentiles. Authorship is still the same agent. Tests include hand calculations and failure injection for shifted dates, missing months, wrong fees and stale summaries. Neither checks nor manifests certify vendor prices or tradability.

Local review and publication

Open the generated site, four-page brief, one-page memo and executed notebook. Check desktop/mobile layouts and data-unavailable behavior. Record the three owner checkpoints separately. The dated project status separates owner readiness, release authorization and observed deployment. The publication receipt records a specific verified commit and file identities; later commits require their own checks. A successful local build alone does not establish a live deployment.