BizHallu Career Package

1. BizHallu 0:00-0:20

I am a current MS student at Johns Hopkins Carey Business School, pursuing the Master of Science in Business Analytics and Artificial Intelligence. BizHallu asks whether AI-generated business conclusions follow from transaction evidence. I directed the project with AI-assisted implementation and review. My accounting and supply-management background motivates the distinction between a number that reconciles and a relationship that supports a decision.

2. April: a correct amount with an incorrect rank 0:20-0:55

Here is April 2011. The answer places WOODEN UNION JACK BUNTING third with revenue of 4,173.18 pounds. We can locate that exact product and amount in the source table. But six shown products have larger values. The correct third product is PAPER CHAIN KIT EMPIRE at 6,619.51 pounds. A check that only asks whether the amount appears in the table would miss the ranking error.

3. September: the same relationship error 0:55-1:25

The September answer repeats the pattern across all three listed products. Their stated ranks are one, two and three, while their ranks in the eight shown evidence rows are three, eight and two. Every product still matches its own amount. I use these cases to distinguish row fidelity from relationship correctness. They are selected historical examples, not evidence of a general error rate or an automatic verifier.

4. Research question: complete business relationships 1:25-1:55

The research question is what uncertainty signals miss when copied facts form an incorrect business relationship. A rank only makes sense with its product, metric and comparison scope. The proposed comparison asks whether an evidence checker adds useful information under matched conditions. Business definitions remain part of the task: negative transaction value can include fees and adjustments, so it cannot establish physical returns. The historical merchandise definition uses a documented stock-code heuristic.

5. Historical evaluation and its sample 1:55-2:30

The pipeline turns retail transactions into deterministic questions and evidence tables, then captures local Qwen answers and token traces. B1 retains the original 205 provisional spans across 35 development and test answers. A separate B2 sensitivity adds nine assistant-provisional atoms for the omitted dev answer, without changing the 103 old test spans. Fifteen selected original spans received additional assistant review. None of this is independent human annotation or end-to-end claim extraction.

6. The F1 advantage remains uncertain 2:30-3:15

The chart preserves B1's original test results. Entropy F1 0.779; flag-every-span F1 0.744. Paired F1 difference +0.0355; exploratory 95% interval [-0.0356, 0.1047]. That interval is conditional on fixed thresholds and is not a B2 interval. B2 adds one previously omitted dev answer with nine provisional correct facts. Refitting on dev lowers entropy F1 to 0.752 on the same old test, with five fewer false alarms and six more missed errors. Margin is unchanged in this check. We report sensitivity, not a new winner or fresh confirmation.

7. A fair comparison needs the same information 3:15-3:45

One methodological correction is especially important. A list marker is generated before the product and amount that follow it. Low uncertainty on that marker cannot establish that the completed relationship was confidently wrong. The saved raw teacher-forced logits also differ from generation probabilities after sampling transformations. Some nominally different features are mathematical aliases: same-step energy gap is token negative log probability. These distinctions prevent an unfair comparison with a checker that sees the whole answer.

8. What the current evidence supports 3:45-4:10

The results describe a limited retrospective study. The annotation queue depended on generated-answer status, development and test share contexts, and independent human agreement has not been measured. The six curated relationships clarify particular cases; they do not replace the 205 labels. No result here establishes performance on larger models, unseen periods, automatically extracted claims or production business reports.

9. Proposed comparison: product-ranking verification 4:10-4:35

The next implementation targets product rankings. It will use the question, generated answer, metric contract and evidence rows to return a support decision, evidence reference and abstention when needed. Gold answers and evaluation labels stay outside prediction. Two independent reviewers will establish relation labels. The comparison will report extraction coverage and abstentions, because a checker can appear accurate by avoiding difficult claims. This verifier is proposed work, not a completed result.

10. A focused discussion with a research mentor 4:35-5:00

My request is a focused discussion of the annotation unit and comparison design. The immediate deliverable would be a small independently reviewed relation set and one evidence checker. The existing 48-context design offers a later evaluation path, but it remains unexecuted and estimation-focused. Model and prompt settings, reviewers, and the prediction protocol must be ready first. A second dataset or model would follow only after this smaller comparison is credible.

Reconcile the amount and its scope

December 2010 through December 9, 2011; December 2011 is partial. Historical eligible population: 534,129 lines, GBP.

Preserve existing valid-net flags: exclude normalized duplicate copies, missing/blank descriptions and nonpositive unit prices. Missing customer IDs are retained for aggregate analysis, not imputed. No anomaly is automatically removed.

Positive transaction value: 10,642,110.80. Signed negative transaction value: -893,979.73. Merchandise net value: 9,771,708.99.

Question: Calculate all-ledger net value and the net contribution outside merchandise. Does the negative value identify physical returns?

Ledger categories and metric contract

Follow-up: What extra records would be needed to link returns to original sales? Why is a partial December unsuitable for an unqualified full-month comparison?

Check the complete ranked statement

April 2011, q_0064. Qwen's exact statement: 3. **WOODEN UNION JACK BUNTING** (rank 3) with a net revenue of GBP 4,173.18.

Use the original eight-row evidence table in the April case. Preserve its product, period, metric and GBP scope.

Question: Find the product and amount. Count shown rows with larger values. Which part is supported, which relationship is wrong, and what belongs at rank three?

April answer and original evidence rows

Follow-up: If a real amount appeared beside a different product, would a number-only lookup be enough? If evidence were incomplete or values tied, what scope or ranking rule would be needed?

Recompute F1 before interpreting it

Original B1: 103 pre-identified test spans with provisional labels. Positive means a span labeled hallucinated, not a positive sales value.

Entropy: TP=53, FP=22, TN=20, FN=8. Flag every span: TP=61, FP=42, TN=0, FN=0.

F1 = 2TP / (2TP + FP + FN). TP is a flagged labeled error; FP is a flagged labeled-correct span; FN is a missed labeled error.

Question: Compute both F1 values. How many fewer false alarms and how many more misses does entropy have? Does its higher point estimate prove stable superiority?

B1 references and separate B2 sensitivity

Follow-up: Why can flagging everything have a high F1 here? What false-alarm and missed-error costs would you need before choosing a reporting policy?

What is the main business finding?

A copied amount can match its product while the product has the wrong rank. April and September demonstrate that distinction within the shown evidence; they do not establish a population error rate.

Why examine spans and also full relationships?

Atomic spans locate an error, but a rank belongs to a product, period and metric. A relationship record preserves those dependencies and avoids interpreting an early list marker as the completed claim.

Why use Qwen3-0.6B?

It fit the local compute budget and exposed inspectable failures. Small size does not establish external validity. Reproduction also needs frozen model/tokenizer versions, decoding settings, data policies and saved artifacts.

Did the detector beat simple references?

Entropy F1 0.779; flag-every-span F1 0.744. Paired F1 difference +0.0355; exploratory 95% interval [-0.0356, 0.1047]. B2 sensitivity adds 9 assistant-provisional q_0048 dev atoms. Dev-only refitting changes entropy F1 from 0.779 to 0.752 on the same 103 old test spans; top-2 margin is unchanged. Original B1 results remain intact. This is not fresh held-out evidence or independent review. These checks do not establish stable F1 superiority.

Why is the fact-type prior not a new winning detector?

It uses supplied annotation categories, some with correctness hints. It diagnoses sample composition rather than providing an information-matched automatic prediction system.

Is this an independently human-labeled benchmark?

No. There are 205 AI-assisted provisional labels and 15 selected presentation spans with additional assistant review. Independent human annotation and agreement remain uncompleted.

What changed about the energy interpretation?

Same-step selected energy gap equals NLL. Probability outside the top two choices is a concentration control. A faithful adjacent-step Spilled Energy comparison still needs a separate formula and time-index audit.

Does negative transaction value measure returns?

Positive and negative transaction values are sign-based accounting quantities. Negative value is not a verified measure of physical returns; fees and adjustments may be included. Merchandise scope uses a stock-code heuristic, and the historical product grain is stock code plus description. Without separate inventory, cost and delivery records, this project cannot prove inventory optimization, profit improvement or supply-chain optimization.

What did you do, and where did AI assist?

I directed the project with AI-assisted implementation and review. The repository documents the calculation, annotation and evaluation work. These artifacts are not evidence that every step was coded unaided or that independent human review occurred.

What is the next comparison?

A small independent top3 verifier versus internal uncertainty, with extraction coverage, abstention and information conditions reported. Semantic Entropy, TOHA and entity probes remain research candidates subject to compatibility and cost checks, not promised results.