Business Intelligence with Power BI

L03 · Power Query II: Combine

Practice — Hot · Solution

1 · The finding, reproduced

A Remove Duplicates across all columns, on a copy, takes the combined table from 26,139 to 26,116 rows: 23 exact duplicates. The manager's check was real. Now the question that separates an investigation from a reaction: duplicates of what?

2 · May is not inflated

If the export system double-wrote May, orders would repeat — within the file or across files. The distinct order counts say otherwise:

FileRowsDistinct orders
January3,9032,019
February3,4121,766
March4,1822,173
April4,4542,290
May5,3042,737
June4,8842,485

The six distinct-order counts sum to 13,470 — exactly the distinct-order count of the combined table. No order appears in two files. May has more rows because May had more orders — 2,737 of them, against January's 2,019. This company sells fire pits and patio furniture; May is not an anomaly, May is the business model. (Lesson 1's line chart made the same argument from the revenue side.)

3 · What the 23 duplicates actually are

Open a few: same order, same product, same quantity, same price. These are two legitimate lines in one order — a customer bought the same item twice in one purchase — and they look identical because the export has no line-number column. Nothing was double-written; the export's schema simply cannot distinguish "this line, twice" from "this line, duplicated." The flaw is in the file's design, not its data.

4 · The reconciliation settles it

Finance's independent H1 figure: $11,515,008.14. The combined table reproduces it with the 23 rows left in. Remove them and the table goes short — 23 real order lines of revenue missing. The manager's proposed fix would have created the very inflation problem they feared, with the sign flipped.

5 · A verdict that would hold the room

"May is genuinely bigger — 2,737 distinct orders against January's 2,019, and no order number appears in two files, so nothing was double-written. The duplicate check found 23 rows that are real order lines: when a customer buys the same item twice in one order, this export writes two identical rows because it carries no line number. The table reconciles to finance's $11.52M to the cent with those rows left in — removing them would understate H1. Remove Duplicates stays out of this recipe, and I am requesting a line-number column from IT so the next person who runs this check gets a clean answer instead of an accusation."

Reflection answers

  • The flipped property: whether the table has a key that makes true duplicates distinguishable from repeated facts. The customer file's rows were one-per-customer — a full-row copy could only be an error. Transaction lines without a line ID can legitimately repeat. Same button, different data contract.
  • The manager had a check; you had a check plus an expectation plus an independent reference. Their "found duplicates" had no answer to "how many should there be, and what total should this table produce?" Evidence without expectation is how real numbers get called fake — the same lesson as Medium's, wearing a manager's face.