Business Intelligence with Power BI

L02 · Power Query I: Shape and Clean

Practice — Hot · Solution

1 · The twenty missing rows

Their dedup keyed on Name only. The correct all-columns dedup takes the file from 514 to 500 — fourteen photocopied rows. Name-only dedup goes further: it also collapses every pair of genuinely different customers who happen to share a name, keeping one and silently deleting the other. Seventy-seven more rows die that way: 500 → 423. Those seventy-seven were real customers with their own IDs, cities, and join dates — more than one customer in seven, deleted by a single checkbox's worth of selection scope, with no warning at any point. Worse, the trailing-space twins ("Avery Thorne" vs "Avery Thorne ") survive this dedup, because untrimmed strings differ — so the step over-deletes real people and under-deletes the near-duplicates a Trim-first ordering would have exposed.

2 · The audit

StepVerdictConsequence
2 · Dedup on Name onlyWrong77 real customers gone — 15% of the file; no error, no warning
3 · Replace N/A with typed "null"WrongCreates the text string "null" — the profile still shows 0 empty where it should show 22; slicers now offer a segment called null
4 · Type Joined before cleaning around itDefensible position, unverified resultThe conversion itself belongs in the recipe — but they never checked the error count. "Dates working" is a claim; the quality bar is the evidence, and they never looked. Position in the order is fine for this step; the sin is shipping unverified
5 · Texas→TX, aloneIncompleteCalifornia, New York, and Michigan still spelled out — 21 of the 29 afflicted rows untreated; State distinct count stuck at 29 instead of 26

3 · The diff, as you'd write it to a colleague

"Kept your overall structure — dedup, nulls, types, states is the right shape. Three changes: (1) Dedup now selects all columns; keyed on Name it was removing seventy-seven real customers who share names — row count is 500, not 423, and I added Trim before it so the trailing-space twins stop evading it. (2) The N/A replacement was inserting the text 'null'; I made it a true empty null — Segment's profile now honestly shows 22 empty. (3) Extended your Texas fix to CA, NY, and MI — distinct states now 26. Also put eyes on Joined's quality bar after your type change: zero errors on my machine, but check on yours; parsing follows locale. Good instinct putting all of it in one recipe — nothing here needed hand-edits."

Reflection answers

  • The theme: none of these produce an error. Five steps, two of them destructive or deceptive, and the Editor executed all five in serene silence. The tool verifies syntax, not intent — you are the only check.
  • The one-minute habit: profile first. Reading the Name column's profile (distinct vs. count) before deduping would have shown that names are not unique in this file — killing the "customers should be unique" assumption before it deleted anyone.