When KPIs Go Wrong: Goodhart's Law for Industrial Engineers

When digging into performance issues observed by industrial engineers, I often hear complaints of the form: “Operators forget to switch machine configuration between runs, and this leads to entire batches getting ruined.” But then, when I ask how the operator’s performance is measured, they say “cycle time.”

This actually has a name in economics, Goodhart’s Law, and a pithy description:

When a measure becomes a target, it stops being a good measure.

Factories continuously pursue multiple goals at the same time: throughput, first-pass yield, avoiding escapes, and keeping people safe. But, it is challenging to connect any of these plant-level outcomes to the behavior of a single operator on a single station, so in day-to-day practice the clearest signal that reaches the station is often just how fast the cycle runs and the informal message becomes “just do it faster,” even when that message is detrimental to the factory’s actual goals.

Where the Number Slides Away From the Work

“Just do it faster” has many failure points, but the big ones are “what is it?” and “at what cost/quality?”.

For changeovers, if you define the window as start-to-start and leave prep undefined, prep gradually moves outside that window and often becomes unmonitored and un-QC’ed. Teams begin staging materials in the prior run, perform warm-up cycles off the clock, and some cleaning work might get skipped. The reported average changeover time decreases, but the time from the last good part on the old SKU to the first good part on the new SKU changes little and quality metrics slip.

The same thing happens with quality metrics when the station’s emphasis is only on the first pass. If an operator is judged mainly on how many units they send downstream per hour, there is a strong temptation to pass borderline parts and let QC decide what to do, which looks efficient on the first pass but pushes work into rework. Even worse, decreases in quality may not be visible in the main KPI at all, and instead show up as a growing amount of off-router repair, sorting, and retest that lives in a hidden factory.

Even if there is a formal rework area, operators who are pressured to clear units quickly rather than to eliminate sources of defects can end up running many small repair loops on the same product, so that a single “good” unit on the output chart has actually consumed several cycles of extra handling and test that never appear in the original station metric.

A Simple Card for Designing KPIs

A practical question is how to give operators and supervisors a KPI they can influence directly without losing the connection to the line or the value stream’s overall performance and value. For any KPI that touches flow, it’s helpful to consider not only the target but also the constraint and health. A constraint is what you are not willing to trade away to hit the target. Health is a metric that indicates whether local improvements are consistent with system performance. The purpose of the card is to ensure the primary metric is always interpreted alongside its main trade-offs.

With the card in place, it became immediately apparent that a changeover that met the time target but failed the first article or pushed extra WIP into the downstream cell was not a success.

The Parts of the Job the KPI Cannot See

When a KPI at a station rewards speed on the first pass, work tends to move into places that the KPI does not measure at all. Some of this is non-value-add time, such as search, walk, fetch, reach, and wait, which is easy to hide before or after the formal cycle. Some of it is quality loss that is pushed into rework and retest, where extra passes and minor fixes are performed without being counted toward the original job. From the cycle time chart, the station appears efficient, but the actual effort spent on that unit is much higher than the KPI suggests.

A straightforward way to address this is to make both kinds of work visible. On the time side, define a small, stable set of non-value-add categories and record how much time each cycle or changeover spends in those categories, alongside the value-add steps, using the same observations or video clips that are already being collected. On the quality side, track how many units require rework or retest and how much time those loops consume, so that local speed gains that generate more rework show up as worse performance rather than better. Once this breakdown is visible, improvements naturally shift toward reducing unnecessary motion and rework, rather than moving work into unmeasured parts of the process.

Measure the Mechanism, Not Only the Outcome

Another way to reduce the gap between the KPI and the real work is to focus not just on outcomes, but also on the mechanisms that create them. Throughput, OEE, and first-pass yield are needed for reporting and planning, but they describe the results of the system rather than the detailed work that produced those results, which makes them easy to shift by changing definitions or shifting work elsewhere. Mechanism metrics describe how the work is actually done: whether setup standard work is followed, whether the sequence of steps is stable, whether the first article passes at the intended point in the cycle, and whether tools and materials are present before the changeover window starts.

These mechanism metrics are easier for supervisors to coach and easier for industrial engineers to connect to specific changes in layout, work instructions, or staffing. The main historical barrier has been the cost of measurement, because creating an element-level timeline usually required many passes through video or long sessions with a stopwatch and paper, which meant that detailed metrics were used sparingly and often went out of date.

This is now changing. It is becoming realistic to turn short clips from phones, GoPros, or overhead cameras into element-level timelines with labeled steps and clear value-add and non-value-add splits, without special hardware installations and without writing new code for every station. When detailed observation is this cheap, it becomes possible to support mechanism-oriented KPIs as part of regular weekly work instead of as rare projects. Check out our previous blog post for a deeper discussion of why under-measuring work creates blind spots and how cheap mechanism measurement lets you redesign KPIs around the real process instead of around incomplete proxies.

How to Try This on Your Line

If you want to try this on your own line, you can start with a small experiment. Pick one KPI that people already know, such as changeover minutes on a specific cell. Write down a clear Target, add one or two Constraints that you are not willing to sacrifice (for example, first article pass and a quality check), and add a simple Health measure such as p95 changeover and WIP at the following constraint. This gives you a basic Target / Constraint / Health card for that station.

Next, collect a short sample of real work and look at the mechanism in a bit more detail. A door-to-door video of a few changeovers is usually enough. From that sample, estimate or measure how much time is spent in value-add steps, how much is spent in non-value-add categories, and in rework. You do not need perfect numbers on the first pass; the goal is to identify the apparent problems and how they relate to the Target, Constraint, and Health metrics on the card.

Finally, review the card and the simple mechanism view together with the team that runs the station. If the Target is green but the Constraints or Health signals are not, discuss which parts of the sequence are creating that gap and agree on one or two concrete changes, such as staging a tool, changing the order of two steps, or tightening the definition of when a unit should go to rework. Run with those changes for a short period, then repeat the same review using the same card. Over a few cycles, this pattern makes it clear that the KPI is there to support better work, not to encourage moving effort into places where it is not measured.

How Material Model Can Help

Material Model exists to make mechanism measurements cheap and fast enough to use in everyday engineering work. Most plants already have video of their stations from phones, GoPros, or fixed cameras, but turning that footage into element-level timelines, value-add and non-value-add breakdowns, and clean tables usually takes hours of manual work, which is why detailed studies are often delayed or skipped.

With Material Model, those same clips can be uploaded with a few plain-English instructions to get back a first draft of the complete study in just minutes.

The outputs are designed to match the way industrial engineers already work. You get timelines that line up with standard work combination tables, motion-level breakdowns that correspond to MODAPTS or similar systems, and easy to use spreadsheets.