Filtering Combine Flow Delay and Ramp-Up Errors

One-sentence answer: measure the delay with a clean stop, convert it to a sample count using each pass’s own interval, shift yield and moisture backwards within the pass, then drop the first and last several seconds and anything recorded below walking pace.

Context

Grain cut at the header reaches the mass-flow sensor ten to twenty seconds later, after travelling through the threshing and cleaning system. Until that offset is removed, every yield value is attributed to ground the combine reached afterwards — at 6 km/h and a twelve-second delay, twenty metres down-track. On a field with 30 m management zones, that is most of a zone width, and it displaces every boundary in the same direction, which is why the resulting prescription is confidently applied to the wrong ground.

The end-of-pass artefacts compound it: the seconds while the threshing system fills read low, the seconds while it empties read high, and the pause to unload reads as enormous yield over almost no area. This guide implements the first two stages of yield monitor data cleaning and telemetry QA.

Measuring the delay from a clean stop A mass flow trace along one pass. Flow ramps up over the first eight seconds after entering the crop, holds steady through the pass, and after the header leaves the crop at the stop point continues for about twelve seconds before decaying to zero. The twelve-second tail is labelled as the grain flow delay to be measured. flow time along the pass (seconds) ramp-up ≈ 8 s — discard header leaves the crop tail = flow delay ≈ 12 s — this is the number steady state — the only part that measures the crop Shift yield back by the tail length, then trim both shaded regions.

Prerequisites

Beyond the parent topic’s stack: telemetry with mass flow or wet yield, moisture, ground speed and a timestamp, projected to metres and split into passes.

Step-by-step

1. Measure the delay once per machine and crop.

What twelve seconds of flow delay costs at working speeds Bars showing how far a twelve-second grain flow delay displaces yield readings at three working speeds: about thirteen metres at four kilometres an hour, twenty at six and twenty-seven at eight. Correcting the delay removes the displacement at any speed. 4 km/h in heavy crop 13 m displacement at 12 s 6 km/h, typical 20 m — most of a zone width 8 km/h in light crop 27 m down-track Corrected, any speed 0 m — the shift is per pass Metres of down-track displacement. Twenty metres is comparable to a management zone's width, and the error runs the same way on every pass.

2. Convert to samples using the pass’s own median interval — monitors do not all log at 1 Hz.

3. Shift yield and moisture backwards within each pass.

4. Trim the ramp regions and near-stationary samples.

5. Verify on a headland.

PYTHON 
import geopandas as gpd
import numpy as np
import pandas as pd


def measure_flow_delay(stop_test: pd.DataFrame, flow_col: str = "flow_kg_s",
                       stop_ts: pd.Timestamp = None, floor_frac: float = 0.05) -> float:
    """Seconds of flow after the header leaves the crop, from a clean stop test."""
    after = stop_test.loc[stop_test["ts"] >= stop_ts].sort_values("ts")
    assert not after.empty, "no samples after the stop timestamp"
    steady = float(stop_test.loc[stop_test["ts"] < stop_ts, flow_col].median())
    assert steady > 0, "no steady-state flow before the stop — wrong test window"

    decayed = after.loc[after[flow_col] <= floor_frac * steady]
    assert not decayed.empty, "flow never decayed — extend the recording after the stop"
    delay = (decayed["ts"].iloc[0] - stop_ts).total_seconds()
    assert 3.0 < delay < 40.0, f"measured delay {delay:.1f}s is implausible — check the stop time"
    return float(delay)


def apply_flow_delay(g: gpd.GeoDataFrame, delay_s: float,
                     value_cols=("yield_wet", "moisture")) -> gpd.GeoDataFrame:
    """Shift measured values backwards within each pass by the flow delay."""
    out = []
    for pass_id, chunk in g.groupby("pass_id", sort=False):
        c = chunk.sort_values("ts").copy()
        interval = c["ts"].diff().dt.total_seconds().median()
        assert interval and interval > 0, f"pass {pass_id}: cannot determine the sample interval"
        shift = int(round(delay_s / interval))
        assert shift < len(c) // 2, (
            f"pass {pass_id}: delay of {delay_s}s is {shift} samples but the pass is only "
            f"{len(c)} — the pass is too short to correct and should be dropped")
        for col in value_cols:
            c[col] = c[col].shift(-shift)
        out.append(c)
    res = pd.concat(out).dropna(subset=list(value_cols))
    print(f"flow-delay shift removed {1 - len(res) / len(g):.1%} of samples from pass ends")
    return gpd.GeoDataFrame(res, geometry="geometry", crs=g.crs)


def trim_ends(g: gpd.GeoDataFrame, ramp_s: float = 8.0,
              min_speed_m_s: float = 0.6) -> gpd.GeoDataFrame:
    """Drop the filling and emptying seconds of each pass, and any near-stationary samples."""
    keep = []
    for _pass_id, chunk in g.groupby("pass_id", sort=False):
        c = chunk.sort_values("ts")
        t = (c["ts"] - c["ts"].iloc[0]).dt.total_seconds()
        span = float(t.iloc[-1])
        if span < 4 * ramp_s:
            continue                       # too short to have a usable middle
        keep.append(c[(t > ramp_s) & (t < span - ramp_s) & (c["speed_m_s"] > min_speed_m_s)])
    out = pd.concat(keep)
    removed = 1 - len(out) / len(g)
    print(f"end trimming removed {removed:.1%} of samples")
    assert removed < 0.40, f"trimming removed {removed:.0%} — passes are too short for these settings"
    return gpd.GeoDataFrame(out, geometry="geometry", crs=g.crs)

Inline verification — the headland test, which shows whether the shift went the right way and far enough:

PYTHON 
inner = cleaned[cleaned.geometry.within(boundary.buffer(-40))]      # field body
edge = cleaned[~cleaned.geometry.within(boundary.buffer(-40))]      # headland ring

ratio = float(edge["yield_wet"].median() / inner["yield_wet"].median())
print(f"headland/interior median yield ratio: {ratio:.3f}")
assert 0.80 < ratio < 1.10, (
    f"headland yield is {ratio:.2f}× the interior — a ratio well below 0.9 usually means the "
    "flow delay is under-corrected and interior grain is still being credited to the headland; "
    "above 1.1 suggests over-correction")

Headlands genuinely yield a little less than the field body from compaction and turning, so a ratio around 0.9–0.95 is expected. A ratio of 0.6 is a data artefact, not agronomy.

Settings worth being deliberate about

Setting Default here Why it matters
Flow delay measured, ~12 s in maize Every second of error displaces yield by the machine’s travel in that second — 1.7 m at 6 km/h, and the error is systematic along every pass
Decay floor 5% of steady flow Defines when the tail has ended in the stop test. A higher floor shortens the measured delay; a lower one waits for sensor noise
Ramp trim 8 s each end Covers the filling and emptying of the threshing system. Under-trimming leaves a low collar around every pass start
Minimum speed 0.6 m/s Removes unloading pauses and stationary samples that report yield over almost no area
Minimum pass span 4× the ramp Below it there is no usable middle after trimming, and the pass should be dropped explicitly rather than silently emptied
Shift scope within a pass A global shift moves the end of one pass onto the start of the next, often at the opposite end of the field

Measure the delay per machine and crop each season. It changes with threshing and cleaning settings as well as with the machine, and a fleet-wide default reintroduces most of the error the correction exists to remove.

Gotchas and edge cases

  • The delay is machine, crop and settings specific. A small-grain machine clears faster than a large-frame combine in maize, and changing the fan or sieve settings changes the transit time. Measure once per machine-crop combination each season; adopting a single default across a mixed fleet reintroduces the error you are correcting.
Reading the headland ratio to check the correction Three panels showing how the ratio of headland to interior yield reveals whether the flow delay correction is right. Under-correction leaves the ratio well below 0.9, a correct shift lands it around 0.90 to 0.95 where compaction explains the difference, and over-correction pushes it above 1.1. Under-corrected delay Shift smaller than the true transit time Interior grain still credited to the headland. Headland yield reads low on every pass. Ratio to interior falls well below 0.9. Correctly measured delay Shift matches the stop-test tail Headland reads slightly below interior. That difference is compaction and turning. Ratio sits around 0.90 to 0.95. Over-corrected delay Shift longer than the transit time Standing-crop yield dragged onto cut ground. Headland reads high, interior low. Ratio above 1.1.

  • Shifting across a pass boundary corrupts both passes. Shifting must happen within a pass, which is why pass reconstruction comes first. A global shift over the whole file moves the last samples of one pass onto the first of the next — often at the opposite end of the field.

  • Log intervals vary. Some monitors log at 1 Hz, some at 0.2 Hz, some irregularly under load. Deriving the sample shift from each pass’s own median interval, as above, is what makes the same delay value correct across machines.

  • Short passes cannot be corrected. A point row of forty samples has no steady middle after a twelve-second shift and an eight-second trim at each end. Drop them explicitly and count them rather than letting a negative-length slice produce an empty frame silently.

  • Trimming is not outlier removal. These samples are not extreme values to be filtered statistically — they are measurements of a machine state rather than of a crop. Removing them by percentile would take genuinely high- and low-yielding ground with them.

This guide is part of Yield Monitor Data Cleaning & Telemetry QA — see there for width correction, moisture normalisation and calibration.

Parent section Yield Monitor Data Cleaning & Telemetry QA Related Removing Yield Outliers with Neighborhood Statistics