Lead Intake / Foreclosure-Notice Scrape

PURPOSE

Pull the day's fresh foreclosure notices before anyone else does, turn raw notices into clean working leads, and make sure nothing gets knocked twice or missed. Speed to lead is the whole game — being first to a same-day notice is worth more than almost anything else in the pipeline.

WHEN TO USE

Every morning before the day's route is built (see sop-kod-daily-route). Also any time a source has been down and needs a catch-up pass.

PREREQUISITES

• Access to our foreclosure-notice sources set up and working.
• Target cities/counties list confirmed current (the standing Rialto-area rotation across San Bernardino / Riverside counties).
• The intake destination (our system) reachable so new notices can land.

STEPS

1. Pull today's fresh notices from our sources for the target cities/counties. Favor free, real-time sources over paid lists.
2. Parse each notice into clean fields — address, auction date, status, case type — so a source changing its layout never breaks the pipeline.
3. Dedup against what we already have (by address, with parcel number as a fallback). A match is the same case, not a new lead.
4. Create the lead / property / case for anything genuinely new, and stamp when it was last seen so later passes can spot changes.
5. On a repeat notice, check for a status change:
   • SOLD → take it off the active route (the surplus/redemption path in sop-kod-stop-the-sale still applies; it just leaves daily work).
   • CANCELLED → move to EXPIRED / re-engagement, paused ~14 days first.
   • POSTPONED → keep it active; cadence does not change.
6. Send new addresses on to enrichment (owner, equity, phones happen there, not here — see sop-kod-lead-enrichment).
7. Watch for the same-day alert on anything new, gold-bucketed, or newly changed — that is the signal a fresh notice needs a same-day door knock, not a future call bucket.

VERIFICATION

• The day's sources were pulled (or a documented fallback was used for any that were down).
• Every new notice produced exactly one lead/property/case — spot-check for duplicates on a shared address.
• Any case that changed status shows the correct new state.
• New or gold-bucket leads fired a same-day alert.

TROUBLESHOOTING

• A source is down or blocking. Don't force it — use a fallback source for the same underlying data and move on.
• The same property keeps creating a "new" lead. The dedup is address-first with parcel-number fallback; normalize the address (unit numbers, abbreviations) if slight variations slip through.
• A must-hit lead didn't get a same-day alert. Treat a missing same-day alert as a gap to flag — same-day is the entire point of this step.
• Paid-list data looks stale. Paid lists can run ~2 weeks behind, which is dangerous in short-timeline states. Check any paid lead against a live source before trusting it.