Garbage in, garbage out? Measuring error rates to get ready for AI

Key Insights

• •

  Errors in data collection and user input are foundational issues that compromise AI and service outcomes.

• •

  Users often 'fudge' answers due to ambiguous questions, privacy concerns, or to achieve a desired outcome.

• •

  Non-uptake, where users abandon a form or process, is a major source of error but is rarely published or measured.

• •

  Mistakes can be categorized as problems along the way, wrong results, unnecessary actions, and delayed impact issues.

• •

  Multiple accounts creation often occurs due to users forgetting existing accounts, leading to data duplicates and service inefficiencies.

• •

  Measuring error rates is complex; different metrics (per person, per attempt, completion vs. start) yield different perspectives.

• •

  Elections provide a useful model for measuring data quality, using turnout, participation, and eligibility rates.

• •

  Data quality deteriorates over time due to changes like moving, name changes, loss of documents, or organizational restructuring.

• •

  AI initiatives can provide a compelling rationale and funding opportunity for improving longstanding data quality problems.

• •

  Frameworks like the UK Government Data Quality Framework help organizations systematically assess and address data issues.

Notable Quotes