Types of Missing Data

Remark

Please be aware that these lecture notes are accessible online in an ‘early access’ format. They are actively being developed, and certain sections will be further enriched to provide a comprehensive understanding of the subject matter.

2.1. Types of Missing Data#

Imagine you’re following a recipe to cook a special dinner, but when you check your pantry, you discover that several key ingredients are missing. Some ingredients might be missing because:

  • You forgot to buy them (random oversight)

  • You used them for another dish and forgot to replace them (missing based on your cooking history)

  • You deliberately avoid buying expensive ingredients (missing based on the ingredient’s characteristics)

Just like missing ingredients can ruin a dish or force you to modify your cooking approach, missing data can lead to inaccurate analysis and poor decision-making in research. Understanding why the data is missing is crucial for choosing the right approach to handle it.

Missing data is a common challenge in research and data analysis, often affecting the validity and reliability of results. Understanding the different types of missing data is crucial for selecting appropriate handling methods and ensuring accurate interpretations.

2.1.1. Introduction#

Missing data occurs when no value is stored for a variable in an observation. This can happen due to various reasons, such as equipment errors, incorrect measurements, or manual data entry procedures. The way data is missing can significantly impact the analysis and the choice of imputation methods [Baio and Leurent, 2016, Gabr et al., 2023, Venugopalan et al., 2019].

2.1.2. Types of Missing Data#

There are three primary types of missing data:

  1. Missing Completely at Random (MCAR)

  2. Missing at Random (MAR)

  3. Missing Not at Random (MNAR)

2.1.3. Missing Completely at Random (MCAR)#

Missing Completely at Random (MCAR) is one of the three primary types of missing data in statistical analysis. It represents the most favorable scenario for handling missing data, as it introduces the least amount of bias into the analysis.

MCAR occurs when there are no systematic differences between the missing values and the observed values [Heitjan and Basu, 1996, Molenberghs and Verbeke, 2005]. In other words, the probability that a value is missing is unrelated to any observed or unobserved data in the dataset. This means that the missingness is entirely random and does not depend on any variables or characteristics of the data.

Characteristics of MCAR

  1. Unbiased nature: The missing data mechanism does not depend on the values of any variables, whether observed or missing [Heitjan and Basu, 1996].

  2. Equal probability: Each data point has an equal chance of being missing, regardless of its value or the values of other variables.

  3. Least problematic: Among the types of missing data, MCAR is considered the least problematic for statistical analysis [Molenberghs and Verbeke, 2005].

Understanding these characteristics is crucial for researchers to identify MCAR in their datasets. However, determining whether data truly meets MCAR criteria can be challenging. To address this, researchers have developed methods to test for MCAR.

Testing for MCAR

Determining whether data is MCAR can be challenging, but there are methods to test this assumption:

  1. Little’s MCAR test: This statistical test, developed by Roderick J. A. Little in 1988, can be used to assess whether the data is MCAR. If the test result is not significant, it suggests that the MCAR assumption may be valid [Li, 2013, Little, 1988].

  2. Covariance matrix analysis: Recent research has proposed methods for testing MCAR based on sample covariance matrices, particularly useful for high-dimensional datasets [Bordino and Berrett, 2024].

Once MCAR is established or assumed, it has specific implications for data analysis. These implications guide researchers in choosing appropriate statistical methods and interpreting results.

Implications for Analysis

When data is MCAR:

  1. Complete case analysis: Simple methods like complete case analysis (listwise deletion) may be appropriate, as they are less likely to introduce bias [Dolan et al., 2005].

  2. Power considerations: While MCAR data doesn’t introduce bias, it can still affect statistical power. Researchers should consider this when planning sample sizes and conducting power analyses [Dolan et al., 2005].

  3. Imputation methods: Although simpler methods can be used, more sophisticated imputation techniques can still improve efficiency and statistical power.

Despite its advantages, MCAR has limitations and considerations that researchers must keep in mind when working with missing data.

Limitations and Considerations

  1. Rarity in practice: True MCAR is relatively rare in real-world scenarios. Often, data that appears to be MCAR may actually be Missing at Random (MAR) or Missing Not at Random (MNAR) [Heitjan and Basu, 1996].

  2. Assumption verification: It’s crucial to verify the MCAR assumption before proceeding with analysis, as violations can lead to biased results [Heitjan and Basu, 1996].

  3. Impact on analysis: Even if data is MCAR, the loss of data can still affect the precision of estimates and reduce statistical power [Heitjan and Basu, 1996].

To better illustrate MCAR, let’s consider some real-world examples:

Example 2.1 (Examples of MCAR)

  1. Equipment malfunction: Equipment failure in a clinic can cause missing data, as in the case of an MRI scanner experiencing machine failure [van Loon et al., 2024].

  2. MRI Scanner Malfunction: In a clinical setting, an MRI scanner unexpectedly breaks down, resulting in missing data for patients scheduled during the downtime. The failure is purely mechanical and unrelated to patient characteristics [van Loon et al., 2024].

  3. Data entry errors: In a large-scale survey, some responses are accidentally not entered into the database due to random human error. The probability of a response being missed is the same for all participants and unrelated to their characteristics or responses.

  4. Sample loss: In a longitudinal study tracking plant growth, some plants are accidentally knocked over and destroyed by a researcher. The chance of a plant being destroyed is equal for all plants and unrelated to their growth rate or any other variable being studied [Colin-Chevalier et al., 2022].

  5. Random survey non-response: In a community health survey, some participants randomly forget to answer certain questions. The likelihood of skipping a question is the same for all participants and unrelated to their health status or any other factor [Colin-Chevalier et al., 2022].

  6. Randomized missing data in clinical trials: In a drug efficacy study, researchers deliberately omit a random subset of data points to test the robustness of their analysis methods. The omissions are truly random and not related to any participant characteristics or study variables [Smith and Elkan, 2004].

Understanding MCAR is essential for researchers and data analysts, as it informs the choice of missing data handling techniques and helps in assessing the potential impact on study results. While MCAR represents the ideal scenario for missing data, it’s important to carefully evaluate whether this assumption holds in practice and consider alternative approaches when it doesn’t.

2.1.4. Missing at Random (MAR)#

Missing at Random (MAR) is a crucial concept in the field of data analysis and statistics, representing one of the three main types of missing data mechanisms. It is more common than Missing Completely at Random (MCAR) and often more realistic in real-world scenarios.

MAR occurs when the probability of missing data on a variable Y depends on other observed variables in the dataset, but not on the values of Y itself (after controlling for the other variables). In other words, the missingness is related to observed data but not to unobserved data.

Characteristics of MAR

  1. Conditional randomness: The missingness is random once you account for the observed data.

  2. Predictability: The pattern of missing data can be predicted from other variables in the dataset.

  3. Partial observability: While the missingness is related to observed variables, it’s not related to the unobserved values of the variable with missing data.

To better understand MAR, it’s helpful to look at practical examples that illustrate how missingness depends on observed data.

Example 2.2 (Examples of MCAR)

  1. Age and Work Experience in Clinical Trials: In clinical studies, older patients or those with less work experience may be more prone to having missing data for variables like blood pressure or salary. This missingness is explainable by observed covariates (age, work experience) and can be addressed through imputation or modeling approaches [Dale et al., 2022].

  2. Age-Related Income Data Missingness: A study on job applicants reveals that the likelihood of missing income data varies with respondents’ age. Both older and younger applicants show a higher tendency to have incomplete income information [Guo et al., 2024].

Education Level Affecting Income Disclosure: Survey data indicates that respondents with higher education levels are less likely to report their income. Importantly, the probability of missing income data is not correlated with the actual income values themselves [Wilson and Lueck, 2014].

  1. Age as a Predictor of Income Missingness: In income surveys, older participants demonstrate a lower propensity to disclose their income, making age a significant predictor of data missingness.

  2. Symptom Severity Influencing Assessment Completion: Medical studies show that patients experiencing more severe symptoms are more likely to complete all assessments. Consequently, symptom severity becomes a predictor of missingness for other variables in the study.

These examples highlight how MAR differs from MCAR by introducing a dependency on observed factors. However, identifying MAR in practice can be challenging due to its reliance on unobserved data mechanisms.

Testing and Identifying MAR

  1. Difficulty in testing: It’s challenging to definitively test for MAR because it involves unobserved data [Little, 1988].

  2. Comparison with observed data: Researchers can examine relationships between missingness and observed variables to support the MAR assumption.

  3. Sensitivity analysis: Conducting sensitivity analyses can help assess the robustness of results under different missing data assumptions.

Once MAR is assumed or identified, it has significant implications for how researchers handle and analyze their data. These implications guide decisions about which statistical methods are most appropriate.

Implications for Analysis

When data is MAR [Briggs et al., 2003]:

  1. Bias reduction: Unlike MCAR, simple methods like complete case analysis can lead to biased results under MAR.

  2. Imputation methods: More sophisticated imputation techniques, such as multiple imputation or maximum likelihood estimation, are often necessary and can provide unbiased estimates.

  3. Auxiliary variables: Including variables related to the missingness mechanism in the analysis can help reduce bias and improve efficiency.

Despite its advantages over MNAR, MAR comes with its own limitations that researchers must consider during analysis and interpretation.

Limitations and Considerations

  1. Assumption strength: MAR is a stronger assumption than MCAR but weaker than Missing Not at Random (MNAR) [Baraldi and Enders, 2010].

  2. Practical implications: In practice, the distinction between MAR and MNAR can be subtle and often requires subject-matter expertise to determine.

  3. Model complexity: Handling MAR data often requires more complex statistical models and imputation techniques compared to MCAR [Austin et al., 2021].

Understanding these limitations underscores why researchers must carefully evaluate whether their data meets the MAR assumption before proceeding with analysis. Despite these challenges, MAR remains an important framework for addressing missingness in many real-world datasets.

Importance in Research

Understanding MAR is crucial for several reasons:

  1. It guides the selection of appropriate missing data handling techniques.

  2. It helps researchers assess potential biases in their analyses.

  3. It informs the design of data collection procedures to minimize the impact of missing data.

In conclusion, Missing at Random represents a common and important missing data mechanism in research and data analysis. While it presents challenges compared to MCAR, understanding MAR and applying appropriate techniques can lead to more accurate and reliable results in the presence of missing data. By leveraging observed variables that predict missingness, researchers can mitigate biases and improve their analyses’ robustness and validity.

2.1.5. Missing Not at Random (MNAR)#

Missing Not at Random (MNAR) is the most complex and challenging type of missing data mechanism. It represents a situation where the missingness is related to unobserved data, making it particularly difficult to handle in statistical analyses.

MNAR occurs when the probability of missing data on a variable Y depends on the unobserved values of Y itself, even after controlling for other observed variables. In other words, the reason for the missingness is directly related to the information that is missing [Pedersen et al., 2017].

Characteristics of MNAR

  1. Non-ignorable missingness: The missing data mechanism cannot be ignored and must be explicitly modeled to obtain unbiased estimates.

  2. Systematic pattern: There is a systematic relationship between the propensity of missing values and the missing values themselves.

  3. Potential for severe bias: MNAR can lead to significant bias in analyses if not properly addressed.

To better understand MNAR, let’s consider some real-world examples that illustrate how missingness can be related to unobserved data:

Example 2.3 (Examples of MNAR)

  1. Disease Status Influencing Data Recording: The presence of a specific disease may increase the likelihood of its status being recorded due to associated medical follow-ups and tests. This creates a direct link between the unobserved disease status and the probability of missingness [Holovchak et al., 2024].

  2. Depression Severity Affecting Follow-up Participation: In a longitudinal study on depression, individuals with severe depression might be less likely to complete follow-up assessments. This results in missingness directly related to the unobserved depression scores.

  3. Latent Variable Influence: Unobserved variables like socioeconomic status (SES) can simultaneously affect the missingness of disease indicators and treatment choices. This creates a complex MNAR scenario where the missing data mechanism is tied to unobserved factors [Holovchak et al., 2024].

  4. Income-Related Self-Censoring: In salary surveys, high-income earners may be more inclined to withhold their income information. This creates a direct relationship between the likelihood of missingness and the unobserved income values.

  5. Self-Censoring in Smoking Studies: Participants who relapse into smoking may be more likely to drop out of a smoking cessation study. This results in missing data on smoking status that is directly caused by the unobserved smoking behavior [Goldberg et al., 2021].

These examples highlight the challenge of MNAR: the very data we’re missing could be crucial for understanding why it’s missing. This characteristic has significant implications for data analysis.

Implications for Analysis

When data is MNAR:

  1. Complexity in handling: Standard imputation methods and complete case analysis are generally inadequate and can lead to biased results.

  2. Model-based approaches: Specialized techniques that explicitly model the missing data mechanism are often required.

  3. Sensitivity analysis: Due to the difficulty in verifying MNAR, sensitivity analyses are crucial to assess the impact of different missing data assumptions on results.

One of the primary challenges with MNAR is its identification. Unlike MCAR and MAR, which can be tested or inferred from observed data, MNAR involves unobserved information, making it particularly elusive.

Challenges in Identifying MNAR

  1. Unobservability: By definition, MNAR involves unobserved data, making it impossible to definitively test for MNAR using only the available data.

  2. Distinction from MAR: In practice, it can be challenging to distinguish between MAR and MNAR without additional information or assumptions.

  3. Subject-matter expertise: Identifying MNAR often requires in-depth knowledge of the subject matter and the data collection process.

Given these challenges, researchers have developed specialized methods to handle MNAR data:

Methods for Handling MNAR

  1. Selection models: These models jointly specify the distribution of the complete data and the missing data mechanism.

  2. Pattern-mixture models: These approaches stratify the data based on missing data patterns and model each stratum separately.

  3. Shared-parameter models: These models assume that the missing data mechanism and the outcome process share some common parameters.

Understanding MNAR is not just a statistical concern; it has significant implications for research integrity and validity.

Importance in Research

Understanding MNAR is critical because:

  1. It highlights the potential for severe bias in analyses that ignore the missing data mechanism.

  2. It emphasizes the need for careful consideration of the reasons behind missing data during study design and data collection.

  3. It underscores the importance of collecting auxiliary information that might be related to the missingness mechanism.

Despite its importance, working with MNAR data comes with several limitations and considerations that researchers must keep in mind:

Limitations and Considerations

  1. Untestable assumptions: The MNAR mechanism often relies on untestable assumptions about the relationship between missingness and unobserved data.

  2. Model sensitivity: Results can be highly sensitive to the chosen model for the missing data mechanism.

  3. Complexity: Handling MNAR data often requires advanced statistical expertise and specialized software.

In conclusion, Missing Not at Random represents the most challenging scenario in missing data analysis. While it poses significant difficulties, recognizing the potential for MNAR and employing appropriate analytical strategies are crucial for maintaining the integrity and validity of research findings in the presence of missing data. Researchers must approach MNAR with caution, leveraging subject-matter expertise, sensitivity analyses, and advanced statistical techniques to mitigate its potential impact on study conclusions.

Contents