Example of Pandas method to count null values ​​in each row of data

Processing missing data (NaN values) is a very common problem during data analysis and processing. Missing data affects the accuracy of the analysis results, so during the data cleaning phase, we usually need to count and process these missing values. Pandas provides a range of methods to process and analyze missing data. This article will describe how to use Pandas to count null values ​​in each row of data.

What is a null value?

In Pandas, null values ​​are usually represented by NaN (Not a Number). Null values ​​can appear in any data type, including numeric values, strings, dates, etc. The null value may be caused by incomplete data acquisition, error in data entry, or other reasons.

Why count empty values?

The purpose of null values ​​is to understand the integrity of the data and help us decide how to deal with these missing values. We can choose to delete rows or columns with a large number of missing values, or we can choose to fill these missing values ​​with other values ​​such as mean, median, or specific values.

Preparation

First, we need to install the Pandas library. If you haven't installed it, you can use the following command to install it:

pip install pandas

Create sample data

We will create an example DataFrame with some null values ​​for demonstration.

import pandas as pd
import numpy as np

# Create a sample DataFramedata = {
    'Name': ['Alice', 'Bob', 'Charlie', 'David', 'Eva'],
    'Age': [24, , 22, , 28],
    'City': ['New York', 'Los Angeles', , 'Chicago', 'Houston'],
    'Score': [85, 92, , 70, ]
}

df = (data)
print("Raw Data:")
print(df)

Output:

Raw data:
       Name   Age         City  Score
0     Alice  24.0     New York   85.0
1       Bob   NaN  Los Angeles   92.0
2   Charlie  22.0          NaN    NaN
3     David   NaN      Chicago   70.0
4       Eva  28.0      Houston    NaN

Count the number of empty values ​​per row

Use the isnull() method to detect null values ​​in DataFrame and return a Boolean DataFrame where True represents null values ​​and False represents non-null values. Then use sum(axis=1) to count the number of empty values ​​for each row.

# count the number of empty values ​​in each rowdf['Missing Values'] = ().sum(axis=1)
print("Number values ​​per line:")
print(df)

Output:

Number of null values ​​per row:
       Name   Age         City  Score  Missing Values
0     Alice  24.0     New York   85.0               0
1       Bob   NaN  Los Angeles   92.0               1
2   Charlie  22.0          NaN    NaN               2
3     David   NaN      Chicago   70.0               1
4       Eva  28.0      Houston    NaN               1

Further analysis

With the number of null values ​​per row, we can further analyze the integrity of the dataset. For example, we can filter out rows with more null values ​​for further processing.

# Filter out rows with null valuesrows_with_missing_values = df[df['Missing Values'] > 0]
print("Line with null value:")
print(rows_with_missing_values)

Output:

Lines with null values:
       Name   Age         City  Score  Missing Values
1       Bob   NaN  Los Angeles   92.0               1
2   Charlie  22.0          NaN    NaN               2
3     David   NaN      Chicago   70.0               1
4       Eva  28.0      Houston    NaN               1

Handle empty values

There are many ways to deal with empty values, and the specific methods depend on business requirements and data characteristics. Common treatment methods include:

Delete rows with null values:

df_dropped = ()
print("Delete the data after the row with null values:")
print(df_dropped)

Fill in empty values:
The empty values ​​can be filled with mean, median, mode, or other specific values. For example, fill in the empty value with the mean of the column:

df_filled = (())
print("Fill in the data after null values:")
print(df_filled)

Summarize

Statistics and processing missing data are an important step in data analysis and processing. Through the features provided by Pandas, we can easily count the null values ​​in each row of data and select the appropriate method to handle these null values ​​according to the specific situation. Hope this article helps you better understand and apply Pandas to process missing data.

This is the article about the example method of Pandas stating null values ​​in each row of data. For more related content on Pandas stating null values ​​in each row, please search for my previous article or continue browsing the related articles below. I hope everyone will support me in the future!