Table of Contents
In python, nan is a special value used to represent missing values. In this article, we will discuss the properties of nan values. We will also look at different ways how we can remove nan values from list in python.
What is nan in python?
The nan values are special floating-point numbers defined in the math module and the numpy module in python. They are normally used to represent missing values in a given data.
You can import and use nan values from the math module as follows.
|
1 2 3 4 5 6 |
from math import nan x = nan print("Value of x is:", x) |
Output:
|
1 2 3 |
Value of x is: nan |
Alternatively, you can import nan values from the numpy module as follows.
|
1 2 3 4 5 6 |
from numpy import nan x = nan print("Value of x is:", x) |
Output:
|
1 2 3 |
Value of x is: nan |
The nan values have a special property that two nan values are not equal to one another. If you will compare nan values for equality, the result will always be False. You can observe this in the following example.
|
1 2 3 4 5 6 |
from numpy import nan x = nan print("x is equal to x?", x == x) |
Output:
|
1 2 3 |
x is equal to x? False |
On the contrary, two nan values are always considered unequal to one another. You can observe this in the following example.
|
1 2 3 4 5 6 |
from numpy import nan x = nan print("x is equal to x?", x != x) |
Output:
|
1 2 3 |
x is equal to x? True |
How to remove nan from list in python?
As seen above, we cannot compare nan values to remove them from list. So, we will have to find some alternative ways.
To check for nan values, we can convert all the elements in the list to string and then compare the string “nan” with the string values of the elements in the list. After that, we can remove the nan values from the list.
Alternatively, there are various in-built methods defined in the math module, numpy module, and the pandas module to check for nan values. We will discuss all these approaches to check for nan value and remove them from list in the next sections.
Remove nan from list using the str() function
The most naive way to remove nan values from list is by using the str() function. The str() function is used to convert an object with any data type to a string. The str() function takes an object as an input argument and returns its string representation.
To remove nan values from list, we will first convert the value nan to a string and create an empty list newList to store the output. After that, we will traverse each element in the input list using a for loop, convert them to a string, and check if it is equal to the string “nan”. If the string representation of the element is not equal to “nan”, we will append the element to the newList using the append() method. The element whose string representation equals “nan” will be a nan value and will not be added to the newList.
You can observe the entire process in the following example.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
from numpy import nan myList = [nan, 1, "java2blog", 123, 123.5, nan] print("The original list is:", myList) newList = list() for element in myList: if str(element) != "nan": newList.append(element) print("Output list is:", newList) |
Output:
|
1 2 3 4 |
The original list is: [nan, 1, 'java2blog', 123, 123.5, nan] Output list is: [1, 'java2blog', 123, 123.5] |
Instead of a for loop, we can also use list comprehension with the str() function to remove the nan values from list in python as follows.
|
1 2 3 4 5 6 7 8 9 |
from numpy import nan myList = [nan, 1, "java2blog", 123, 123.5, nan] print("The original list is:", myList) newList = [element for element in myList if str(element) != "nan"] print("Output list is:", newList) |
Output:
|
1 2 3 4 |
The original list is: [nan, 1, 'java2blog', 123, 123.5, nan] Output list is: [1, 'java2blog', 123, 123.5] |
Further reading:
Remove nan from list using math.isnan() function
The math module in python provides us with the isnan() function with which we can check if a value is a nan value or not. The isnan() function takes a floating-point number or a number that can be converted into a floating-point number as an input argument and returns True if the input is nan. Otherwise, it returns False. You can observe this in the following example.
|
1 2 3 4 5 6 7 8 9 10 11 |
from math import nan import math x = nan y = 123 print("x is:", x) print("y is:", y) print("x is nan?", math.isnan(x)) print("y is nan?", math.isnan(y)) |
Output:
|
1 2 3 4 5 6 |
x is: nan y is: 123 x is nan? True y is nan? False |
However, if the input argument cannot be changed into a floating-point number, the function will raise the TypeError exception as shown below.
|
1 2 3 4 5 6 7 |
from math import nan import math x = "java2blog" print("x is nan?", math.isnan(x)) |
Output:
|
1 2 3 4 5 6 |
Traceback (most recent call last): File "/home/aditya1117/PycharmProjects/pythonProject/string1.py", line 5, in <module> print("x is nan?", math.isnan(x)) TypeError: must be real number, not str |
You should understand that we cannot use math.isnan() function to check for nan values in a list if it has string values in it because it will cause an error.
To remove nan values from list in python using the math.isnan() function, we will first create an empty list named newList. After that, we will traverse each element of the list using a for loop and check if it is a nan value or not using the math.isnan() function. If the element is not a nan value, we will append it to the newList using the append() method. Otherwise, we will reject the element. After traversing the entire input list, we will get all the values that are not nan values in the newList.
You can observe this in the following example.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
from math import nan import math myList = [nan, 1, 1117, 123, 123.5, nan] print("The original list is:", myList) newList = list() for element in myList: if not math.isnan(element): newList.append(element) print("Output list is:", newList) |
Output:
|
1 2 3 4 |
The original list is: [nan, 1, 1117, 123, 123.5, nan] Output list is: [1, 1117, 123, 123.5] |
Instead of a for loop, we can also use list comprehension with the math.isnan() function to remove nan values from list in python as follows.
|
1 2 3 4 5 6 7 8 9 10 |
from math import nan import math myList = [nan, 1, 1117, 123, 123.5, nan] print("The original list is:", myList) newList = [element for element in myList if not math.isnan(element)] print("Output list is:", newList) |
Output:
|
1 2 3 4 |
The original list is: [nan, 1, 1117, 123, 123.5, nan] Output list is: [1, 1117, 123, 123.5] |
Remove nan from list using numPy.isnan() function
Instead of the isnan() function defined in the math module, we can also use the isnan() function defined in the numpy module to check if an object has a value nan or not. The numpy.isnan() function works in the same way as the math.isnan() function.
|
1 2 3 4 5 6 7 8 9 10 11 |
from numpy import nan import numpy x = nan y = 123 print("x is:", x) print("y is:", y) print("x is nan?", numpy.isnan(x)) print("y is nan?", numpy.isnan(y)) |
Output:
|
1 2 3 4 5 6 |
x is: nan y is: 123 x is nan? True y is nan? False |
If the input argument cannot be converted to a floating-point value, the function will raise a TypeError exception as shown below.
|
1 2 3 4 5 6 7 |
from numpy import nan import numpy x = "java2blog" print("x is nan?", numpy.isnan(x)) |
Output:
|
1 2 3 4 5 6 |
Traceback (most recent call last): File "/home/aditya1117/PycharmProjects/pythonProject/string1.py", line 5, in <module> print("y is nan?", numpy.isnan(x)) TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'' |
You can use the numpy.isnan() function with a for loop to remove nan values from list. The approach used is similar to what we used while discussing the approach to remove nan values from list using math.isnan() function as you can see in the following example.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
from numpy import nan import numpy myList = [nan, 1, 1117, 123, 123.5, nan] print("The original list is:", myList) newList = list() for element in myList: if not numpy.isnan(element): newList.append(element) print("Output list is:", newList) |
Output:
|
1 2 3 4 |
The original list is: [nan, 1, 1117, 123, 123.5, nan] Output list is: [1, 1117, 123, 123.5] |
Instead of using a for loop, you can use the list comprehension and numpy.isnan() function to remove a nan value from list in python as follows.
|
1 2 3 4 5 6 7 8 9 |
from numpy import nan import numpy myList = [nan, 1, 1117, 123, 123.5, nan] print("The original list is:", myList) newList = [element for element in myList if not numpy.isnan(element)] print("Output list is:", newList) |
Output:
|
1 2 3 4 |
The original list is: [nan, 1, 1117, 123, 123.5, nan] Output list is: [1, 1117, 123, 123.5] |
Remove nan from list using pandas.isnull() function
Similar to math.isnan() and numpy.isnan(), the pandas module in python also provides us with the isnull() function to check for nan values. The pandas.isnull() function takes an object as an input argument and returns True if the object has nan value. Otherwise, it returns False.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
from numpy import nan import pandas x = nan y = "123" z = 123 print("x is:", x) print("y is:", y) print("x is nan?", pandas.isnull(x)) print("y is nan?", pandas.isnull(y)) print("z is nan?", pandas.isnull(z)) |
Output:
|
1 2 3 4 5 6 7 |
x is: nan y is: 123 x is nan? True y is nan? False z is nan? False |
You can observe that the isnull() function can also take strings as input arguments. So, we can use the pandas.isnull() function to remove nan values from list that may contain string values.
We will use the pandas.isnull() function to remove nan values from list using the for loop. The approach is similar to what we discussed to remove nan values from list using the math.isnan() function as you can see in the following example.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
from math import nan import pandas myList = [nan, 1, "java2blog", 123, 123.5, nan] print("The original list is:", myList) newList = list() for element in myList: if not pandas.isnull(element): newList.append(element) print("Output list is:", newList) |
Output:
|
1 2 3 4 |
The original list is: [nan, 1, 'java2blog', 123, 123.5, nan] Output list is: [1, 'java2blog', 123, 123.5] |
Instead of using a for loop, you can use the list comprehension and pandas.isnull() function to remove a nan value from list in python as follows.
|
1 2 3 4 5 6 7 8 9 |
from math import nan import pandas myList = [nan, 1, "java2blog", 123, 123.5, nan] print("The original list is:", myList) newList = [element for element in myList if not pandas.isnull(element)] print("Output list is:", newList) |
Output:
|
1 2 3 4 |
The original list is: [nan, 1, 'java2blog', 123, 123.5, nan] Output list is: [1, 'java2blog', 123, 123.5] |
Conclusion
In this article, we have discussed what is a nan value in python. We also discussed why we cannot check for a nan value using the equality operator. Finally, we discussed four ways to remove nan values from list in python. I would suggest you use math.isnan() function for checking for a nan value while removing nan values from list. This is because both numpy.isnan() and pandas.isnull() have higher execution cost compared to math.isnan(). However, if there might be string values present in the list, then you should use the approach using the str() function or the pandas.isnull() function.
I hope you enjoyed reading this article. Stay tuned for more informative articles.
Happy Learning!
