pandas unique: distinct values, order and result types

Before grouping data, I often want to see which values actually occur in a column. Are there five statuses or fifty? Do some labels differ only by a trailing space? unique gives me distinct values in their order of first appearance. It is a useful starting point for understanding an unfamiliar source.

I will use small synthetic examples. I executed them with Python 3.14.4, pandas 3.0.6 and 3.1.0, and NumPy 2.5.4, without PyArrow. Run the blocks in order. Printing lists keeps attention on the values; I inspect the result type separately because it is not always a NumPy array.

Preserve the order of first appearance

For a column represented by a Series, calling .unique() is convenient. The function pd.unique(series) also accepts a Series. Both return distinct values without automatically sorting them. That order is useful when I want to see how a source introduced its labels rather than immediately reorganizing them alphabetically or numerically.

# example: numbers
import pandas as pd
import numpy as np

numbers = pd.Series([3, 1, 2, 3, 1], dtype="int64")
result = numbers.unique()
print(result.tolist())
print(isinstance(result, np.ndarray))
print(numbers.tolist())

[3, 1, 2]
True
[3, 1, 2, 3, 1]

The source still contains five elements. unique does not delete rows from the original table. It returns a separate result for further work. If you need ascending order, make sorting a separate operation. Do not assume that another function with a similar name, including one from NumPy, follows the same ordering rules.

The Series.unique documentation describes ordering and result types. Also notice what “unique” means here: distinct values, including values that occurred repeatedly. The number 3 remains in the output despite appearing twice. This operation does not select only values that occur exactly once in the input.

Text values and missing entries

I use an explicit string dtype next. That demonstrates pandas’ extension-array behavior without relying on automatic dtype inference. Missing entries remain represented in the unique result. If I do not want them in a list of labels, I first remove missing entries from the Series used for that particular read.

# example: strings
labels = pd.Series(["web", "store", "web", pd.NA, "store", pd.NA], dtype="string")
unique_labels = labels.unique()
print(unique_labels.tolist())
print(isinstance(unique_labels, pd.api.extensions.ExtensionArray))
print(labels.dropna().unique().tolist())

['web', 'store', <NA>]
True
['web', 'store']

The result of Series.unique() depends on the dtype. Our int64 example produces an ndarray, while the string example produces an extension array. Categories and timezone-aware datetimes also have their own representations. Saying that unique always returns NumPy would hide a distinction that may matter in the next step of your program.

An empty string or a space is not the same as a missing value. I do not automatically classify either as incorrect, but I investigate how they arose. A distinct-value list helps reveal the issue; it cannot decide which labels should be merged. That decision depends on the column’s meaning and potentially on identifiers used by other systems.

Convert a Python list explicitly

In the tested pandas versions, 3.0.6 and 3.1.0, passing a plain Python list to pd.unique raises TypeError. Convert it to a Series with a chosen dtype or to a NumPy array. This also makes your intended treatment of mixed values and missing entries more explicit than leaving everything to inference.

# example: list-input
try:
    pd.unique([3, 1, 3])
except TypeError:
    print("TypeError")

print(pd.Series([3, 1, 3], dtype="int64").unique().tolist())
print(pd.unique(np.array([3, 1, 3], dtype="int64")).tolist())

TypeError
[3, 1]
[3, 1]

The exception handler demonstrates a specific test case. In ordinary code I would supply the supported object directly. Hiding every failure behind a broad except would be unhelpful: a misspelled variable and an unsupported input type require different corrections. If your data is an Index, the Index.unique() method is also available.

Choose a column or a combination of fields

A DataFrame is two-dimensional. For unique values, select a single column. If I need combinations of channel and region, I select both fields and use drop_duplicates. The unit of comparison changes: two orders can share a channel while belonging to different regions, and both combinations should remain visible.

# example: dataframe
orders = pd.DataFrame({
    "channel": ["web", "web", "store", "web"],
    "region": ["north", "south", "north", "north"],
})
print(orders["channel"].unique().tolist())
print(orders[["channel", "region"]].drop_duplicates().values.tolist())

['web', 'store']
[['web', 'north'], ['web', 'south'], ['store', 'north']]

All columns used to construct this table have the same length. An example with seven entries in one list and five in another fails during construction, before unique values can be inspected. To demonstrate different data types independently, create separate Series rather than forcing unrelated samples into a single rectangular table.

Use nunique when you only need the number of distinct values. Use drop_duplicates to choose representative rows deliberately. These operations can work together, but they answer different questions and produce different kinds of results.

Exercise: measure a normalization decision

This synthetic channel column contains different letter case, a trailing space and a missing entry. Suppose the agreed rule allows lowercasing and removing surrounding spaces. I keep the transformed result in a new variable so that I can compare it with the source and show exactly which labels have been merged.

# example: normalization
raw = pd.Series(["Web", "web", "web ", pd.NA], dtype="string")
cleaned = raw.str.strip().str.lower()
print(raw.unique().tolist())
print(cleaned.unique().tolist())
print(raw.nunique(), cleaned.nunique())

['Web', 'web', 'web ', <NA>]
['web', <NA>]
3 1

Three known labels became one. The missing entry remains missing and is excluded by the default nunique count. Before applying the same transformation elsewhere, check the source rules: letter case can carry meaning in codes. I also make no blanket performance promise against every alternative. For a large column, measure the actual workload with its dtype and number of distinct values.