How pandas apply works in Python? Best example

How pandas apply works in Python? Best example

When working with data in Python, especially in pandas, one of the most useful methods available is apply(). It enables you to apply a function along an axis of a DataFrame or to each element of a Series. This method provides flexibility and efficiency, making it an essential tool for data manipulation and transformation.

Understanding pandas apply()

The apply() method allows us to perform custom operations on data stored in a DataFrame or a Series. Instead of using a loop to iterate through rows or columns, apply() enables us to apply a function much more efficiently.

The basic syntax of apply() is as follows:


DataFrame.apply(func, axis=0, raw=False, result_type=None, args=(), **kwds)
  • func: The function to apply to each row or column.
  • axis: Defines whether to apply the function along rows (1) or columns (0).
  • raw: If set to True, passes ndarray instead of Series.
  • result_type: Determines the return type: “expand”, “reduce”, or “broadcast”.
  • args: Additional positional arguments passed to the function.

Applying a Function to a pandas Series

The simplest use case for apply() is applying a function to a pandas Series. Below is an example where we apply a function to square each value in a column:


import pandas as pd  

# Create a Series
data = pd.Series([1, 2, 3, 4, 5])

# Apply a function to the Series
squared_values = data.apply(lambda x: x ** 2)

print(squared_values)

In this example, the function lambda x: x ** 2 squares each value in the Series.

Using apply() on a DataFrame

When working with DataFrames, apply() can function along rows or columns, depending on the axis parameter.


# Create a DataFrame
df = pd.DataFrame({
    'A': [1, 2, 3],
    'B': [4, 5, 6]
})

# Apply a function along columns
df_result = df.apply(lambda x: x + 10, axis=0)

print(df_result)

The above example adds 10 to each element in the DataFrame.

Row-wise Operations with apply()

Sometimes, we may want to combine values from multiple columns in a row. Setting axis=1 allows us to apply a function row-wise.


# Apply a function along rows
df['Sum'] = df.apply(lambda row: row['A'] + row['B'], axis=1)

print(df)

Here, we create a new column Sum that contains the sum of columns ‘A’ and ‘B’.

Using apply() with Custom Functions

We can also use apply() with defined functions instead of lambdas.


# Define a function
def multiply_values(x):
    return x * 2

# Apply the function
df_result = df.apply(multiply_values)

print(df_result)

This multiplies every element in the DataFrame by 2.

Returning Multiple Values

Applying functions that return multiple values can be useful, and using result_type="expand" ensures that the result is properly formatted as separate columns.


# Define a function that returns multiple values
def split_values(x):
    return x, x**2, x**3

# Apply the function and expand results into columns
df_result = df['A'].apply(split_values).apply(pd.Series)

df_result.columns = ['Original', 'Squared', 'Cubed']

print(df_result)

Performance Considerations

While apply() is powerful, it can be slower than vectorized operations available in pandas. If possible, leverage built-in functions such as df['column'] * 2 instead of using apply() for simple calculations.

Example of apply() in Data Cleaning

One practical use case for apply() is data cleaning, where we may want to format or manipulate string data.


# Create a DataFrame with messy text
df = pd.DataFrame({
    'Names': [' Alice ', 'BOB', ' Charlie']
})

# Apply a function to clean names
df['Cleaned Names'] = df['Names'].apply(lambda x: x.strip().title())

print(df)

Comparison Table: apply() vs. Other Methods

Method Use Case Performance
apply() Custom transformations Medium (depends on complexity)
Vectorized operations Calculations (+, -, *, /) High (best for large data)
map() Element-wise transformations on Series High
applymap() Element-wise transformations on DataFrame Medium

Final Thoughts

The apply() function in pandas is an essential tool for applying custom functions to a DataFrame or Series. Whether for simple transformations, data cleaning, or executing row-wise operations, it provides immense flexibility. However, performance considerations should always be kept in mind, favoring vectorized operations where possible. Hopefully, this guide provided you with clear insights into how apply() works in Python through real-world examples.

 

Other interesting article:

How pandas agg works in Python? Best example