GROUP BY Functions

The groupby() function in Pandas is used to split a DataFrame into groups based on some criterion — typically the values in one or more columns. This is not just a way to rearrange data; it is the foundation for performing separate calculations on each group.

The function works on a split-apply-combine strategy, which happens in three clear steps:

1. Split the original DataFrame into groups by creating a GroupBy object.
2. Apply the required function (like sum, mean, or a custom function) to each group independently.
3. Combine the results into a new DataFrame or Series.

To see this in action, imagine a simple two-column DataFrame: one column called key (with values A, B, C) and another called data. If you want the sum of data for each key, you first split the rows into three groups (all A rows together, all B rows together, all C rows together). Then you apply the sum function to each group. Finally, you combine the three sums into a new DataFrame showing A: 15, B: 30, C: 45 (as in the textbook's Figure 3.1).

---

Creating a GroupBy Object

You create a GroupBy object by calling df.groupby('column_name'). This does not compute anything yet — it just stores the grouping information.

g1 = df.groupby('Name')

Once you have this object, you can inspect the groups in several ways:

• g1.first() — displays the first row from each group. For the textbook's DataFrame df, this shows each student's first UT scores.
• g1.size() — returns the number of rows in each group. All four students (Ashravy, Mishti, Raman, Zuhaire) have 3 rows each.
• g1.groups — a dictionary where keys are group names and values are the row indices belonging to that group. For example, 'Ashravy': Int64Index([6, 7, 8], dtype='int64') means rows 6, 7, and 8 belong to Ashravy.
• g1.get_group('Raman') — extracts the entire DataFrame for a single group. This returns all rows where Name is 'Raman'.

---

Grouping by Multiple Columns

You can group by more than one column by passing a list of column names:

g2 = df.groupby(['Name', 'UT'])

This creates groups for each unique combination of student and unit test. Calling g2.first() shows the first occurrence of each combination — effectively a multi-indexed view of the data.

---

Aggregation: Applying Functions to Groups

Once groups are created, the next step is to apply functions. This is called aggregation — it returns a single aggregated value (like mean, sum, or standard deviation) for each group.

Aggregation is done using the agg() or aggregate() function. By default, functions are applied column-wise.

Example 1: Average marks scored by all students in each subject for each UT.

df.groupby(['UT']).aggregate('mean')

This produces a table with UT numbers as rows and subjects as columns, each cell containing the average marks for that UT.

Example 2: Average marks in Maths for each UT.

group1 = df.groupby(['UT'])
group1['Maths'].aggregate('mean')

Here, you first create the GroupBy object, then select only the Maths column before applying the mean. The result is a Series with UT numbers as the index.

---

Applying Multiple Aggregate Functions

You can apply several different functions at once by passing a list of function names to agg().

Program 3-10 from the textbook demonstrates this: to print the mean, variance, standard deviation, and quartile of Maths marks for each student across all UTs.

df.groupby(by='Name')['Maths'].agg(['mean', 'var', 'std', 'quantile'])

The output is a DataFrame where each row is a student and each column is one of the requested statistics. For example, Ashravy has a mean of 19.67, variance 44.33, standard deviation 6.66, and quartile 23.0. …