Jingwen Zheng - page 3

Data transformation with pandas vs. pyspark

This blog compares "pandas" and "pyspark" on data transformation with the following points: checking dataframe size, checking unique values of a column, creating a new column, filtering, selecting a list of columns, aggregating, renaming columns, joining 2 dataframes, creating a new dataframe, creating a pivot table.

Read more