Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Hi,
I have created Dataset and Dataframe in PySpark and want to create partition using one column. could you please help how can I achieve this.
Similarly by using SPARKSQL need to create partition on non date column and that is part of schema, could you please help how can I create partition on it.
Hi,
You may find this doc useful: https://doc.dataiku.com/dss/latest/spark/datasets.html#other
Thanks,