Survey banner
The Dataiku Community is moving to a new home! We are temporary in read only mode: LEARN MORE

Join recipe processing too slow

Level 2
Join recipe processing too slow

I am using join recipe to join two datasets with the number of records of each dataset about 100 million lines, I store these 2 datasets as CSV files, the join process takes about 3 hours. Is there a way to speed this up?

Operating system used: Ubuntu 18.04

0 Kudos
1 Reply
Dataiker Alumni


Is the recipe running in the DSS engine (check below the Run button)? In what kind of machine is DSS installed on?

If you have a Database connection, you can Sync both datasets to the database first and then run the recipe on the SQL engine.

I hope this helps.