Dataset run

Options
mhamad78
mhamad78 Partner, L2 Designer, Registered Posts: 2 Partner
in Using Dataiku

I have a dataset with 5Gb excel sheet. When trying to get a sample of 10K rows it takes a long time and it does not end!

Any advice please.

· Share on FacebookShare on Twitter

Answers

  • Liev
    Liev Dataiker Alumni Posts: 176 ✭✭✭✭✭✭✭✭
    Options

    hi @mhamad78

    While your data is in Excel, the entire file will need to be loaded into order to create the sample you require.

    One recommendation would be to transform your file into a CSV and then use this in DSS.

    Good luck!

    · Share on FacebookShare on Twitter
  • mhamad78
    mhamad78 Partner, L2 Designer, Registered Posts: 2 Partner
    Options

    Thank you for your reply. But what we have is the CSV. It took around 30 minutes to load 10M rows. Knowing that under the sample design I mentioned taking the first 10K rows only!!

    · Share on FacebookShare on Twitter
  • Liev
    Liev Dataiker Alumni Posts: 176 ✭✭✭✭✭✭✭✭
    Options

    Hi @mhamad78

    In your original post you said your file was in Excel, so the answer was related to this.

    Where are you creating your sample? Are you in the dataset creation stage or the explore view of the dataset?

    · Share on FacebookShare on Twitter
Bookmark this topic

Categories

Setup Info
    Tags
      Help me…