Is there a limit of size when uploading a csv?

UserBird
UserBird Dataiker, Alpha Tester Posts: 535 Dataiker
I tried to upload a csv which had a size of 17Go but it didn't work

Best Answer

  • AdrienL
    AdrienL Dataiker, Alpha Tester Posts: 196 Dataiker
    Answer ✓
    There is no limit enforced by Dataiku DSS. Some common problems causing this symptom include a full disk, a connection timeout, a reverse proxy in front of Dataiku DSS that enforces a limit.

    Compressing your CSV (using gzip) may help mitigate the issue.

    If you are a Dataiku customer, I suggest you send us a support ticket for additional help.

Answers

  • phribeiro
    phribeiro Registered Posts: 2 ✭✭✭✭
    I have the same problem. I am trying to upload a 9GB CSV file, the file appears to read, but a few seconds after the meter finishes, it says "no file yet".
    The computer have 16GB or RAM and 45GB of disk available.
    What can be the problem, RAM?
    Thanks.
  • AdrienL
    AdrienL Dataiker, Alpha Tester Posts: 196 Dataiker
    I would need more details on your setup to answer this properly, for example some reverse-proxy, generally set up by your system administrator / IT department, can limit the files sent to DSS.

    Can you check on the "Network" tab of your browser's element inspecter if you see a call named "add-file" that fails? If so, there is a known bug that prevents DSS from showing an error in this case, this will be fixed in the next release of DSS.
  • phribeiro
    phribeiro Registered Posts: 2 ✭✭✭✭
    Yes, there is a network error:
    Failed to load resource: The network connection was lost.
    but this is a local file in a personal macbook pro, running MacOSSierra.
    Can I make any configuration change to allow me to use this file?
  • AdrienL
    AdrienL Dataiker, Alpha Tester Posts: 196 Dataiker
    You can add a Filesystem Connection (in Administration > Connections) pointing to some directory on your Mac. Then place the file in [a subfolder of] this directory, and create a Dataset of type Filesystem, using that connection, and browse to the file in question.

    https://doc.dataiku.com/dss/latest/connecting/connections.html
Setup Info
    Tags
      Help me…