Load from HDFS to Oracle

ubethke
ubethke Registered Posts: 21 ✭✭✭✭
in Using Dataiku
When inserting data from HDFS to Oracle or another supported database such as Greenplum, does the data flow through the DSS server (memory)?
Tagged:
· Share on Facebook

Answers

  • Mattsco
    Mattsco Dataiker, Registered Posts: 125 Dataiker

    Hi Ubethke,

    No, hopefully you won't load all your data into the memory of the DSS server.

    The data is "streamed" by chunk from HDFS to Oracle.

    You should check this learn article for more details.

    Matt

    · Share on Facebook
  • ubethke
    ubethke Registered Posts: 21 ✭✭✭✭
    Based on the article it is streamed from HDFS to DSS server to Oracle. Correct?
    If so then I would argue that this is not efficient as it requires an unnecessary roundtrip of the data through the DSS server.
    Your thoughts?
    Thanks
    Uli
    · Share on Facebook
  • Mattsco
    Mattsco Dataiker, Registered Posts: 125 Dataiker
    Correct!
    I agree but it's not always technically possible for us to avoid this roundtrip.
    · Share on Facebook
Bookmark this topic

Categories

Setup Info
    Tags
      Help me…