Docker and Hadoop

Options
UserBird
UserBird Dataiker, Alpha Tester Posts: 535 Dataiker
How can I use the docker version of DSS on a server which is an Hadoop Client? I don't want to set the DSS docker image as an Hadoop client itself, but I want to use the underlying system.

Thanks in advance
Tagged:

Answers

  • pbertin
    pbertin Dataiker Posts: 27 Dataiker
    Options
    Hello Lorenzo,

    in any case, the docker image must have access to the (distribution-specific) Hadoop JARs and the cluster configuration files (typically from /etc/hadoop/conf/) to be able to talk to the Hadoop cluster.

    There are several ways to achieve that, not sure which is simpler, depending on your setup. Maybe you could build a docker image derived from our base DSS image by adding the relevant Hadoop packages (the base image is Debian-7 based) and grab the configuration files by mounting the underlying system's /etc/hadoop directory?

    For more precise / personalized help, we'd need to know more about your setup. I suggest you go to support.dataiku.com and open a ticket there so you can add relevant the information

    Hope this helps

    Patrice Bertin
Setup Info
    Tags
      Help me…