Training for fleet manager for AWS.

tgb417
tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,650 Neuron

All,

Is there any training on using Fleet manager on AWS to setup Dataiku DSS. Particularly around best practices on setting up Kubernetes. Over the coming months I hope to be setting up a research environment of DSS, that will be used to support researchers and automate workflows, and support some model building.


Operating system used: Linux

Tagged:

Answers

  • tgb417
    tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,650 Neuron

    I came across the following that seems to help with these questions.

    https://content.dataiku.com/nurturing-mlops-aws

    However, it does not get into the nitty gritty of choices and steps to setting up fleet manager. Which is my focus with this question.

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023, Circle Member Posts: 2,726 Neuron
  • tgb417
    tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,650 Neuron

    @Turribeach

    Thanks for the feedback. If I’m guessing correctly about the architecture I want to use, this seems to cover much of my use case, from a buttons and controls point of view.

    https://doc.dataiku.com/dss/latest/installation/cloudstacks-aws/guided-setup-new-vpc-elastic-compute.html

    However, I wish there was a more complete descriptive discussion, around making environment design and architecture choices. Less of a manual, and more of a review of possible configurations with pros and cons describing ease of use, scaleability, cost, future expandability.

  • Abdoulaye
    Abdoulaye Dataiku DSS Core Designer, Registered Posts: 49 ✭✭✭✭✭

    The biguest issue with dataiku is docs! complete nothing(almost) about infra deployement tutorials or something that got deeper than wha we see in docs.

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023, Circle Member Posts: 2,726 Neuron

    Can you be more specific about what you see missing?

  • Abdoulaye
    Abdoulaye Dataiku DSS Core Designer, Registered Posts: 49 ✭✭✭✭✭

    For example if you want to config s3/others.. connection there is no deeper explanation on docs.

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023, Circle Member Posts: 2,726 Neuron
  • Abdoulaye
    Abdoulaye Dataiku DSS Core Designer, Registered Posts: 49 ✭✭✭✭✭

    As said, not deep. How to config with sts profile? Ive saw another doc in knowledge? Seems dataiku has two documentation websites.

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023, Circle Member Posts: 2,726 Neuron
    edited July 2024

    https://knowledge.dataiku.com/latest/admin-configuring/connection-security/tutorial-aws-assumerole-mechanism.html

    The Knowledge site covers deeper subjects not covered in the documentation. There is also the developer site:

    https://developer.dataiku.com/

    You can easily search each of this sites using Google. Or if you don't know where something could be you can just search them all like this:

    site:https://dataiku.com/ s3 sts

  • tgb417
    tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,650 Neuron
    edited July 2024

    @Turribeach

    I use an even simpler version of that kind of search all the time with Dataiku Documentation.

    site:dataiku.com s3 sts

    You do not seem to need the back slashes and https: etc. (Which saves a few keystrokes.)

    One of my wife's (the academic reference librarian) favorite tricks for US Government Documents is a google search like this.

    site:.gov <Your topic here>

    site: is a great part of the google search capabiliteis.

Setup Info
    Tags
      Help me…