-
Unable to connect to DSS from container
Hi All, I am trying to setup AKS in dataiku. I have successfully attached the AKS cluster with the dataiku from clusters tab in administration. I am facing issue while testing it at the container execution tab in Setting. Below is the error I am getting while I try to test it. Unable to connect to DSS from container :…
-
Questions on Dataiku, EKS, and EMR Serverless for Efficient Data Processing
Hi Dataiku Community, I hope you're all doing well. I wanted to reach out with some questions regarding our current implementation, where we are utilizing EMR Serverless with TBs of data flowing between Snowflake and S3. * Intermediate Datasets Avoidance: We are looking into EKS setup compared to EMR Serverless. How does…
-
Dss running Out of memory too fast
Hello ! I'm currently working on a DSS instance, having fun building data piplines, but it's running out of memory after 2hrs. DSS is installed manually on a docker image. We tryed several setup with 'xmx' configuration but we ends-up with many idle processes (~20 jek) that filled our allocated memory. The number of jek is…
-
How to check if push_container_exec_base_images() succeeded
Hi, I was looking at using push_container_exec_base_images() in connection with some other steps to work on building and updating some base images for Dataiku using the Python API…
-
Base Image for containerized execution
Hi All, DSS 12 made the switch for the base OS to AlmaLinux. The logs on the fleet manager seem to indicate that the base image for containerized execution is still centos7. Is that correct? What are the plans/timelines to update the base image if indeed we are still on centos7 that is fast approaching EOL? Thanks, Yash
-
Running node initialization scripts
Hi All, Is there a way to run customized initialization scripts on the nodes set up in the clusters managed by DSS? Looking for ways to set up customized telemetry from the individual nodes on the clusters. Thanks, Yash
-
What ports needs to be open for Elastic AI jobs in Kubernetes?
Assuming the DSS base port is 9000 I guess I need to allow incoming connections to ports 9000-9010 from the EKS CIDR, But then when I used the DSS > Administration > Settings > Containerized execution > TEST I see that it also tries to connect to 33249 [2023-08-16 12:54:19,130] [1/MainThread] [INFO] [root] Try to ping…
-
Which docker images are save to remove?
I notices that docker images make up a significant amount of disk usage of my dataiku vm. The only other place I found a mention of it was https://community.dataiku.com/t5/General-Discussion/Remove-docker-image-old-version/m-p/13147 I wonder which docker images are safe to remove. If I understand correctly the only place…
-
Best Practices For Updating and Renaming Spark and Container Configurations
Hello Dataiku Community, Hope all is well! Our team is looking to implement new Spark and container configuration settings on our instances. We are curious to understand what the best practices are for updating the existing configurations. For context we have existing Spark configurations already being used by end users,…
-
Impossible to build Python code_envs for containerized execution
Hello, I have a problem to build python code_envs for GKE. When i build the code_env without containerized exection : everything goes well, but as soon as i add an containrized env, it fails. It fails because during the docker build phase, execution of pip fail with 'can't connect errors' because the container does not…