-
Spark Configuration for Managed K8s cluster
Hi all, I am working through setting up Spak on managed K8s. And it is not clear to me what should be my configurations under Admin > Settings > Spark. My cluster has been attached to the DSS, I have created the stand alone spark image and pushed that into my repo. But now have no idea where to do. I attempt to execute a…
-
DSS setup of spark on AKS
Hi everyone, I’m currently trying to set up my DSS instance (running on a VM) to run Spark on AKS, and I'm feeling a bit lost about where to start. Could you please guide me on where Spark should be installed? Should it be on the AKS cluster or the VM? I realize this might be a basic question, but any assistance or…
-
HDFS - Force Parquet as default settings for recipe output
Greetings ! I'm currently on a platform with Dataiku 11.3.1 and writing datasets on HDFS. IT requires all dataset to be written in Parquet, but the default setting is on CSV (Hive) and it can generate errors. Is there a way to configure the connection to force the default settings to be Parquet ? Best regards,
-
Configurations Needed After Installing Automation Node via Fleet Manager
Hi everyone, I am planning to install an Automation Node using Fleet Manager and am curious about the post-installation process. Specifically, are there any configurations that need to be done on the Automation Node immediately after its installation? Also, how do I link this Automation Node with a Design Node? Thanks in…
-
Dataiku on Windows
This is a great post I just came across so reposting for visibility as I think it's really interesting: https://blog.dataiku.com/rethinking-dataiku-for-windows-users
-
Building a UI for workflow I built
I am building workflows for end users in the DSS, and I want to build a UI. traditionally we have people run "Scenarios" , but would love to develop a UI that kicks off the scenarios , and takes any necessary arguments to specify partitions, filepaths , etc. Are there any resources I can look at? Building a User UI for…
-
How to setup Athena connection using s3 connection
Hello, i've already setup an s3 connection which is working perfectly. I'd like to use it to setup Athena connection. My s3 connection use sts-assume-role type. Iuse dataikucloudstacj (aws) Can you tell me why thing do not work? do I need specific policies in roles of s3 and intances profile? I've this error in screenshots.
-
How can I write to Minio as parquet file?
Hi, I am using Dataiku version 12.5.2. I am trying to write a simple csv file as a parquet file to Minio. For this, I have successfully installed Hadoop integration with the following steps: $ cd DATADIR $ ./bin/dss stop $ ./bin/dssadmin install-hadoop-integration -standalone generic-hadoop3 -standaloneArchive…
-
Best settings of Flush size and interval for Event Server
Hi! Dataiku, Recently, I installed Event Server and got log files from DSS nodes, thanks to it. and tried to search Flush Size and Flush Interval from Dataiku Documentation, but there's nothing. so, I have no idea how much I have to input values for them. Can you give me the best example settings for Flush Size and Flush…
-
aks deployment error
Hello, I hope this message finds you well. I wanted to bring to your attention an error I've been encountering while attempting to deploy our project API to the AKS cluster. Specifically, I'm receiving the following error: 'Failed to solve: failed to prepare j4mbejdth39r5n6d6bjynlgca as dd5fvbttf7fw7esem98vfra3o: max depth…