Discover this year's submissions to the Dataiku Frontrunner Awards and give kudos to your favorite use cases and success stories!READ MORE

Provision of connecting your own compute resource to dataiku

Solved!
mansit_suman
Level 2
Provision of connecting your own compute resource to dataiku

What is the provision of connecting your own compute resource(instance or K8 cluster) from Azure(or AWS) to dataiku for running specific jobs such as model training or any recipes?

0 Kudos
1 Solution
AlexT
Dataiker
Dataiker

You can install DSS on EC2 and possible use docker on the EC2 machine but this would not be recommended. 

https://doc.dataiku.com/dss/latest/containers/docker.html#why-kubernetes-rather-than-docker

As for Azure App Service this is unsupported. 

We recommend using K8s to distribute computing from DSS. 

https://knowledge.dataiku.com/latest/kb/data-prep/where-compute-happens.html

View solution in original post

0 Kudos
4 Replies
AlexT
Dataiker
Dataiker
0 Kudos
mansit_suman
Level 2
Author

What about any EC2 instance or Azure app service?

0 Kudos
AlexT
Dataiker
Dataiker

You can install DSS on EC2 and possible use docker on the EC2 machine but this would not be recommended. 

https://doc.dataiku.com/dss/latest/containers/docker.html#why-kubernetes-rather-than-docker

As for Azure App Service this is unsupported. 

We recommend using K8s to distribute computing from DSS. 

https://knowledge.dataiku.com/latest/kb/data-prep/where-compute-happens.html

0 Kudos
mansit_suman
Level 2
Author

Consider for use-cases where we just have to deal with ETL processes for around 5-6lakhs of rows. Can we connect our computes(Ec2 instances) to the DSS and use it for running certain ETL jobs(code or visual recipes) only, for  optimizing the cost and don't want to leverage K8s, can that be done?

With that, Instead of leveraging our own, can also something that can be provided by Dataiku? If yes, what are their costings?

Also, is there any documentation that is available that specifically talks about cloud computing and infra management. If yes you can attach it in the reply, this would be really helpful.

Thanks a lot.

0 Kudos

Labels

?
Labels (2)

Setup info

?
A banner prompting to get Dataiku