Survey banner
The Dataiku Community is moving to a new home! We are temporary in read only mode: LEARN MORE

How can I write to Minio as parquet file?

Solved!
oguzselvi
Level 1
How can I write to Minio as parquet file?

Hi,

I am using Dataiku version 12.5.2. I am trying to write a simple csv file as a parquet file to Minio. For this, I have successfully installed Hadoop integration with the following steps:

$ cd DATADIR
$ ./bin/dss stop
$ ./bin/dssadmin install-hadoop-integration -standalone generic-hadoop3 -standaloneArchive /data/dataiku-dss-hadoop-standalone-libs-generic-hadoop3-12.5.2.tar.gz
$ ./bin/dss start

However, when I try to write a csv to Minio as a parquet file using Sync recipe, it gives the following error:


java.lang.NoSuchMethodError: 'void org.apache.hadoop.util.SemaphoredDelegatingExecutor.<init>(java.util.concurrent.ExecutorService, int, boolean)'

How can I overcome this problem?

Regards,

Oguz


Operating system used: Linux Ubuntu 22.04

0 Kudos
1 Solution
oguzselvi
Level 1
Author

Hi,

We solved the problem with Dataiku support. The problem was a conflict with phoenix jars. So moving the phoenix jars to another directory solved the problem.

View solution in original post

0 Kudos
2 Replies
AlexT
Dataiker

Hi @oguzselvi ,
Could please open a support with the instance diagnostics so we can check if the standalone hadoop integration ran successfully?


Thanks

0 Kudos
oguzselvi
Level 1
Author

Hi,

We solved the problem with Dataiku support. The problem was a conflict with phoenix jars. So moving the phoenix jars to another directory solved the problem.

0 Kudos