Survey banner
The Dataiku Community is moving to a new home! We are temporary in read only mode: LEARN MORE

"Connection Refused" when querying value via Pandas on Spark

Level 3
"Connection Refused" when querying value via Pandas on Spark


When I try to find a value via pandas on spark, I get some error saying Connection refused. 

Here the size of the data is 24lakh rows. Plus when I run on pandas or core pyspark it works fine.

Can someone help?

Operating system used: Ubuntu

(Topic title edited by moderator to be more descriptive. Original title "Spark")

0 Kudos
1 Reply

Hi @Rushil09 ,

Can you please share a snippet of your actual code and what spark version you are using? 

Are you trying to Pandas API on Spark? 


When interacting with pyspark you should use Dataiku API and Spark APIs:


0 Kudos


Labels (1)
A banner prompting to get Dataiku