Survey banner
Switching to Dataiku - a new area to help users who are transitioning from other tools and diving into Dataiku! CHECK IT OUT

"Connection Refused" when querying value via Pandas on Spark

Level 3
"Connection Refused" when querying value via Pandas on Spark


When I try to find a value via pandas on spark, I get some error saying Connection refused. 

Here the size of the data is 24lakh rows. Plus when I run on pandas or core pyspark it works fine.

Can someone help?

Operating system used: Ubuntu

(Topic title edited by moderator to be more descriptive. Original title "Spark")

0 Kudos
1 Reply

Hi @Rushil09 ,

Can you please share a snippet of your actual code and what spark version you are using? 

Are you trying to Pandas API on Spark? 


When interacting with pyspark you should use Dataiku API and Spark APIs:


0 Kudos


Labels (1)
A banner prompting to get Dataiku