How to run Pyspark dataframe in Scenarios?

Ramya
Ramya Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 8 ✭
in Using Dataiku

I'm encountering a issue while attempting to run Spark Dataframe in specific scenario to retrive data,However Im facing error in this process.

Tagged:
· Share on FacebookShare on Twitter

Answers

  • Alexandru
    Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,226 Dataiker

    Hi @Ramya
    ,
    When you say running spark dataframe in a scenario are you running a pyspark recipe by building an output dataset of that recipe?

    Or are you trying to run PySpark code inside a Python Scenario step( this would not be supported)

    You can run Spark code in PySpark Notebook or PySpark recipe you can only run Python code in Python scenario steps.
    Thanks,

    · Share on FacebookShare on Twitter
Bookmark this topic

Categories

Setup Info
    Tags
      Help me…