Scenario - Run spark-scala

Options
Bader
Bader Registered Posts: 46 ✭✭✭✭✭

Is it possible to run spark-scala notebook isolated form flow in scenario.

I have a case of loading csv files from folder ( the folder updated every time) and would like to run the spark-scala in the scenario.

Best Answer

  • Clément_Stenac
    Clément_Stenac Dataiker, Dataiku DSS Core Designer Posts: 753 Dataiker
    Answer ✓
    Options

    Hi,

    This is not possible. We would advise you to create a "dummy" recipe in the Flow, that outputs into a dummy managed folder. You don't have to do anything with your output in the actual code, you just need an output that you can "build".

Setup Info
    Tags
      Help me…