Survey banner
Switching to Dataiku - a new area to help users who are transitioning from other tools and diving into Dataiku! CHECK IT OUT

PyCharm API not supporting PySpark recipes

Level 3
PyCharm API not supporting PySpark recipes

Hi, the API you recently created and detailed here: does not appear to support any recipe other than made in Python.

If I connect per the instructions, I can only see recipes that were made in Python. Since there is no real difference between PySpark and Python (you can pip install the former), why do you not support PySpark? Could you not make the API ambiguous for all recipes, so therefore potentially allowing something like Scala via intelliJ?

3 Replies
Level 2

I also raised the same issue with them - glad to hear I'm not alone!

0 Kudos

Apparently not everybody agrees, but in my opinion I think @Clément_Stenac provided a very good explanation / reason as an answer to another thread. It makes sense to be able to modify the code on one's personal favorite IDE, but then running the recipe on a local machine different than the DSS node, why would you need that?

0 Kudos
Level 3

Are there any plans to support this? I would really like to be able to edit the recipes via the Dataiku plugin in pycharm, event if they are not immediately runnable.

0 Kudos


Labels (2)
A banner prompting to get Dataiku