Announcing the winners & finalists of the Dataiku Frontrunner Awards 2021! Read their inspiring stories

PyCharm API not supporting PySpark recipes

jmccartin
Level 3
PyCharm API not supporting PySpark recipes

Hi, the API you recently created and detailed here: https://academy.dataiku.com/latest/tutorial/code/pycharm.html does not appear to support any recipe other than made in Python.



If I connect per the instructions, I can only see recipes that were made in Python. Since there is no real difference between PySpark and Python (you can pip install the former), why do you not support PySpark? Could you not make the API ambiguous for all recipes, so therefore potentially allowing something like Scala via intelliJ?

3 Replies
Peter_R_Knight
Level 2

I also raised the same issue with them - glad to hear I'm not alone!

0 Kudos
Ignacio_Toledo

Apparently not everybody agrees, but in my opinion I think @Clément_Stenac provided a very good explanation / reason as an answer to another thread. It makes sense to be able to modify the code on one's personal favorite IDE, but then running the recipe on a local machine different than the DSS node, why would you need that?

0 Kudos
emher
Level 3

Are there any plans to support this? I would really like to be able to edit the recipes via the Dataiku plugin in pycharm, event if they are not immediately runnable.

0 Kudos
Labels (2)
A banner prompting to get Dataiku DSS