Tab to auto-complete isn't working with created code envs?
![ak12](https://us.v-cdn.net/6038231/uploads/lithium_attachments/n6836iC4899853F52971D3.jpg)
Hi,
When writing code I am able to use tab to auto-complete when I am using the default Python 2 environment in DSS, but when I change the kernel of my notebook to be any of the python envs I have created I lose the ability to use tab to auto-complete?
I have looked in the setting and the documentation and I cannot see any way of making sure it is enabled. If it helps I am on Dataiku version 7
Thank you
Best Answer
-
Sergey Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts Posts: 365 Dataiker
Hi @ak12
We've seen this before. It is likely that your impersonated end-user does not have a valid home directory. The mechanism that IPython uses to perform the tab autocompletion when running python 3 makes use of a disk-based cache. Such disk-based cache requires a valid home directory.Generally speaking, we strongly recommend that every impersonated end-user have a valid and writable home directory. Failure to have this is known to cause a number of issues such as this one, alongside with Jupyter exports, matplotlib, xgboost, and others.
Answers
-
tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,598 Neuron
+1, I think that I've seen the same thing.
I'm not clear that I know exactly what you mean by an impersonated user.
That leaves me unclear on how to resolve this issue on a local Macintosh design node install?
What needs to be done to resolve this:
- The User name in DSS security must match the name of the short macintosh login name? (Same as login directory)
- Or does one have to create a Dataiku account on the computer?
- Or something else...
For the community edition where you can not change the login name is there a way to resolve these issues? Thanks for your help.
-
Sergey Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts Posts: 365 Dataiker
OK, so impersonated user is the Unix user that runs the jupyter notebook when UIF is enabled:
https://doc.dataiku.com/dss/latest/user-isolation/index.html
While there is still a requirement to have a valid home directory for the user if UIF is enabled to make autocomplete working, there is also one thing that breaks the autocompletion feature in Jupyter notebooks.The built-in code-env contains hard-coded ipython==5.5.0 (ipython>=5.5,<5.6) package while managed code-envs don't have that limitation and install the latest available for corresponding Python version:- for Python3.6 -> 7.16.0,- for Python3.5 -> 7.9.0- for Python2.7 -> 5.10.0I haven't checked the latest versions of this package for all Python versions, but installing ipython==5.5.0 to the code-env solves the autocomplete issue.Please note, you will need to reload the kernel after installing the new package. -
tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,598 Neuron
@sergeyd
,Based on your instructions I made the following changes to one of the code environments I use with Jupyter Notebooks. Is this what you had intended? I do have a tab-based auto-complete.
Is there a way to get an auto-complete popup like MS Intelesense to come up?
-
Sergey Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts Posts: 365 Dataiker
Hi @tgb417
I mentioned v7.16.0 as the version that is being installed by default for Python3.6 code envs. During my tests, autocomplete and autosuggestion wasn't working with this ipython version so I installed 5.5.0 (the same one as we have in built-in code env) and get both features working as expected:
-
tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,598 Neuron
On the first attempt, I was not able to downgrade to this version. I ended up with some sort of problem with the fbprophet library that was previously installed in this code environment.
ERROR: Failed building wheel for fbprophet
And ended up with some other problem that kept me from Python all together. Not clear why. And I now have a broken code environment. I don't have time to sort this out at the moment.
However, the experience left me with a question:
That seems like a fairly large downgrade...
Do you have any sense of anything important we might loose in terms of features, or increased security vulnerabilities?
Do you have any sense when more recent version will be usable with DSS?
-
Sergey Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts Posts: 365 Dataiker
Of course, every code-env has a specific set of packages and one package change can break others. I cannot say that fbprophet is a relatively easy-to-install package. So if you have deps issues with python 5.5.0 you will need to revert back it to 7.16.0 (original version) and your env should return back to normal. Sure, no guarantee that autocomplete feature will work in this case.
-
tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,598 Neuron
The following seems to have worked. I had to completely rebuild the code environment.
Still testing to see if all is working OK.
-
Sergey Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts Posts: 365 Dataiker
I just run a test to check fbprophet package compatibility with ipython==5.5.0 and had no issues installing it:
From your screenshot, you are using conda while I have used regular pip env. That might be the root cause of the issue for your specific env.
-
tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,598 Neuron
It does appear that one has to hit the tab key to activate. This does not automatically pop up on the screen as you type like SSMS or VS Code might do.
-
Sergey Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts Posts: 365 Dataiker
Well
The topic is called "Tab to autocomplete" so using Tab to activate is expected here. -
Suman Partner, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Registered Posts: 8 Partner
We are using conda (only) for DSS 7.0
and by default when creating python3.6 code environment ipython==7.6.1 is getting installed.
How can we include ipython>=5.5,<5.6 part of conda?
Will it affect any other inbuilt environment?
Thanks & regards,
Suman
-
tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,598 Neuron
@sergeyd
,Sorry agreed. Sorry for cluttering up this thread.
So, the tool that I think I'm thinking about that allows the IntelliSense-like auto-completion seems to be hinterland a nbextensions. I'm not clear if it is supported at some level by DSS.
And I eventually found this thread as well that seems to reference a way back to auto-complete experience. (w/o tab)
-
Sergey Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts Posts: 365 Dataiker
Based on https://github.com/ipython/ipython/issues/12740 to get autocomplete to work with the latest available ipython for Py3.6 (7.16.0) we need to downgrade jedi from 0.18.0 to 0.17.2.
-
Suman Partner, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Registered Posts: 8 Partner
Thanks @sergeyd
Is there any reference link which can be followed like which version support what.... for example if I want to use Python3.6 with ipython 7.9.0 what version of jedi or any other package need to upgrade/downgrade to make jupyter autocomplete work.
-
Sergey Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts Posts: 365 Dataiker
I am not aware of such a compatibility matrix.
-
For anyone still struggling with tab autocomplete inside of Jupyter I add the following magic at the top of my notebook and it fixed the issue :
%config Completer.use_jedi = False