Use Selenium through Jupyter
![Reynholds](https://us.v-cdn.net/6038231/uploads/Dataiku/nAvatar14.png)
Hi,
I've got some issues trying to use selenium to parse web page on jupyter-notebook-recipe.
Does someone already use selenium : https://towardsdatascience.com/web-scraping-using-selenium-python-8a60f4cf40ab
I've got some problem, because even if I downloaded the chromedriver or geckodriver I've still got some problem
Does someone can help me ?
Best regards,
Reynholds
Answers
-
Hi,
Are you using Jupyter Notebooks in DSS?
If yes, the only way to use Selenium in Jupyter Notebooks in DSS is to download the chromedriver or the geckodriver on your server either by ssh or gui if it is a mac.
I am at your disposal should you have more questions.
Kind regards,
Ambr'
-
Hello,
Thanks for your answer.
Yes I'm using Jupyter Notebooks in DSS.
my DSS running on a Container runing on linux, how I'm suppose to install chromedriver or the geckodriver ?
Could you give me some indications ?
Best regards,
Reynholds
-
anpuke Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 3 ✭✭✭✭
Hi,
we experienced similar difficulties getting chromdriver running on SLES and had to set some options. You may try something like this:
# setting the chromedriver Options
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox') # required when running as root user. otherwise you would get no sandbox errors.driver=webdriver.Chrome(executable_path='/usr/lib64/chromium/chromedriver', options=options, service_args=['--verbose', '--log-path=/tmp/chromedriver.log'])
Best regards,
Andreas
-
mgirard Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1 Partner
Hello, thank you for your help! I can't download chromedriver on dataiku. I don't know how to put '/usr/lib64/chromium/chromedriver' folder in Dataiku. Can you help me please ?
-
anpuke Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 3 ✭✭✭✭
Hi @mgirard
,easiest way is to download chromedriver from google and do the setup with your package manager. What's the operating system of your dss instance?
Best regards,
Andreas