Use Selenium through Jupyter

Reynholds
Level 2
Use Selenium through Jupyter

Hi,

I've got some issues trying to use selenium to parse web page on jupyter-notebook-recipe.

Does someone already use selenium  : https://towardsdatascience.com/web-scraping-using-selenium-python-8a60f4cf40ab

I've got some problem, because even if I downloaded the chromedriver or geckodriver I've still got some problem

Does someone can help me ?

Best regards,

Reynholds

0 Kudos
5 Replies
ambr
Dataiker Alumni

Hi,

Are you using Jupyter Notebooks in DSS?

If yes, the only way to use Selenium in Jupyter Notebooks in DSS is to download the chromedriver or the geckodriver on your server either by ssh or gui if it is a mac.

I am at your disposal should you have more questions.

Kind regards,

Ambr'

Reynholds
Level 2
Author

Hello,

Thanks for your answer.

Yes I'm using Jupyter Notebooks in DSS.

my DSS running on a Container runing on linux, how I'm suppose to install  chromedriver or the geckodriver ?

Could you give me some indications ?

Best regards,

Reynholds

0 Kudos
anpuke
Level 2

Hi,

we experienced similar difficulties getting chromdriver running on SLES and had to set some options. You may try something like this:

# setting the chromedriver Options
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox') # required when running as root user. otherwise you would get no sandbox errors.

driver=webdriver.Chrome(executable_path='/usr/lib64/chromium/chromedriver', options=options, service_args=['--verbose', '--log-path=/tmp/chromedriver.log'])

 

Best regards,

Andreas

0 Kudos
mgirard
Level 1

Hello, thank you for your help! I can't download chromedriver on dataiku. I don't know how to put '/usr/lib64/chromium/chromedriver' folder in Dataiku. Can you help me please ?

0 Kudos
anpuke
Level 2

Hi @mgirard,

easiest way is to download chromedriver from google and do the setup with your package manager. What's the operating system of your dss instance?

Best regards,

Andreas 

0 Kudos