Use Selenium through Jupyter

Reynholds

Hi,

I've got some issues trying to use selenium to parse web page on jupyter-notebook-recipe.

Does someone already use selenium : https://towardsdatascience.com/web-scraping-using-selenium-python-8a60f4cf40ab

I've got some problem, because even if I downloaded the chromedriver or geckodriver I've still got some problem

Does someone can help me ?

Best regards,

Reynholds

ambr

Hi,

Are you using Jupyter Notebooks in DSS?

If yes, the only way to use Selenium in Jupyter Notebooks in DSS is to download the chromedriver or the geckodriver on your server either by ssh or gui if it is a mac.

I am at your disposal should you have more questions.

Kind regards,

Ambr'

Reynholds

Hello,

Thanks for your answer.

Yes I'm using Jupyter Notebooks in DSS.

my DSS running on a Container runing on linux, how I'm suppose to install chromedriver or the geckodriver ?

Could you give me some indications ?

Best regards,

Reynholds

anpuke

Hi,

we experienced similar difficulties getting chromdriver running on SLES and had to set some options. You may try something like this:

# setting the chromedriver Options
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox') # required when running as root user. otherwise you would get no sandbox errors.

driver=webdriver.Chrome(executable_path='/usr/lib64/chromium/chromedriver', options=options, service_args=['--verbose', '--log-path=/tmp/chromedriver.log'])

Best regards,

Andreas

mgirard

Hello, thank you for your help! I can't download chromedriver on dataiku. I don't know how to put '/usr/lib64/chromium/chromedriver' folder in Dataiku. Can you help me please ?

anpuke

Hi @mgirard
,

easiest way is to download chromedriver from google and do the setup with your package manager. What's the operating system of your dss instance?

Best regards,

Andreas