Use Selenium through Jupyter

Reynholds
Reynholds Registered Posts: 9 ✭✭✭✭

Hi,

I've got some issues trying to use selenium to parse web page on jupyter-notebook-recipe.

Does someone already use selenium : https://towardsdatascience.com/web-scraping-using-selenium-python-8a60f4cf40ab

I've got some problem, because even if I downloaded the chromedriver or geckodriver I've still got some problem

Does someone can help me ?

Best regards,

Reynholds

Tagged:

Answers

  • ambr
    ambr Dataiker Alumni Posts: 1 ✭✭✭✭

    Hi,

    Are you using Jupyter Notebooks in DSS?

    If yes, the only way to use Selenium in Jupyter Notebooks in DSS is to download the chromedriver or the geckodriver on your server either by ssh or gui if it is a mac.

    I am at your disposal should you have more questions.

    Kind regards,

    Ambr'

  • Reynholds
    Reynholds Registered Posts: 9 ✭✭✭✭

    Hello,

    Thanks for your answer.

    Yes I'm using Jupyter Notebooks in DSS.

    my DSS running on a Container runing on linux, how I'm suppose to install chromedriver or the geckodriver ?

    Could you give me some indications ?

    Best regards,

    Reynholds

  • anpuke
    anpuke Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 3 ✭✭✭✭

    Hi,

    we experienced similar difficulties getting chromdriver running on SLES and had to set some options. You may try something like this:

    # setting the chromedriver Options
    options = webdriver.ChromeOptions()
    options.add_argument('--headless')
    options.add_argument('--no-sandbox') # required when running as root user. otherwise you would get no sandbox errors.

    driver=webdriver.Chrome(executable_path='/usr/lib64/chromium/chromedriver', options=options, service_args=['--verbose', '--log-path=/tmp/chromedriver.log'])

    Best regards,

    Andreas

  • mgirard
    mgirard Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1 Partner

    Hello, thank you for your help! I can't download chromedriver on dataiku. I don't know how to put '/usr/lib64/chromium/chromedriver' folder in Dataiku. Can you help me please ?

  • anpuke
    anpuke Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 3 ✭✭✭✭

    Hi @mgirard
    ,

    easiest way is to download chromedriver from google and do the setup with your package manager. What's the operating system of your dss instance?

    Best regards,

    Andreas

Setup Info
    Tags
      Help me…