Use Selenium through Jupyter

Options
Reynholds
Reynholds Registered Posts: 9 ✭✭✭✭
in Using Dataiku

Hi,

I've got some issues trying to use selenium to parse web page on jupyter-notebook-recipe.

Does someone already use selenium : https://towardsdatascience.com/web-scraping-using-selenium-python-8a60f4cf40ab

I've got some problem, because even if I downloaded the chromedriver or geckodriver I've still got some problem

Does someone can help me ?

Best regards,

Reynholds

Tagged:
· Share on FacebookShare on Twitter

Answers

  • ambr
    ambr Dataiker Alumni Posts: 1 ✭✭✭✭
    Options

    Hi,

    Are you using Jupyter Notebooks in DSS?

    If yes, the only way to use Selenium in Jupyter Notebooks in DSS is to download the chromedriver or the geckodriver on your server either by ssh or gui if it is a mac.

    I am at your disposal should you have more questions.

    Kind regards,

    Ambr'

    · Share on FacebookShare on Twitter
  • Reynholds
    Reynholds Registered Posts: 9 ✭✭✭✭
    Options

    Hello,

    Thanks for your answer.

    Yes I'm using Jupyter Notebooks in DSS.

    my DSS running on a Container runing on linux, how I'm suppose to install chromedriver or the geckodriver ?

    Could you give me some indications ?

    Best regards,

    Reynholds

    · Share on FacebookShare on Twitter
  • anpuke
    anpuke Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 3 ✭✭✭✭
    Options

    Hi,

    we experienced similar difficulties getting chromdriver running on SLES and had to set some options. You may try something like this:

    # setting the chromedriver Options
    options = webdriver.ChromeOptions()
    options.add_argument('--headless')
    options.add_argument('--no-sandbox') # required when running as root user. otherwise you would get no sandbox errors.

    driver=webdriver.Chrome(executable_path='/usr/lib64/chromium/chromedriver', options=options, service_args=['--verbose', '--log-path=/tmp/chromedriver.log'])

    Best regards,

    Andreas

    · Share on FacebookShare on Twitter
  • mgirard
    mgirard Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1 Partner
    Options

    Hello, thank you for your help! I can't download chromedriver on dataiku. I don't know how to put '/usr/lib64/chromium/chromedriver' folder in Dataiku. Can you help me please ?

    · Share on FacebookShare on Twitter
  • anpuke
    anpuke Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 3 ✭✭✭✭
    Options

    Hi @mgirard
    ,

    easiest way is to download chromedriver from google and do the setup with your package manager. What's the operating system of your dss instance?

    Best regards,

    Andreas

    · Share on FacebookShare on Twitter
Bookmark this topic

Categories

Setup Info
    Tags
      Help me…