Using selenium with a python recipe

Hi Paul, I would suggest we go the support route if you can't figure it out.

Hard to say what could be different in terms of the 2 code environments and your code that could cause it to fail. Dependency wise as long as selenium is added to the code env it should work, you would need to have chrome installed and chrome driver available at the defined path. I've tested on my end both in recipe and notebook with the same code env and it worked fine, I've used the following sample code. Please note I've installed Chrome and chrome driver for this to work.

import dataiku
from selenium import webdriver
import time
import pandas as pd

# selenium stuff 
options = webdriver.ChromeOptions() ;
prefs = {"download.default_directory" : "/tmp", "prompt_for_download": "false"};
output_dataset = dataiku.Dataset("fitness2")
# options added to get it to on Linux Server , installed latest chrome and compatible chrome driver
chromeOptions = webdriver.ChromeOptions()
chromeOptions.add_argument("--headless")
chromeOptions.add_argument("--download.prompt_for_download=false")
chromeOptions.add_argument("--download.default_directory=/tmp")
chromeOptions.add_experimental_option("prefs",prefs);
driver = webdriver.Chrome('/home/dataiku/chromedriver', chrome_options=chromeOptions)

try:

    driver.get('https://www.browserstack.com/test-on-the-right-mobile-devices');
    downloadcsv= driver.find_element_by_css_selector('.icon-csv');
    gotit= driver.find_element_by_id('accept-cookie-notification');
    gotit.click();    
    downloadcsv.click();
    time.sleep(5)
    driver.close()

except:
     print("Invalid URL")
        
# read downloaded file and create dataset
cereal_df = pd.read_csv("/tmp/BrowserStack - List of devices to test on.csv")

output_dataset.write_with_schema(cereal_df)