Possible to run python script to create CSVs by scraping website?

[info-rchitect]

Hi,

I have a Python script that hits internal website and scrapes it, creating CSV files (2 per product).  The creation of CSV files is because I was running it manually on my desktop.  In the end, I just want to load the data contained in these CSV files into two distinct Snowflake tables.  What is the best approach?

thx

---

Operating system used: Windows 10

[Manuel]

Hi,

If you search Selenium in this community, you will find a few different posts, including this one, https://community.dataiku.com/t5/Using-Dataiku/Web-scrapping-any-tuto-to-explain-how-to-sart-with-da...

I hope this helps.

[info-rchitect]

Hi Manuel,

I used a Python recipe and got the dataframes I needed but having issues with the write_with_schema function.  Started a specific ticket related to that so I think this ticket can be closed.

thx