Scraping Pyspark jobs without input data sets

Options
zjacobs23
zjacobs23 Registered Posts: 1

Hello,

I am currently creating PySpark jobs that do not have defined input data sets within my Pyspark notebooks. The tables are executed with spark sql within the actual notebook itself. I am wanting to see if there is a way to access all the tables executed within the spark sql across multiple projects. See screen shot below. The 'SELECT * FROM DB.TABLE' is where I am trying to grab the data sets being used. As you can see in the screen shot there is no inputs within the Pyspark notebook.

Tagged:

Answers

Setup Info
    Tags
      Help me…