Why is the recipe saying Dataset doesn't exist after the dataset has been created?

pnkj

I am facing a strange issue where even after a dataset is getting created, the code is saying that dataset doesn't exists. Following is the function that I am using which has 3 steps -
1. Creating a table in the database
2. Creating a dataset in DSS and connecting it to the table created in 1
3. Writing a pandas dataframe with schema in the dataset created, thereby in the table

The function is working fine when I am running it manually in a cell in a notebook. But it throws an error saying that the dataset doesn't exist when I try to run it as a recipe.

def create_preview_data_in_sqls(df, filename, preview_tab):
  preview_tabs = ['raw', 'eda']


  preview_file_name = filename


  # 1. Create a temporary table in SQL Server using SQLExecutor2

  ms_proj_connection = dataiku.get_custom_variables()["ms_project_connection"]

  ms_proj_database = dataiku.get_custom_variables()["ms_project_database"]

  ms_proj_schema = dataiku.get_custom_variables()["ms_project_schema"]

  ms_executor = SQLExecutor2(connection=ms_proj_connection)


  query = f"""CREATE TABLE {preview_file_name} (temp_col int);"""

  ms_executor.query_to_df(query, post_queries=['COMMIT'])

  time.sleep(5)


  # 2. Create a dataset in Dataiku using create_sql_table_dataset connecting with the above table

  client = dataiku.api_client()

  project = client.get_project(dataiku.default_project_key())

  zone = project.get_flow().get_zone("1ldxijf")


  dataset = project.create_sql_table_dataset(dataset_name = preview_file_name,

                                             type = 'SQLServer',

                                             connection = ms_proj_connection,

                                             table = preview_file_name,

                                             schema = ms_proj_schema,

                                             catalog = None)


  settings = dataset.autodetect_settings()

  settings.save()

  dataset.move_to_zone(zone)

  time.sleep(5)


  # 3. Write with schema the df in the dataset/table

  dkuDataset = dataiku.Dataset(preview_file_name, ignore_flow=True)

  dkuDataset.write_with_schema(df)



  return f'Preview table name {preview_file_name}'

Turribeach

Can you please post the full error trace? What exactly are you trying to achieve with this code? Why can you not preview tables using the Explore tab?