How to define a dataset name including the current date for a dataset of type file

rona

Hello,

I would like to generate a dataset of type file with a file name including the current date.

I read that we can use project variable in file name. Is there an easy way to setup this variable dynamically with the current date ? (not with python code and API)

Thanks,

Annie

Liev

It seems that it would be hard to accommodate to the specificity of your design while having a no-code approach. I would consider using the API if the logic can only be meaningfully described by it.

Liev

Hi @rona
,

A dataset in dss is a rich pointer that specifies location, access and properties in order to access the actual data. As such, the pattern which you describe is not something commonly seen in DSS.

I suggest that instead of having several datasets with different names, you look into the concept of partitioning. This might prove a more sustainable and useful pattern of flow design.

I hope this helps!

rona

Hi

Unfortunately, it doesn't fit our use case.

We want to generate files that will be used outside DSS landscape.

From your answer, should I conclude that there is no way to build this file name without using some python code ?

Thanks

Liev

If you use partitioning on a dataset that is in the Filesystem, DSS will generate new files for each given partition. You could then use those files and process as required outside DSS if that's what you want.

So under dataset_A you could have several files such as:

- dataset_A_2020_06_01

- dataset_A_2020_06_02

- dataset_A_2020_06_03

...

depending on the granularity of your partitioning settings.

I hope this clarifies.

rona

Hi,

With partitioning, I can generate a folder path including the partition dimension. But it's a path, not the file name.

Is there a way to use the partition information in the 'output file base name' in the Advanced settings of the dataset ?

If I use this file outside the context of DSS, I need to include the date in the file name.

Thanks

Thanks