Registered users can ask their own questions, contribute to discussions, and be part of the Community!
I am using Share drive to read files in Dataiku and i want the files which are older than 30 days to get removed from share drive.
Can we handle this from Dataiku?
Thanks in Advance
There is no built-in functionality for this in DSS. Usually, storage providers like S3 would have lifecycle rules you could leverage for this.
If you don't currently have such native functionality you may be able to leverage managed folders API in DSS in a Scenario to delete files older than x days.
from datetime import datetime, timedelta
client = dataiku.api_client()
project_key = "DKU_TUTORIAL_FIRSTPLUGIN"
odb_id = "eZBgqcZ3"
folder = dataikuapi.dss.managedfolder.DSSManagedFolder(client, project_key, odb_id)
contents = folder.list_contents()
thirty_days_ago = int((datetime.now() - timedelta(days=30)).timestamp() * 1000)
for item in contents['items']:
if item['lastModified'] < thirty_days_ago: