Copying a file to a different folder using R

jake_overbeek

I am trying to use an R-script to copy a file (model trained using the forecast plugin, saved as .rdata) from one folder to another folder, recreating the directory hierarchy. One DSS folder is used as input while another is used as output.

I am trying to achieve this with

dkuManagedFolderCopyFromLocal

using parameters resembling dkuManagedFolderCopyFromLocal("KH78xJhS", "YIYWlHgw/versions/.../"). Am I completely missing something or using this function wrong?
Kind regards in advance!

Alex_Combessie

Hi,

Note: unlink assumes that your folder is on your local filesystem (usually the filesystem_folders connection). You can use dkuManagedFolderPath on the copied folder . Then, you can unlink("<dkuManagedFolderPath>/versions/<specific timestamp>", recursive=TRUE).

Best regards,

Alex

Alex_Combessie

Hi,

The forecast plugin expects a specific folder structure to hold its model. The general approach you are taking is correct, but needs caution.

The required structure is:

• If your forecasting model is not partitioned: ./versions/{timestamp}/models.RData
• If your forecasting model is partitioned: ./${partition_id}/versions/{timestamp}/models.RData

What exactly do you want to achieve by copying the file to a different folder? Depending on your goal, we will be able to advise the best way forward.

Cheers,

Alex

jake_overbeek

Thanks for the kind and quick response! As far as I understood, the forecasting recipe will pick the best model based on some error metric. I want to compare the metrics to those of the previously trained models (in case this isnt the case already, correct me if I'm wrong) and use one of those models to do forecasting. Thus I want to copy the model and hierarchy of the folder to a new folder on which I run the forecasting recipe.

Alex_Combessie

Hello,

Thanks, I understand what you want to achieve. Unfortunately, it is not possible to retrieve previous evaluation results by manipulating the model folder. The folder only stores saved models and does not keep evaluation results. I have passed your request to our product team to incorporate it in a future plugin release.

[ADDITION]

However, it would be possible to run the "Forecast future values and get historical residuals" recipe on the main model folder and again on a copy of the folder, where you will manually delete the latest version. You can then compare the two future values and residuals to the actuals. Note that this would not be equivalent to the evaluation methods used in the Train models recipe.

Best regards,

Alex

jake_overbeek

I am already able to figure out how to compare the error metrics of the models and the path of were the 'best' model is saved. So it would be solely copying that .rdata file to a different folder. Would that be possible?

Alex_Combessie

Hi,

The simplest way is to copy the folder using the "Merge folder" recipe with one input. Then you can delete specific versions by filtering on the timestamp folder under versions/.

That way the folder will still be valid for use by the Forecast future values recipe.

Hope it helps,

Alex

jake_overbeek

How would you then delete certain versions (automatically)?

Alex_Combessie

Hi,

You can list the directory names in the "versions" folder and sort them by date. Then you can delete the version you want recursively. You can checkout the R unlink function.

Hope it helps,

Alex

jake_overbeek

But would it then be
unlink("YckcGxpE/versions/timestamp", recursive=TRUE)
or
unlink("trained_models_copy/versions/timestamp", recursive=TRUE)
or something else?
And would the folder with copied models (through Merge folder) then be the input or the output or neither?