Discover this year's submissions to the Dataiku Frontrunner Awards and give kudos to your favorite use cases and success stories!READ MORE

Import project. Connection remapping for dataiku-managed-storage

DrEdu
Level 1
Import project. Connection remapping for dataiku-managed-storage

Hi! 

I'm trying to export a Dataiku project from the Dataiku online service/version into a local instance. Export goes well but at import issues appear as below:

Issues were encountered

  • ERROR
    Missing connection
    Missing connection: Connection missing for dataset baseline_fixed (not remapped): dataiku-managed-storage (EC2)
 
 
I tried to remap but the available options are related for the local file system and It seems that a EC2-like storage option is expected (as illustrated below) . Any ideas or advice?
 
Screen Shot 2022-08-07 at 14.10.29.png
Operating system used: macOS 12
Operating system used: macOS 12
0 Kudos
2 Replies
AlexT
Dataiker
Dataiker

Hi @DrEdu,

Indeed if you export a project with S3 connections and try to map it to local filesystem, this will not be compatible. Dataiku Online default datasets are typically parquet which is not compatible with the local filesystem. The best solution would be to add an actual S3 connection if possible.

If not  I would suggest you try the following workaround :

1) Create a mock S3 connection on your local DSS, it doesn't have to be a working connection. 

2) Then edit the connection/dataset types in the flow to the local filesystem instead and change any parquet to CSV. 

You can do this from dataset view G + D, Action select all S3 datasets - Other Actions change connection and choose the CSV filesystem instead of parquet then rebuild your flow. 

Screenshot 2022-08-08 at 09.02.44.png

0 Kudos
CatalinaS
Dataiker
Dataiker

Hi @DrEdu ,

The issue occurs at project import because you don't have EC2 connection type on your local instance. For connection mapping you will need a similar connection type to match the connection.

The solution to this issue is to change the connection type of your dataset on your Dataiku online. You can change this by selecting the dataset -> "Change Connection" to a connection that you do have available for use on your local instance. 

After that you can re-export the project and you should be able to import it into your local environment.

0 Kudos

Labels

?
Labels (2)
A banner prompting to get Dataiku