Not getting JSON file with key-value format as output in Azure Blob Storage

Options
Shah
Shah Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 2

Hi , I am trying to store dataiku dataset in Azure blob storage as JSON file with proper key-value pair format but what I am getting in Azure blob storage is like nested list (without key names). I am using DSS engine in sync recipe. Please check the attached details and suggest me the solution. Thanks


Operating system used: Windows

Answers

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,717 Neuron
    Options

    Dataiku does not support using visual recipes to write the output as JSON. That preview that you see is to read the dataset as written by Dataiku, it's not really there for you to change the format of the output. In fact I would argue that drop down really shouldn't be there as you can't release select anything from there. This preview tab is used in Input datasets when you are loading files from folders or buckets. If you want to write a JSON file to a Azure bucket you need to create a managed folder insude the blob and then write the file to it using a Python recipe. See the following post:

    https://community.dataiku.com/t5/Using-Dataiku/How-to-save-the-output-of-the-python-code-recipe-which-is-in/td-p/12879

  • Shah
    Shah Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 2
    Options

    Hi @Turribeach
    , I get your point here. Seems the approach you suggested to create managed folder inside the Azure Blob and then write the file to it using Python recipe. But in my case I have directory defined in Azure blob storage and blob connection I am using is pointing to that and I have to place JSON file generated by Dataiku Python recipe there as a single JSON file. is it feasible to do in Dataiku and if yes then how ?

Setup Info
    Tags
      Help me…