Best Dataiku schema for a human-review dataset of AI-generated product images?

MartynFoster735
MartynFoster735 Registered Posts: 1

Context

Our creative team has started pulling early visual drafts from AI image tools for concepting, including product mockups, storyboard frames, and social graphics. One source is Nano Banana 2 Lite (https://nanobanana2lite.tools/), a third-party independent site not affiliated with Google or DeepMind. It supports prompt-based generation and object-reference workflows. It is useful for fast iteration, but we now need a proper review pipeline before anything moves downstream.

The problem

I want to build a Dataiku dataset that lets human reviewers score and annotate drafts while keeping full lineage back to the prompt and reference images. Fields under consideration:

- image_path: pointer to the stored file (managed folder or external storage)
- prompt_text: full prompt used to generate the image
- reference_set_id: links to object-reference images
- reviewer_score: numeric or categorical scale
- rejection_reason: free text or controlled vocabulary
- version_lineage: parent/child relationship for regenerated or edited drafts

Questions for the community

1. Is a managed folder plus dataset-with-path-column pattern right for images, or is there a better Dataiku storage strategy?
2. For version lineage, has anyone modeled iterative generations with a self-referencing key, or is there a cleaner recipe-based approach?
3. For reviewer scoring, would you recommend a Dataiku app, a webapp, or a plugin-based review flow?

I am not looking for a finished pipeline, just advice on how others structure similar review datasets before committing to a schema. Happy to share what we land on once tested.

Answers

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023, Circle Member Posts: 2,719 Neuron
    edited July 21
    1. I would use a managed folder using local storage for fast reading for images that need to be reviewed. Then I would move them to a different managed folder under a cloud storage bucket so you can benefit for cheaper storage rates and PAYG rates rather than provisioning with overhead in local storage.
    2. Use a surrogate key and add versioning to it. You could use the file timestamp for versioning.
    3. I would use a Dataiku Webapp. If you are clever about it you could expose the images in the managed folder in your webapp using a symlink so that you can display them in the webapp without having to copy them to the static files folder for webapps. See link below:


    https://developer.dataiku.com/latest/tutorials/webapps/standard/custom-static-files-kb/index.html#gsc.tab=0

Setup Info
    Tags
      Help me…