ValueError Load .md file format using Langchain DirectoryLoader

FarahAmirah
FarahAmirah Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 4

Hi, I was trying to read .md file format using Langchain library DirectoryLoader.

Here's the code.

import dataiku
import os
from langchain.document_loaders import DirectoryLoader

FILE_NAME = "alice_in_wonderland.md"

# Load the file and split it into smaller chunks
docs_folder = dataiku.Folder("file1") # Replace with your input folder id
file_path = os.path.join(docs_folder.get_path(),
FILE_NAME)
loader = DirectoryLoader(file_path)
doc = loader.load()

can you help?

md1.png


Operating system used: Almalinux (8.9)

Answers

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,117 Neuron

    The error is pretty clear, the package expects a directory path not a file path. Try passing a directory.

Setup Info
    Tags
      Help me…