LLM Batch requests

toppe

I am experiencing rate limit errors for my LLM requests.

Is there a simple method to batch my requests?

My thanks in advance.

Tom

Operating system used: Mac OS

Alexandru

Hi,

Currently, there is no direct way to batch request; for example, in the prompt recipe,e each row will be sent a seperate request to the LLM.

You can adjust the connection settings to increase the retries, reduce parallelism, or try obtain higher LLM rate limits on the LLM side.

Thanks