TrevorHall Registered Posts: 15 ✭✭✭✭

November 2023 in Using Dataiku

Is it possible to control the rate at which a prompt recipe issues requests? I ran my first one yesterday and very quickly hit the TPM rate cap on our OpenAI model deployment.

0 · Share on FacebookShare on Twitter

Best Answer

• 

  AdrienL Dataiker, Alpha Tester Posts: 196 Dataiker

  November 2023 Answer ✓

  Hi,
  On the connection, there are some settings to control the rate of queries. There is no direct limit for TPM, but reducing parallelism will usually fix it.

  1 · Share on FacebookShare on Twitter

Answers

• 

  TrevorHall Registered Posts: 15 ✭✭✭✭

  November 2023

  Thanks! I set "Max parallelism" to 1 and that seems to be working. I also upped the retry delay to 2 seconds.

  1 · Share on FacebookShare on Twitter

Welcome!

It looks like you're new here. Sign in or register to get started.

Sign In

Register