Using the next page url provided in the API Plugin

Emiel
Emiel Registered, Frontrunner 2022 Finalist, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant Posts: 22 ✭✭✭✭✭
edited July 16 in Plugins & Extending Dataiku

Hi all!

I'm trying to access the list applications endpoint of the Greenhouse API with the API plugin. The endpoint uses pagination and a next page is provided in the header. I've tried different setups with the API plugin, but I can't get it to work (while it works with a python script). The error I get is: 

Failed to read from python dataset : <class 'rest_api_client.RestAPIClientError'> : The api-connect plugin is stuck in a loop. Please check the pagination parameters.

I'm wondering how I can debug this and get it to work. Help is much appreciated!

Tagged:
· Share on FacebookShare on Twitter

Best Answer

  • AlexB
    AlexB Dataiker, Registered Posts: 68 Dataiker
    Answer ✓

    Hi !

    This type of pagination is not yet supported by the API plugin. For now you can only retrieve the first page, or feed the page numbers using the api-connect recipe and a "page" query parameter for instance.

    · Share on FacebookShare on Twitter

Answers

  • PhilipMannering
    PhilipMannering Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 1 ✭✭✭

    @AlexB
    Is this now supported? I cannot get it to work.

    · Share on FacebookShare on Twitter
  • AlexB
    AlexB Dataiker, Registered Posts: 68 Dataiker

    @PhilipMannering
    first check that the API you are trying to access is using RFC-5988 type pagination.

    - You will need to have a recent version of the plugin (v1.1.2 or later).

    - Then set the pagination mechanism to "Next page URL provided" and keep the "Key to next request URL" empty.

    - You might still have to set the "Next page URL is relative" on or off according to the API's specifications.

    · Share on FacebookShare on Twitter
Bookmark this topic

Categories

Setup Info
    Tags
      Help me…