Community Conundrum 28: News Engagement is live! Read More

python indexing issue while executing a python code on dataset.

Level 3
python indexing issue while executing a python code on dataset.

I am having time_stampindexed dataset where it is supposed to be run accordingly in time_stamp order.

Even though it looks ordered in dataset, while running the plugin recipe index jumped from random dates.

after 2018 it ran over 2017, my algorithm needs to run in time sequential order to build the required output.

MicrosoftTeams-image.png

 

I need a quick suggestion for this kind of issue. 

 

@Pythonista or any valuable members of this group. please help me out

0 Kudos
1 Reply
Neuron
Neuron

@sasidharp can you provide a little more details? Are you running your Python code as part of a Python Code Recipe? Or a scenario script? Or other?

As far as I am aware, if your data (in a Pandas dataframe) is not sorted on date adn you create a datetime index from some date field/fields, you will indeed have a datetime index, but not necessarily in any order. If the data is small enough to fit in memory in DSS you can run the following to sort (before processing):

data_frame = data_frame.sort_index()

Alternatively, you could probably use a different recipe or logic to ensure that the input dataset is correctly sorted to begin with.

@sasidharp, Let me know if this is obvious to you and I missed some other detail that can help me provide better guidance. Hope this helps.

 

 

A banner prompting to get Dataiku DSS