Try your hand at analyzing royal sentiment in Dataiku DSS! Learn more

Problems using async in scenarios

Level 2
Problems using async in scenarios

I've gotten this error in many scenarios across a lot of versions of DSS, and always assumed it surely must be a known bug and that it would be fixed. But by this point I'm starting to wonder if somehow I'm doing something wrong. Using async = True seems pretty straight forward, but a lot of times I end up with the error below. Any ideas?



Thanks,

John



 




[2018/12/16-17:15:27.415] [Exec-279] [INFO] [process] - Traceback (most recent call last):
[2018/12/16-17:15:27.415] [Exec-279] [INFO] [process] - File "/home/dataiku/dss_data/scenarios/ERM_MASTER_PORTFOLIO/BUILD_MODEL_OUTPUTS/2018-12-16-17-10-38-992/custom-scenario/script.py", line 27, in <module>
[2018/12/16-17:15:27.416] [Exec-279] [INFO] [process] - while not sev_br.is_done():
[2018/12/16-17:15:27.416] [Exec-279] [INFO] [process] - File "/home/dataiku/dataiku-dss-5.0.4/python/dataiku/scenario/step.py", line 98, in is_done
[2018/12/16-17:15:27.416] [Exec-279] [INFO] [process] - }, err_msg="Failed to track step future")
[2018/12/16-17:15:27.416] [Exec-279] [INFO] [process] - File "/home/dataiku/dataiku-dss-5.0.4/python/dataiku/core/intercom.py", line 82, in backend_json_call
[2018/12/16-17:15:27.417] [Exec-279] [INFO] [process] - return _handle_json_resp(backend_api_post_call(path, data, **kwargs), err_msg = err_msg)
[2018/12/16-17:15:27.417] [Exec-279] [INFO] [process] - File "/home/dataiku/dataiku-dss-5.0.4/python/dataiku/core/intercom.py", line 148, in _handle_json_resp
[2018/12/16-17:15:27.417] [Exec-279] [INFO] [process] - raise Exception("%s: %s" % (err_msg, _get_error_message(err_data).encode("utf8")))
[2018/12/16-17:15:27.417] [Exec-279] [INFO] [process] - Exception: Failed to track step future: JobID not found : OwNLUtij. Running jobs: ["LZ7ni3cz","KHyjzJ2a","GtxHfmlh","1h3oQSvT","cd9JhCAF","JViR3tLW","z5gJUDu5"]


 

0 Kudos
2 Replies
Dataiker
Dataiker

Hi John,



We had a deeper look at this issue. The explanation is that after .is_done() has returned True once, it cannot be called again. So ideally you should write something like




scenario_1_done = False
scenario_2_done = False

while not scenario_1_done or not scenario_2_done:
if not scenario_1_done:
scenario_1_done = scenario_1.is_done()
if not scenario_2_done:
scenario_2_done = scenario_2.is_done()
time.sleep(1)


Hope it helps,



Alex

0 Kudos
Level 2
Author
Thanks, this appears to be working! This also explains why when I did it the way suggested in the example code:

while not scenario_1.is_done() or not scenario_2.is_done():
time.sleep(1)

that it seemed to always fail, while using nested loops the way that I did only failed sometimes. With nested loops, it should only fail if the scenario being checked by the inner loop finishes first.
0 Kudos
Labels (3)