New to Dataiku DSS? Try out our NEW Quick Start Programs today and get onboarded on the product in just one hour! Let's go

Problems using async in scenarios

Solved!
John_Kelly
Level 2
Problems using async in scenarios

I've gotten this error in many scenarios across a lot of versions of DSS, and always assumed it surely must be a known bug and that it would be fixed. But by this point I'm starting to wonder if somehow I'm doing something wrong. Using async = True seems pretty straight forward, but a lot of times I end up with the error below. Any ideas?



Thanks,

John



 




[2018/12/16-17:15:27.415] [Exec-279] [INFO] [process] - Traceback (most recent call last):
[2018/12/16-17:15:27.415] [Exec-279] [INFO] [process] - File "/home/dataiku/dss_data/scenarios/ERM_MASTER_PORTFOLIO/BUILD_MODEL_OUTPUTS/2018-12-16-17-10-38-992/custom-scenario/script.py", line 27, in <module>
[2018/12/16-17:15:27.416] [Exec-279] [INFO] [process] - while not sev_br.is_done():
[2018/12/16-17:15:27.416] [Exec-279] [INFO] [process] - File "/home/dataiku/dataiku-dss-5.0.4/python/dataiku/scenario/step.py", line 98, in is_done
[2018/12/16-17:15:27.416] [Exec-279] [INFO] [process] - }, err_msg="Failed to track step future")
[2018/12/16-17:15:27.416] [Exec-279] [INFO] [process] - File "/home/dataiku/dataiku-dss-5.0.4/python/dataiku/core/intercom.py", line 82, in backend_json_call
[2018/12/16-17:15:27.417] [Exec-279] [INFO] [process] - return _handle_json_resp(backend_api_post_call(path, data, **kwargs), err_msg = err_msg)
[2018/12/16-17:15:27.417] [Exec-279] [INFO] [process] - File "/home/dataiku/dataiku-dss-5.0.4/python/dataiku/core/intercom.py", line 148, in _handle_json_resp
[2018/12/16-17:15:27.417] [Exec-279] [INFO] [process] - raise Exception("%s: %s" % (err_msg, _get_error_message(err_data).encode("utf8")))
[2018/12/16-17:15:27.417] [Exec-279] [INFO] [process] - Exception: Failed to track step future: JobID not found : OwNLUtij. Running jobs: ["LZ7ni3cz","KHyjzJ2a","GtxHfmlh","1h3oQSvT","cd9JhCAF","JViR3tLW","z5gJUDu5"]


 

0 Kudos
1 Solution
Alex_Combessie
Dataiker
Dataiker

Hi John,



We had a deeper look at this issue. The explanation is that after .is_done() has returned True once, it cannot be called again. So ideally you should write something like




scenario_1_done = False
scenario_2_done = False

while not scenario_1_done or not scenario_2_done:
if not scenario_1_done:
scenario_1_done = scenario_1.is_done()
if not scenario_2_done:
scenario_2_done = scenario_2.is_done()
time.sleep(1)


Hope it helps,



Alex

View solution in original post

0 Kudos
2 Replies
Alex_Combessie
Dataiker
Dataiker

Hi John,



We had a deeper look at this issue. The explanation is that after .is_done() has returned True once, it cannot be called again. So ideally you should write something like




scenario_1_done = False
scenario_2_done = False

while not scenario_1_done or not scenario_2_done:
if not scenario_1_done:
scenario_1_done = scenario_1.is_done()
if not scenario_2_done:
scenario_2_done = scenario_2.is_done()
time.sleep(1)


Hope it helps,



Alex

View solution in original post

0 Kudos
John_Kelly
Level 2
Author
Thanks, this appears to be working! This also explains why when I did it the way suggested in the example code:

while not scenario_1.is_done() or not scenario_2.is_done():
time.sleep(1)

that it seemed to always fail, while using nested loops the way that I did only failed sometimes. With nested loops, it should only fail if the scenario being checked by the inner loop finishes first.
0 Kudos
Labels (3)
A banner prompting to get Dataiku DSS