Thread safe exclusive set variable in projects

[tomas]

Hi,

 just discovered when running multiple python recipes the the DSS Project API - `update_variables` is not thread safe, if when two updates run in the same time, one process can set a variable and the other can unset it.

Example:

project.get_variables().get('local')
{}
---
# These two recipes run in the same time:
# RecipeA - updates only A
project.update_variables({'A': 1}, 'local')
# RecipeB - updates only B
project.update_variables({'B': 2}, 'local')
---
project.get_variables().get('local')
{'B': 2}
# The result can vary also sometimes {'A': 1} or {'A': 1, 'B': 2}
# The correct result should be always {'A': 1, 'B': 2}

Before I bumped into this issue I was using get_variables to fetch the dict, modify a key, and set_variables.

So I assumed the update_variables (taking only a set of keys) will be implemented in a atomic way. But it looks like the implementation is "get all vars" and "set all vars" without any kind of locking.

[Clément_Stenac · Accepted solution]

Hi,

We confirm the behavior you are observing.

Thanks for the feature request, we will be considering it for a future release.

[tomas]

Thanks for confirming. I came up with not a 100% reliable, but still usable workaround:

def set_project_var(project, key, value, where='local'):
    # Sets the key:value to the project's variable
    # where - can be local or standard
    # Race condition can happen so we set and check until it is set
    project.update_variables({key: value}, type=where)
    while True:
        # wait randomly up to 500ms
        sleep(0.5 * random())
        set_value = project.get_variables().get(where, {}).get(key)
        if not set_value or set_value != value:
            print('>> Race condition the previous set was not succesful')
            sleep(0.5 * random())
            project.update_variables({key: value}, type=where)
        else:
            return