get the scenario name from get_details function

arielma2304

Hi

I've read this article:

jenkins-pipeline

and I have question regards the file dss_pipeline-master.zip\dss_pipeline-master\3_preprod_test\run_test.py :

dss_pipeline-master.zip

in line 17, there is this assert:

assert scenario_result.get_details()["scenarioRun"]["result"]["outcome"] == "SUCCESS"

Is there a way to add also the scenario name to this line? I'm asking since I want later in the Junit results to see also the name

fsergot

Hello,

This is a specific question on pytest so I will try to answer but have no advanced knowledge of it.

I guess this string added after the test is the parametrization parameter. It is added in conftest.py in the fucntion pytest_generate_tests(metafunc).

In the example, we add the scenario ID so I guess this is the scenario ID from DSS that you see. If you want to have another value, you can build the list in pytest_generate_tests using the scenario name instead of scenario ID. The issue is that you need to retrieve the ID from the name when executing the test (in run_test.py), so adding a little piece of code here.

An alternative might be to test if you can add both scenario ID & name in list_scenarios. Worst case, if this requires a simple string and not a dict or array, you can concatenate the ID & name using a specific separator (like '-' or '@') and parse it when executing the scenario.

Note that I am not sure what will be displayed in the unit test report in this case.

François

fsergot

Hello @arielma2304
,

You can indeed retrieve the scenario details through scenario.get_settings().get_raw(). Here is a short sample:

import dataikuapi
import pprint

pp = pprint.PrettyPrinter()

client = dataikuapi.DSSClient("http://localhost:13000/", "XYZ")
project = client.get_project("DKU_TSHIRTS")
print('-------------------')
print('SCENARIOS')
pp.pprint(project.list_scenarios())
print('-------------------')

print('-------------------')
scenario = project.get_scenario("test")
print('SCENARIOS')
pp.pprint("scenario name = {}".format(scenario.get_settings().get_raw()['name']))

print('-------------------')
print('SCENARIO RAW DEFINITION')
pp.pprint(scenario.get_settings().get_raw())

You can also add a specific message to an assert by adding a second parameter, see Using Assertions Effectively .

I would try something like:

assert scenario_result.get_details()["scenarioRun"]["result"]["outcome"] == "SUCCESS", "Scenario '%s' failed" % scenario.get_settings().get_raw()['name']

arielma2304

In your 2nd solution, it works only in case scenario fails, but how can I print the scenario name in case of success? doesn't the function get_details() stores it anywhere?

fsergot

Hello,

It depends where you want to print it?

Of course a simple print would print this in the log of the execution.

If you want to display more information in Jenkins (or another orchestration tool), you can investigate using "record property" of pytest to add information in the junit xml file produced: https://docs.pytest.org/en/stable/usage.html#record-property

You may have to tweak the Jenkins display to leverage this new data though.

arielma2304

Can you give me an example based on the article I've mentioned? In the article its says:
"Here, we make this nicer by dynamically creating one unit test per scenario. In the final report, we will have one report per scenario executed, making the report more meaningful"

But still, when I implement it in my Jenkins, what I'm missing is the names of the successful scenarios in Test Results:

run_test.test_run_scenario[scenario0] 

how can I see the real scenario name instead of scenario0?