Invalid Scenario step logic condition should cause scenario failure

Turribeach

I have noted a very dangerous behavior in the latest v12 release although I believe this has been in DSS for a long while. DSS will not cause a scenario failure or even a warning if you have an invalid Scenario step logic. For instance I created a scenario step and set two variables:

{"var1": 123,   "var2": 456}

Then I create another step and I set the Run this step to "If condition satisfied" and set the condition to:

var1 <> var2 && outcome == 'SUCCESS'

This is an invalid condition since the not equal boolean operator is not <> but != (see formula documentation). I then run the scenario and the scenario ends in success and the conditional second step is not executed. The following INFO message is shown in the scenario log:

[2024/07/17-14:10:39.470] [FT-ScenarioThread-IDmwE1V8-10847] [INFO] [dip.reporter.lookup] scenario CT_TEST.TEST_BAD_LOGICAL_EXPRESSION#2024-07-17-14-10-39-429 - [ct: 13] Cannot evaluate expression 'var1 <> var2 && outcome == 'SUCCESS'' as formula
java.lang.IllegalArgumentException: Incorrect formula: 'var1 <> var2 && outcome == 'SUCCESS'' : Missing number, string, identifier, regex or parenthesized expression (Parsing error at offset 6)
	at com.dataiku.dip.expressions.Expression.<init>(Expression.java:102)
	at com.dataiku.dip.expressions.Expression.<init>(Expression.java:88)
	at com.dataiku.dip.expressions.Expression.<init>(Expression.java:78)
	at com.dataiku.dip.expressions.Expression.<init>(Expression.java:70)
	at com.dataiku.dip.expressions.Expression.<init>(Expression.java:62)
	at com.dataiku.dip.server.notifications.VariableLookup.lookup(VariableLookup.java:119)
	at com.dataiku.dip.scheduler.scenarios.StepBasedScenarioRunner.shouldStepRun(StepBasedScenarioRunner.java:307)
	at com.dataiku.dip.scheduler.scenarios.StepBasedScenarioRunner.run(StepBasedScenarioRunner.java:225)
	at com.dataiku.dip.scheduler.ScenarioThread.execute(ScenarioThread.java:146)
	at com.dataiku.dip.futures.FutureThreadBase.run(FutureThreadBase.java:136)
Caused by: com.google.refine.expr.ParsingException: Missing number, string, identifier, regex or parenthesized expression (Parsing error at offset 6)
	at com.google.refine.grel.Parser.makeException(Parser.java:93)
	at com.google.refine.grel.Parser.parseFactor(Parser.java:300)
	at com.google.refine.grel.Parser.parseTerm(Parser.java:194)
	at com.google.refine.grel.Parser.parseSubExpression(Parser.java:175)
	at com.google.refine.grel.Parser.parseExpression(Parser.java:165)
	at com.google.refine.grel.Parser.parseBooleanFactor(Parser.java:149)
	at com.google.refine.grel.Parser.parseBooleanTerm(Parser.java:128)
	at com.google.refine.grel.Parser.parseBooleanExpression(Parser.java:111)
	at com.google.refine.grel.Parser.parseExpressionRoot(Parser.java:99)
	at com.google.refine.grel.Parser.<init>(Parser.java:79)
	at com.google.refine.grel.Parser.<init>(Parser.java:68)
	at com.dataiku.dip.expressions.Expression.<init>(Expression.java:94)

The expected result is that the scenario ends in failure since the step condition is invalid and can not be evaluated.  In general terms the expectation is that a tool that lets you write user code should produce a failure if the user code is invalid. This is of particular importance for scenario steps which are a key component of the Dataiku automation feature. Adding to this is that in any other place where you can use the formula language an invalid formula will result in a failure. Yet in this case the error is completely ignored and logged as an "INFO" event. This is highly inconsistent, dangerous (as it can fail silently) and makes using the "If condition satisfied" option a non-starter.