Log nested exceptions

natejgardner
natejgardner Neuron, Registered, Neuron 2022, Neuron 2023 Posts: 151 Neuron
edited July 16 in Product Ideas

It seems that when running jobs, if a nested exception occurs, only the top level exception is logged. This makes debugging very difficult, since the true error is not visible. Many Teradata errors occur as nested exceptions, so it would be very helpful if the Dataiku logger could tree down to the complete exception and write that to the log so the underlying issue behind these errors can be understood. I've gotten these messages for everything from out of space to invalid characters to overflowing a field. While I usually know to look for these conditions manually, it's very time consuming. Unfortunately, this sort of error also usually happens after a long execution time. For example, this one took 56 minutes to error out. That's a long iteration loop for trial and error processes, so a little more info here would save me hours. Example:

[14:48:05] [ERROR] [dku.output.sql] - Nested exception
java.sql.SQLException: [Teradata JDBC Driver] [TeraJDBC 16.20.00.12] [Error 1339] [SQLState HY000] A failure occurred while executing a PreparedStatement batch request. The parameter set was not executed and should be resubmitted individually using the PreparedStatement executeUpdate method.
   at com.teradata.jdbc.jdbc_4.util.ErrorFactory.cloneJDBCException(ErrorFactory.java:171)
   at com.teradata.jdbc.jdbc_4.util.ErrorFactory.makeChainedBatchJDBCException(ErrorFactory.java:164)
   at com.teradata.jdbc.jdbc_4.statemachine.PreparedBatchStatementController.handleRunException(PreparedBatchStatementController.java:98)
   at com.teradata.jdbc.jdbc_4.statemachine.StatementController.runBody(StatementController.java:145)
   at com.teradata.jdbc.jdbc_4.statemachine.PreparedBatchStatementController.run(PreparedBatchStatementController.java:58)
   at com.teradata.jdbc.jdbc_4.TDStatement.executeStatement(TDStatement.java:389)
   at com.teradata.jdbc.jdbc_4.TDPreparedStatement.executeBatchDMLArray(TDPreparedStatement.java:256)
   at com.teradata.jdbc.jdbc_4.TDPreparedStatement.executeBatch(TDPreparedStatement.java:2754)
   at com.dataiku.dip.datasets.sql.SQLTableOutput$SQLTableOutputWriter.emitRow(SQLTableOutput.java:230)
   at com.dataiku.dip.dataflow.exec.stream.ToDatasetStreamer$Streamer.processRow(ToDatasetStreamer.java:131)
   at com.dataiku.dip.datalayer.ProcessorOutputToSIP.emitRow(ProcessorOutputToSIP.java:24)
   at com.dataiku.dip.recipes.code.sql.SQLQueryRecipeUtils.streamSingleRow(SQLQueryRecipeUtils.java:296)
   at com.dataiku.dip.recipes.code.sql.AbstractSQLQueryRecipeRunner.runRegularToDataset(AbstractSQLQueryRecipeRunner.java:225)
   at com.dataiku.dip.dataflow.exec.sql.SQLQueryRecipeRunner.runRegular(SQLQueryRecipeRunner.java:223)
   at com.dataiku.dip.dataflow.exec.sql.SQLQueryRecipeRunner.run(SQLQueryRecipeRunner.java:164)
   at com.dataiku.dip.dataflow.exec.MultiEngineRecipeRunner.run(MultiEngineRecipeRunner.java:203)
   at com.dataiku.dip.dataflow.jobrunner.ActivityRunner$FlowRunnableThread.run(ActivityRunner.java:380)

0
0 votes

New · Last Updated

Comments

  • natejgardner
    natejgardner Neuron, Registered, Neuron 2022, Neuron 2023 Posts: 151 Neuron

    Just following up on this, it'd really help quality of life for Teradata users to have this. Basically, there's very little error logging for Teradata right now, since the majority of errors are nested exceptions. It's very difficult to troubleshoot failing recipes without errors being logged.

Setup Info
    Tags
      Help me…