DSS INTERNAL DATABASE ERROR

[LucasM]

Bonjour,

Je dispose de la version 7 de dss sur un serveur debian 9.

En l'occurrence, lors d'une tentative de redémarrage de l'application (backend, python...) via la commande DATA_DIR/bin/dss stop et start, il s'avère que j'ai rencontré l'erreur suivante (le backend.log renvoit la même chose) : DSS INTERNAL DATABASE ERROR: Failed to acquire connection to dss DB: jobs, caused by : CodedSQLException : Unable to open the following internal database file:........

La solution qui m'est proposée, est soit de reprendre à partir d'un backup soit de déplacer la bdd.

Or, je ne vois pas de quelle base de données il est question et comment l'y prendre concrètement 😞

Auriez-vous une idée à proposer à propos de cette erreur ?

Cordialement,

LucasM

---

Operating system used: Linix debian 9

[sergeyd]

Hi  @LucasM 

Usually, there should be a DB name right after this message. Something like this: 

Caused by: com.dataiku.dip.exceptions.CodedSQLException: Unable to open the following internal database file: jdbc:h2:/opt/dataiku/data/timelines/TEST;TRACE_LEVEL_SYSTEM_OUT=0;TRACE_LEVEL_FILE=4;LOCK_TIMEOUT=300000. Restore a backup or move the file away then restart DSS.

So in your case, this is looks to be an internal jobs.mv.db located inside <DATA_DIR>/databases/ directory. You will need to stop DSS, move the affected DB and start DSS back. 

If needed, you can share with us the backend.log file so we can review it.

[LucasM]

Thanks for your reply.

So, if i understand, in your case we need to move /opt/dataiku/data/timelines/TEST in another location ?

Sorry, i haven't handled the file backend.log at this moment 😞

[sergeyd]

Hi @LucasM,

Yes, in my example I was needed to move /opt/dataiku/data/timelines/TEST.mv.db file elsewhere. You should have another one. Based on the snippet you provided, I assume you have something like this: 

DSS INTERNAL DATABASE ERROR: Failed to acquire connection to DSS DB: jobs, caused by: CodedSQLException: Unable to open the following internal database file: jdbc:h2:tcp://<host>:<port>/<id>;TRACE_LEVEL_SYSTEM_OUT=0;TRACE_LEVEL_FILE=4;LOCK_TIMEOUT=300000. Restore a backup or move the file away then restart DSS., caused by: JdbcSQLException: General error: "java.lang.IllegalStateException: File corrupted in chunk 173734, expected page length 4..384, got 0 [1.4.195/6]" [50000-195]

 So as I mentioned previously, you will need to stop DSS, move elsewhere <DATA_DIR>/databases/jobs.mv.db (on older DSS versions it's jobs.h2.db) and start DSS back. 

[importthepandas]

Hi @sergeyd we just had something similar happen recently:

com.dataiku.dip.exceptions.CodedSQLException: Failed to acquire connection to DSS DB: dss_usage

 which silently fried our triggers for scenarios in certain instances for quite some time. is this similar to what is mentioned above and can be triaged the same way? also wondering why this occurs.

[AlexT]

Hi @importthepandas,
This can usually happen when there is a forceful shutdown of the DSS instance.

To improve stability for the runtime database, we usually suggest moving to external runtime database(Postgres) :

https://doc.dataiku.com/dss/latest/operations/runtime-databases.html#externally-hosting-runtime-data...


Thanks

[importthepandas]

Thanks @AlexT - that makes sense. We've since done exactly what you said 🙂 i think some of the old runtime stuff is still trying to reference the db in the error thrown. Is it worth moving it as well?

[AlexT]

Hi @importthepandas ,
It may be worth raising a support ticket with the instance diagnostics to get more context around the error in this case. 

[importthepandas]

im on it, thanks!