New to Dataiku DSS? Try out our NEW Quick Start Programs today and get onboarded on the product in just one hour! Let's go

Dataiku not starting after reboot

Solved!
sandeep
Level 2
Level 2
Dataiku not starting after reboot

Hi,

I ran the script that allowed Dataiku installer script to run automatically upon system boot.

sudo -i "/home/dataiku/dataiku-dss-8.0.4/scripts/install/install-boot.sh" "/home/dataiku/DATA_DIR" dataiku

However, after reboot, Dataiku doesn't turn on (see below code snippet). It seems that there is some issue with running nginx.

-bash-4.2$ DATA_DIR/bin/dss status
backend                          RUNNING   pid 7677, uptime 0:01:55
ipython                          RUNNING   pid 7675, uptime 0:01:55
nginx                            FATAL     Exited too quickly (process log may have details)

 Based on https://community.dataiku.com/t5/Setup-Configuration/dss-does-not-start-anymore-nginx-backoff/m-p/25... I examined the nginx logs in DATA_DIR/run/nginx.log based on  and , I found the following:

nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/01/08 15:13:46 [error] 17560#0: *1 open() "/home/dataiku/dataiku-dss-8.0.4/frontend/robots.txt" failed (2: No such file or directory), client: 10.85.235.54, server: , request: "GET /robots.txt HTTP/1.1", host: "10.85.235.54:11000"
2021/01/08 15:13:46 [error] 17560#0: *1 open() "/home/dataiku/dataiku-dss-8.0.4/frontend/static/dataiku/img/glyphicons-halflings.png" failed (2: No such file or directory), client: 10.85.235.54, server: , request: "GET /static/dataiku/img/glyphicons-halflings.png HTTP/1.1", host: "10.85.235.54:11000", referrer: "http://10.85.235.54:11000/static/dataiku/css/style.css"
2021/01/08 15:13:46 [error] 17560#0: *1 open() "/home/dataiku/dataiku-dss-8.0.4/frontend/static/dataiku/img/glyphicons-halflings-white.png" failed (2: No such file or directory), client: 10.85.235.54, server: , request: "GET /static/dataiku/img/glyphicons-halflings-white.png HTTP/1.1", host: "10.85.235.54:11000", referrer: "http://10.85.235.54:11000/static/dataiku/css/style.css"
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/03/05 15:53:01 [emerg] 9059#0: open() "/home/dataiku/DATA_DIR/install-support/nginx.conf" failed (13: Permission denied)
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/03/05 15:53:02 [emerg] 9759#0: open() "/home/dataiku/DATA_DIR/install-support/nginx.conf" failed (13: Permission denied)
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/03/05 15:53:05 [emerg] 10156#0: open() "/home/dataiku/DATA_DIR/install-support/nginx.conf" failed (13: Permission denied)
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/03/05 15:53:08 [emerg] 11377#0: open() "/home/dataiku/DATA_DIR/install-support/nginx.conf" failed (13: Permission denied)
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/03/05 17:41:08 [emerg] 7680#0: open() "/home/dataiku/DATA_DIR/install-support/nginx.conf" failed (13: Permission denied)
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/03/05 17:41:10 [emerg] 7779#0: open() "/home/dataiku/DATA_DIR/install-support/nginx.conf" failed (13: Permission denied)
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/03/05 17:41:12 [emerg] 7814#0: open() "/home/dataiku/DATA_DIR/install-support/nginx.conf" failed (13: Permission denied)
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/03/05 17:41:15 [emerg] 7849#0: open() "/home/dataiku/DATA_DIR/install-support/nginx.conf" failed (13: Permission denied)
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/03/05 17:57:35 [error] 8723#0: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 10.85.239.3, server: , request: "GET /dip/api/shaker/get-processors-library/ HTTP/1.1", upstream: "http://127.0.0.1:11001/dip/api/shaker/get-processors-library/", host: "10.85.235.54:11000", referrer: "http://10.85.235.54:11000/home/"
2021/03/05 17:57:35 [error] 8723#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 10.85.239.3, server: , request: "GET /dip/api/shaker/list-custom-formulas-functions HTTP/1.1", upstream: "http://127.0.0.1:11001/dip/api/shaker/list-custom-formulas-functions", host: "10.85.235.54:11000", referrer: "http://10.85.235.54:11000/home/"
2021/03/05 17:57:35 [error] 8723#0: *2 connect() failed (111: Connection refused) while connecting to upstream, client: 10.85.239.3, server: , request: "GET /dip/api/shaker/get-expression-syntax HTTP/1.1", upstream: "http://127.0.0.1:11001/dip/api/shaker/get-expression-syntax", host: "10.85.235.54:11000", referrer: "http://10.85.235.54:11000/home/"
2021/03/05 17:57:35 [error] 8723#0: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 10.85.239.3, server: , request: "GET /dip/api/datasets/get-types/ HTTP/1.1", upstream: "http://127.0.0.1:11001/dip/api/datasets/get-types/", host: "10.85.235.54:11000", referrer: "http://10.85.235.54:11000/home/"
2021/03/05 17:57:35 [error] 8723#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 10.85.239.3, server: , request: "GET /dip/api/datasets/get-common-charsets/ HTTP/1.1", upstream: "http://127.0.0.1:11001/dip/api/datasets/get-common-charsets/", host: "10.85.235.54:11000", referrer: "http://10.85.235.54:11000/home/"
2021/03/05 17:57:35 [error] 8723#0: *2 connect() failed (111: Connection refused) while connecting to upstream, client: 10.85.239.3, server: , request: "GET /dip/api/datasets/get-format-types/ HTTP/1.1", upstream: "http://127.0.0.1:11001/dip/api/datasets/get-format-types/", host: "10.85.235.54:11000", referrer: "http://10.85.235.54:11000/home/"
2021/03/05 17:57:35 [error] 8723#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 10.85.239.3, server: , request: "GET /dip/api/flow/recipes/get-types-descriptors HTTP/1.1", upstream: "http://127.0.0.1:11001/dip/api/flow/recipes/get-types-descriptors", host: "10.85.235.54:11000", referrer: "http://10.85.235.54:11000/home/"
2021/03/05 17:57:35 [error] 8723#0: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 10.85.239.3, server: , request: "GET /dip/api/get-configuration HTTP/1.1", upstream: "http://127.0.0.1:11001/dip/api/get-configuration", host: "10.85.235.54:11000", referrer: "http://10.85.235.54:11000/home/"
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/03/05 18:05:19 [emerg] 7673#0: open() "/home/dataiku/DATA_DIR/install-support/nginx.conf" failed (13: Permission denied)
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/03/05 18:05:21 [emerg] 7780#0: open() "/home/dataiku/DATA_DIR/install-support/nginx.conf" failed (13: Permission denied)
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/03/05 18:05:23 [emerg] 7817#0: open() "/home/dataiku/DATA_DIR/install-support/nginx.conf" failed (13: Permission denied)
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/03/05 18:05:27 [emerg] 7848#0: open() "/home/dataiku/DATA_DIR/install-support/nginx.conf" failed (13: Permission denied)
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/03/05 18:19:34 [emerg] 7676#0: open() "/home/dataiku/DATA_DIR/install-support/nginx.conf" failed (13: Permission denied)
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/03/05 18:19:35 [emerg] 7783#0: open() "/home/dataiku/DATA_DIR/install-support/nginx.conf" failed (13: Permission denied)
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/03/05 18:19:38 [emerg] 7821#0: open() "/home/dataiku/DATA_DIR/install-support/nginx.conf" failed (13: Permission denied)
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/03/05 18:19:42 [emerg] 7853#0: open() "/home/dataiku/DATA_DIR/install-support/nginx.conf" failed (13: Permission denied)

Can you help me to interpret this log file or help me resolve this issue to enable the nginx process to start properly after reboot?

0 Kudos
1 Solution
sergeyd
Dataiker
Dataiker

Hi @sandeep 

You have SELinux enabled on the host. Set it to permissive instead of enforcing. 

That's one of the requirements: https://doc.dataiku.com/dss/8.0/installation/requirements.html

 

View solution in original post

3 Replies
sergeyd
Dataiker
Dataiker

Hi @sandeep 

You have SELinux enabled on the host. Set it to permissive instead of enforcing. 

That's one of the requirements: https://doc.dataiku.com/dss/8.0/installation/requirements.html

 

View solution in original post

sandeep
Level 2
Level 2
Author

Thanks Sergey - this worked perfectly! Just wanted to check with you on something.

While SELinux was enabled, I was able to manually start nginx by running:

DATA_DIR/bin/dss start

However, after SELinus is disabled, nginx started automatically on reboot. Can you explain why this is the case?

0 Kudos
sergeyd
Dataiker
Dataiker

Yes, that's correct behaviour. The difference is that during reboot systemctl starts DSS. You should have added a boot-up script for this. That falls under SELinux security while starting/restarting DSS manually via dss binary has fewer limitations. 

A banner prompting to get Dataiku DSS