Connect to a password protected opendsitro instance

DrissiReda

I tried to set up an opendistro connection.

I created an elasticsearch connection and entered these parameters:

Host: localhost

Port: 9200

Username: admin

Password: password

I get this error: "Connection Error: Server didn't answer 200 but 401 Unauthorized"

But when I do :

Host: admin:password@localhost

Port: 9200

Username:

Password:

I get this output : "Connection OK, found ES v. 7.10.2"

So I know this isn't a password or permission mismatch problem. Using curl both formats work:

- curl http://admin:password@localhost:9200

- curl -u admin:password http://localhost:9200

Did anyone ever run into the same problem? I'd like to avoid putting the password as plaintext.

Alexandru

Hi,

I was able to connect with either methods mentioned to an Opendistro elastic search :

e.g. adding username password directly in the connection worked fine

Also including the password in the URL username:password@locahost also works.

Could you confirm the version of DSS and Dialect chosen?

DrissiReda

Hello this is me using the method I want:

This is me using the method that works but that bothers me because of the plaintext password

Alexandru

Understood, can't really understand why that is not working for you since there should be no difference in how the credentials are passed with either method. Setting the username and password works for me on DSS 9 with SSL enabled on ElasticSearch.

Are you planning on using HTTP? If not can you try with HTTPS ( even a self-signed cert with Trust any Cert option enabled) and see if you get the same results. The way the credentials are passed via the username/password field is not different as far as I can tell.

DrissiReda

I'm only using http, not planning to switch to tls, since it's inside a kubernetes cluster. I don't understand why BasicAuth would be passed differently just because we add SSL.

Alexandru

I am struggling to set up a HTTP only with authentication.

What version of Open Distro ES do you currently have?

Would you be able to share your elasticsearch.yml config and plugins/opendistro_security/securityconfig/config.yml to see how you managed to get port 9200 workings without SSL and with authentication working?

I've tried several options including opendistro_security.ssl.http.enable to false doesn't seem to be enough to disabled SSL. While setting opendistro_security.disabled this will also remove the authentication.

DrissiReda

Opendistro version is 1.13.1

securityconfig/config.yml :

      _meta:
          type: "config"
          config_version: 2
        
        config:
          dynamic:
            authc:
              basic_internal_auth_domain:
                http_enabled: true
                transport_enabled: true
                order: 0
                http_authenticator:
                  type: basic
                  challenge: false
                authentication_backend:
                  type: internal

elasticsearch.yml :

network.host: 0.0.0.0
opendistro_security.ssl.http.enabled: false
        opendistro_security.ssl.transport.pemcert_filepath: node-crt.pem
opendistro_security.ssl.transport.pemkey_filepath: node-key.pem
opendistro_security.ssl.transport.pemtrustedcas_filepath: elk-root-ca.pem
        opendistro_security.ssl.transport.enforce_hostname_verification: false
opendistro_security.allow_unsafe_democertificates: false
opendistro_security.allow_default_init_securityindex: false
        
opendistro_security.audit.type: internal_elasticsearch
        opendistro_security.enable_snapshot_restore_privilege: true
        opendistro_security.check_snapshot_restore_write_privileges: true
opendistro_security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
cluster.routing.allocation.disk.threshold_enabled: false
        opendistro_security.audit.config.disabled_rest_categories: none
        opendistro_security.audit.config.disabled_transport_categories: none
        
opendistro_security.authcz.admin_dn:
    - "CN=admin,ST=Paris,C=FR"
        
opendistro_security.nodes_dn:
    - "CN=node,ST=Paris,C=FR"

internal_users:

_meta:
    type: "internalusers"
    config_version: 2
        
admin:
    hash: "$2y$12$j8RkEn5Gm1/OY2zDHN2nyO6v2mzT34P1w6v61LO95k.9fvsAOIPuC"
    reserved: true
    backend_roles:
    - "admin"
      description: "admin user"

Plugins used:

[2021-06-28T11:25:53,769][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded module [aggs-matrix-stats]
[2021-06-28T11:25:53,770][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded module [analysis-common]
[2021-06-28T11:25:53,770][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded module [geo]
[2021-06-28T11:25:53,770][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded module [ingest-common]
[2021-06-28T11:25:53,770][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded module [ingest-geoip]
[2021-06-28T11:25:53,770][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded module [ingest-user-agent]
[2021-06-28T11:25:53,770][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded module [kibana]
[2021-06-28T11:25:53,770][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded module [lang-expression]
[2021-06-28T11:25:53,770][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded module [lang-mustache]
[2021-06-28T11:25:53,770][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded module [lang-painless]
[2021-06-28T11:25:53,770][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded module [mapper-extras]
[2021-06-28T11:25:53,770][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded module [parent-join]
[2021-06-28T11:25:53,770][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded module [percolator]
[2021-06-28T11:25:53,770][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded module [rank-eval]
[2021-06-28T11:25:53,770][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded module [reindex]
[2021-06-28T11:25:53,770][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded module [repository-url]
[2021-06-28T11:25:53,771][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded module [transport-netty4]
[2021-06-28T11:25:53,771][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded plugin [opendistro-asynchronous-search]
[2021-06-28T11:25:53,771][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded plugin [opendistro-index-management]
[2021-06-28T11:25:53,771][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded plugin [opendistro-job-scheduler]
[2021-06-28T11:25:53,771][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded plugin [opendistro-reports-scheduler]
[2021-06-28T11:25:53,771][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded plugin [opendistro_security]
[2021-06-28T11:25:53,771][INFO ][o.e.p.PluginsService     ] [elasticsearch-0] loaded plugin [repository-s3]

I hope this has sufficient information for you

Alexandru

Hi,

Thanks for the additional information. I spun a cluster with your exact config.

However I was unable to replicate this issue on DSS 9.0.1+. Entering the username and password directly worked fine :

I was only able to replicate " Server didn't answer 200 but 401 Unauthorized when entering a bad username or password.

Does your actual password have any special characters? could you try generating another password? Perhaps that does not have special characters we perhaps not escaping correctly when passing the credentials.

Can you try to type instead of copy-pasting the password?

DrissiReda

Just for the purpose of this test, my credentials are actually just "admin" and "password". I've been entering them by hand to avoid any trailing characters remnant from a copy paste. And I still see the same behavior. I'm using dataiku 9.0.0 by the way.

EDIT: I have upgrade my dss instance to 9.0.1. I completely reinstalled everything and I'm still getting the same behavior....

Alexandru

Hi,

I've checked again the doing a tcpdump on the ES container I can see that request send is correctly formatted with Basic Auth and the base64 encoded username and password :

The request on the ES side is correct passing base64 encoded credential, also the actual request is identical if I switch to use admin:admin@ in the URL

22:04:56.085776 IP 172.17.0.1.57486 > 64200279f9fe.9200: Flags [P.], seq 118:274, ack 157, win 501, options [nop,nop,TS val 299027620 ecr 26709149], length 156
E.....@.@..y..........#..}N..%.)....X......
........GET / HTTP/1.1
Host: localhost:9200
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.10 (Java/11.0.10)
Authorization: Basic YWRtaW46YWRtaW4=

Could you check you browser console to see if you see the correct username/password being read. Also might want to double check via a tcpdump, if ES is in docker you can follow these steps :

docker build -t tcpdump - <<EOF 
FROM ubuntu 
RUN apt-get update && apt-get install -y tcpdump 
CMD tcpdump -i eth0 
EOF

docker run --tty --net=container:name-of-es-docker-container tcpdump tcpdump -N -A 'port 9200'

DrissiReda

Thank you for your reply.

I have tried listening with tcpdump and it seems that while using username/password fields instead of the host, dataiku does not pass any basicAuth parameters, here is a snippet of the result of tcpdumping dataiku's request:

06:39:44.073693 IP dataiku-0.38408 > elasticsearch-0.9200: Flags [S], seq 3900732977, win 28800, options [mss 1440,sackOK,TS val 21874076 ecr 0,nop,wscale 7], length 0
E..<..@.>..R
..j
.....#...v1......p.*..........
.M..........
06:39:44.073713 IP elasticsearch-0.9200 > dataiku-0.38408: Flags [S.], seq 323616196, ack 3900732978, win 28560, options [mss 1440,sackOK,TS val 953089302 ecr 21874076,nop,wscale 7], length 0
E..<..@.@.l.
...
..j#....I....v2..o............
8....M......
06:39:44.073978 IP dataiku-0.38408 > elasticsearch-0.9200: Flags [.], ack 1, win 225, options [nop,nop,TS val 21874076 ecr 953089302], length 0
E..4..@.>..Y
..j
.....#...v2.I.......q.....
.M..8...
06:39:44.074214 IP dataiku-0.38408 > elasticsearch-0.9200: Flags [P.], seq 1:134, ack 1, win 225, options [nop,nop,TS val 21874076 ecr 953089302], length 133
E.....@.>...
..j
.....#...v2.I.......U.....
.M..8...GET / HTTP/1.1
Host: elasticsearch.doc-store:9200
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.10 (Java/1.8.0_292)


06:39:44.074221 IP elasticsearch-0.9200 > dataiku-0.38408: Flags [.], ack 134, win 232, options [nop,nop,TS val 953089302 ecr 21874076], length 0
E..4x+@.@...
...
..j#....I....v............
8....M..
06:39:44.076433 IP elasticsearch-0.9200 > dataiku-0.38408: Flags [P.], seq 1:120, ack 134, win 232, options [nop,nop,TS val 953089304 ecr 21874076], length 119
E...x,@.@..?
...
..j#....I....v......~.....
8....M..HTTP/1.1 401 Unauthorized
content-type: text/plain; charset=UTF-8
content-length: 29

Authentication finally failed
06:39:44.076792 IP dataiku-0.38408 > elasticsearch-0.9200: Flags [.], ack 120, win 225, options [nop,nop,TS val 21874079 ecr 953089304], length 0
E..4..@.>..W
..j
.....#...v..I.<.....p.....
.M..8...
06:39:44.076999 IP dataiku-0.38408 > elasticsearch-0.9200: Flags [F.], seq 134, ack 120, win 225, options [nop,nop,TS val 21874079 ecr 953089304], length 0
E..4..@.>..V
..j
.....#...v..I.<.....o.....
.M..8...
06:39:44.077167 IP elasticsearch-0.9200 > dataiku-0.38408: Flags [F.], seq 120, ack 135, win 232, options [nop,nop,TS val 953089305 ecr 21874079], length 0
E..4x-@.@...
...
..j#....I.<..v............
8....M..
06:39:44.077426 IP dataiku-0.38408 > elasticsearch-0.9200: Flags [.], ack 121, win 225, options [nop,nop,TS val 21874079 ecr 953089305], length 0
E..4..@.>..U
..j
.....#...v..I.=.....m.....
.M..8...

And in the browser, only what I enter is reflected, username and password are indeed correct:

data   "{\"params\":{\"host\":\"elasticsearch.doc-store\",\"username\":\"admin\",\"password\":\"password\",\"port\":9200,\"ssl\":false,\"trustAnySSLCertificate\":false,\"dialect\":\"ES_7\"},\"name\":\"test_es\",\"type\":\"ElasticSearch\",\"creationTag\":{\"versionNumber\":0,\"lastModifiedBy\":{\"login\":\"admin\"},\"lastModifiedOn\":1624452461932},\"allowWrite\":true,\"allowManagedDatasets\":true,\"allowManagedFolders\":false,\"useGlobalProxy\":false,\"maxActivities\":0,\"customFields\":{},\"credentialsMode\":\"GLOBAL\",\"customBasicConnectionCredentialProviderParams\":[],\"usableBy\":\"ALL\",\"allowedGroups\":[],\"detailsReadability\":{\"readableBy\":\"NONE\",\"allowedGroups\":[]},\"indexingSettings\":{\"indexIndices\":false,\"indexForeignKeys\":false,\"indexSystemTables\":false},\"$canHaveProxy\":false}"