Monitoring & Resource Utilization

[AbeJoseph]

Need to augment DSS monitoring Tab with Resource Utilization - for jobs running on Local and Spark Clusters for Admins and End user usage.

Is it possible to get Sysdig integration as it already captures System, EKS, and AWS cloud watch metrics as of now. We need to get DSS processes and other metrics with partnership with Sysdig SaaS deployment which tracks and keeps history in much better graphical display.

[fsergot]

Hello,

DSS already has itself a feature to fetch resource consumption across the tools it orchestrates. It is called Resource Consumption Usage.

You can find the documentation of this feature here => Compute resource usage reporting

And a sample project walk-through here => How to Leverage Compute Resource Usage Data 

Regarding sysdig, their solution is very interesting although with a heavy inclination toward security rather than monitoring. Can you be more specific into what kind of integration you would like to see and what for?

Thanks

[fsergot]

Hello,

DSS already has itself a feature to fetch resource consumption across the tools it orchestrates. It is called Resource Consumption Usage.

You can find the documentation of this feature here => Compute resource usage reporting

And a sample project walk-through here => How to Leverage Compute Resource Usage Data 

Regarding sysdig, their solution is very interesting although with a heavy inclination toward security rather than monitoring. Can you be more specific into what kind of integration you would like to see and what for?

Thanks

[AbeJoseph]

We know the details of the Event Server and its usage which is complex structures and still are not able to parse the data properly without lack of data dictionary on the events datasets/schema.

We would like to get near real time dashboard for each instance aligned within DSS Monitoring tab – which should be available for all users not just admins (need to be out of Administration tab).

We were able to get Sysdig monitoring and gathering data via API for EKS Cluster usage for Dataiku. We can demo this very easily. Also, Sysdig has integration with AWS cloud watch for other application data to be integrated.

[AbeJoseph]

We know the details of the Event Server and its usage which is complex structures and still are not able to parse the data properly without lack of data dictionary on the events datasets/schema.

We would like to get near real time dashboard for each instance aligned within DSS Monitoring tab – which should be available for all users not just admins (need to be out of Administration tab).

We were able to get Sysdig monitoring and gathering data via API for EKS Cluster usage for Dataiku. We can demo this very easily. Also, Sysdig has integration with AWS cloud watch for other application data to be integrated.