The Diagnostics admin page provides links to external tools that can be used to troubleshoot issues in database storage and data transformation.

The following links to diagnostic tools are provided:

  • Data ingestion and ETL jobs:The Spark Master UI provides details of workers, running applications, running drivers, completed applications and completed drivers.

    For more details, see http://spark.apache.org/docs/latest/spark-standalone.html

  • Data lake and raw data storage:The HDFS or Hadoop system provides details of datanodes, datanode volume failures, snapshot summaries, startup progress, and utilities such as logs and browsing the file system.

    For more details, see https://en.wikipedia.org/wiki/Apache_Hadoop

  • Data warehouse:The Druid Coordinator or Console provides details of the data sources, cluster and indexing services.

    For more details, see http://druid.io/docs/latest/design/coordinator.html

Figure 1. Diagnostics