datadog-init-script(Scala)

Import Notebook

Init Script: Install Datadog Agent for Spark and System Monitoring

This init script installs the Datadog agent to collect system metrics on every node in a cluster. It also configures the cluster for Spark monitoring.

  1. Configure <init-script-folder> with the location to put the init script.
  2. Replace <your-api-key> in the DD_API_KEY parameter with your Datadog account key.
  3. Run this notebook to create the script datadog-install-driver-only.sh.
  4. Configure a cluster with the datadog-install-driver-only.sh cluster-scoped init script using the UI, Databricks CLI, or by invoking the Clusters API.
%python 
 
dbutils.fs.put("dbfs:/<init-script-folder>/datadog-install-driver-only.sh","""
#!/bin/bash
 
echo "Running on the driver? $DB_IS_DRIVER"
echo "Driver ip: $DB_DRIVER_IP"
 
cat <<EOF >> /tmp/start_datadog.sh
#!/bin/bash
 
if [ \$DB_IS_DRIVER = "TRUE" ]; then
  echo "On the driver. Installing Datadog ..."
  
  # install the Datadog agent
  DD_API_KEY=<your-api-key> bash -c "\$(curl -L https://raw.githubusercontent.com/DataDog/datadog-agent/master/cmd/agent/install_script.sh)"
  
  # WAITING UNTIL MASTER PARAMS ARE LOADED, THEN GRABBING IP AND PORT
  while [ -z \$gotparams ]; do
    if [ -e "/tmp/driver-env.sh" ]; then
      DB_DRIVER_PORT=\$(grep -i "CONF_UI_PORT" /tmp/driver-env.sh | cut -d'=' -f2)
      gotparams=TRUE
    fi
    sleep 2
  done
 
  current=\$(hostname -I | xargs)  
  
  # WRITING SPARK CONFIG FILE FOR STREAMING SPARK METRICS
  echo "init_config:
instances:
    - resourcemanager_uri: http://\$DB_DRIVER_IP:\$DB_DRIVER_PORT
      spark_cluster_mode: spark_driver_mode
      cluster_name: \$current" > /etc/datadog-agent/conf.d/spark.yaml
 
  # RESTARTING AGENT
  sudo service datadog-agent restart
 
fi
EOF
 
# CLEANING UP
if [ \$DB_IS_DRIVER = "TRUE" ]; then
  chmod a+x /tmp/start_datadog.sh
  /tmp/start_datadog.sh >> /tmp/datadog_start.log 2>&1 & disown
fi
""", True)
Wrote 1220 bytes. Out[1]: True