Section 1: Grafana Responsibilities & Skills · Develop and maintain intuitive dashboards, visualizations, and panels for real-time monitoring. · Create and manage alert rules, notification policies, and contact points for effective alerting. · Build dashboards using stat, table, pie chart, bar chart, time series, and heatmap visualizations. · Perform advanced panel transformations including filtering, joining, merging, grouping, and calculations. · Implement dashboard variables, dynamic filtering, and dashboard linking. · Use JSON API and Infinity plugin (UQL) for nested API responses. · Integrate data sources including Elasticsearch, CloudWatch, Druid, and custom JSON APIs. · Manage dashboard and alert configurations as code using Bitbucket/GitHub repositories. · Deploy observability changes using GoCD pipelines following GitOps practices. · Validate dashboard and alert behavior post deployment. · Manage Grafana role-based access controls and security group integrations. · Configure alert silencing for maintenance windows and known downtimes. · Integrate alerts with Email, Teams, Slack, PagerDuty, ServiceNow, and Webhooks. · Perform troubleshooting and root cause analysis using dashboards, metrics, and Kibana logs. · Work with PromQL and Lucene queries for monitoring and log analysis. Section 2: Prometheus & VictoriaMetrics Responsibilities & Skills · Design and manage observability solutions using Prometheus and VictoriaMetrics. · Configure and maintain VictoriaMetrics single-node and clustered deployments. · Automate onboarding of services and nodes using service discovery, Ansible, and Terraform. · Create and manage Prometheus scrape jobs, relabeling rules, recording rules, and federation. · Configure Prometheus remote_write and VictoriaMetrics ingestion pipelines. · Manage metric retention policies, scrape intervals, and query optimizations.
