Configuring nginx as reverse proxy for graylog

Introduction

Graylog2 is a powerful tool for log management and analysis tool. One such use case we had in my company is collect all logs of rails application running in 5 different servers in a single location so as to make debugging easy. It is built on the top of ElasticSearch, MongoDB and Java. First you need to set up graylog on your server. These links are likely to help you.

Once it is setup you want to access the web interface. It is running on port 9000.You could actually use a single port to connect with graylog REST api and web interface or two separate ports. This is my nginx configuration.

location / {
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Graylog-Server-URL https://example.in/api;
proxy_pass http://127.0.0.1:9000;
}

location /api/ {
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:12900/;
}

Graylog Configuration


web_listen_uri = http://0.0.0.0:9000/
rest_listen_uri = http://0.0.0.0:12900/
rest_transport_uri = http://127.0.0.1:12900/

Graylog as a very good community at https://community.graylog.org/. You may post your issues there .

Script to Monitor RabbitMQ Queue Messages

Below is a code I used to monitor rabbitmq queues. We were  using a microservices architecture. In our architecture each services are communicating using a rabbitmq broker. Services tend to publish message to broker and the broker  forwards these messages. One bottle neck is in this communication part.Due to some memory issues the consumers that consumes these messages got hang and the messages in a particular queue went too high .So I decided to write a script to sent an SMS and alert in slack channel whenever the message count becomes greater than the threshold.Note that I have used a gem slack notifier link: https://github.com/stevenosloan/slack-notifier. You could also just curl

 require 'net/http'
 require 'uri'
 require 'json'
 require 'slack-notifier'
 require 'sevak_publisher'

 CONFIG = YAML.load_file(File.join(__dir__, 'rabbitmq_config.yml'))
 def monitor_rabbitmq
   rabbitmqctl_url = CONFIG['rabbitmqctl']['url']
   rabbitmqctl_user = CONFIG['rabbitmqctl']['username']
   rabbitmqctl_password = CONFIG['rabbitmqctl']['password']
   uri = URI.parse("#{rabbitmqctl_url}/api/queues")
   request = Net::HTTP::Get.new(uri)
   request.basic_auth(rabbitmqctl_user, rabbitmqctl_password)
   req_options = { use_ssl: uri.scheme == 'https' }
   response = Net::HTTP.start(uri.hostname, uri.port, req_options)  do |http|
      http.request(request)
   end
   queue_details = JSON.parse(response.body)
   queue_details.each do |queue|
     output = { name: queue['name'],
                messages: {
                  total: queue['messages'],
                  ready: queue['messages_ready'],
                  Unacknowlged: queue['messages_unacknowledged']
                  },
                 node: queue['node'],
                 state: queue['state'],
                consumers: queue['consumers'] }
      if output[:messages][:ready] > 100
         sent_alert_slack("RabbitMQ QUEUE High! \n #{output[:messages][:ready]} :\n #{output}")
       end
    end
   end
   def sent_alert_slack(message)
      notifier = Slack::Notifier.new CONFIG['slack_settings']    ['notification_api'],
                             channel: '#rabbitmq-monitoring',
                             username: 'notifier'
      notifier.ping message
    end
   begin
    puts "\n", Time.now
    monitor_rabbitmq
  rescue => e
    puts "Error: #{e.message}"
end