Scalable Logging in Rails using Log4r and graylog

Rails default logger is quite simple.but it becomes quite difficult to debug in production as the app grows big. Imagine  you have 5 different app servers and you have to grep all the log files. Some simple shell scripting could ease the pain but still log file is quite messed up and it still wont help you to analyse  which url returned 500, time taken by the api etc. It would then be great to have all logs in a centralised location easing up the pain of developers. Yes ! There is a gem for that! 🙂

Log4r is a powerful and flexible ruby gem used for logging in ruby inspired by the popular java library log4j. It supports multiple output destinations per log level, custom log levels etc. Checkout the library .

You might also need to check graylog. Graylog is an open source log management solution that can be used to monitor your logs.  It is built on the top of java, mongodb, elasticsearch . To know more about it click here. Graylog also comes with a web interface. Check out the link to see to setup graylog web interface using nginx as a reverse proxy .

I used gem  ‘log4r-gelf‘ along with log4r to sent my logs to graylog. All log4r and  ‘log4r-gelf‘ in your gem file

gem 'log4r'
gem 'log4r-gelf'

and do

bundle install 

2. Now create a yml file log4r.yml.

log4r_config:
 loggers:
  - name: production
    level: INFO
    trace : false
    outputters:
     - production
     - gelf

  - name: development
    level: DEBUG
    trace: true
    outputters:
      - development
 outputters:
  - type: DateFileOutputter
    name: production
    filename: production.log
    dirname: "log"
    formatter:
     date_pattern: '%H:%M:%S'
     pattern: '%d %l: %m '
     type: PatternFormatter

  - type: GelfOutputter
    name: gelf
    gelf_server: "example.graylog.in"
    gelf_port: "12219"
    level: INFO

3. Next in the application.rb

require 'log4r'
require 'log4r/yamlconfigurator'
require 'log4r/outputter/datefileoutputter'
include Log4r

module DemoApi

 log4r_config = YAML.load_file(File.join(Rails.root, 'config', 'log4r.yml'))
 YamlConfigurator.decode_yaml( log4r_config["log4r_config"] )
 config.logger = Log4r::Logger[Rails.env]

end 
Advertisements

Configuring nginx as reverse proxy for graylog

Introduction

Graylog2 is a powerful tool for log management and analysis tool. One such use case we had in my company is collect all logs of rails application running in 5 different servers in a single location so as to make debugging easy. It is built on the top of ElasticSearch, MongoDB and Java. First you need to set up graylog on your server. These links are likely to help you.

Once it is setup you want to access the web interface. It is running on port 9000.You could actually use a single port to connect with graylog REST api and web interface or two separate ports. This is my nginx configuration.

location / {
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Graylog-Server-URL https://example.in/api;
proxy_pass http://127.0.0.1:9000;
}

location /api/ {
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:12900/;
}

Graylog Configuration


web_listen_uri = http://0.0.0.0:9000/
rest_listen_uri = http://0.0.0.0:12900/
rest_transport_uri = http://127.0.0.1:12900/

Graylog as a very good community at https://community.graylog.org/. You may post your issues there .