Heroku plugin providing insight into the Postgres database for your application

Heroku provides a lovely plugin called heroku-pg-extras which can provide some insight into the inner workings of the postgres databases connected to your application. By default it will hit the database listed in the DATABASE_URL variable for Heroku. However you can specify the database using the database color url; e.g. HEROKU_POSTGRES_PURPLE_URL.

The plugin does the following:

• cache_hit - calculates your cache hit rate (effective databases are at 99% and up)
• index_usage - calculates your index hit rate (effective databases are at 99% and up)
• ps - view active queries with execution time
• locks - display queries with active locks
• blocking - display queries holding locks other queries are waiting to be released
• kill - -f,--force; terminates the connection in addition to cancelling the query
• total_index_size - show the total size of the indexes in MB
• index_size - show the size of the indexes in MB descending by size
• seq_scans - show the count of seq_scans by table descending by order
• long_running_queries - show queries taking longer than 5 minutes 
• bloat - show table and index bloat in your database ordered by most wasteful
• mandlebrot - show the mandelbrot set

These are based on the stats collections process used by postgres which has its own body of literature in collection and analysis starting with their main documentation. It may be worth opening a ticket with Heroku to clear the stats first as they accumulate over time. Changes to the application will not be readily apparent if you still have the stats from the previous app state. Therefore you may want to open a ticket with Heroku to clean the stats (you need to be a superuser to do this yourself and Heroku does not give you superuser access).

Disclaimer: I am a contributer to the project

Errors building Nokogiri on Lion/Mountain Lion

I kept getting an odd error with Nokogiri on my new Macbook:

WARNING: Nokogiri was built against LibXML version 2.9.0, but has dynamically loaded 2.7.8

With help from this post I was able to resolve the issue. It seems that "This happens because the Lion system default libxml2 (loaded at bootstrap) is used, regardless of which libxml2 Nokogiri was built against". No real original work on my part here other than a Google search. However I wanted to spread the effort of Michele Gerarduzzi a litte further and give credit where credit is due.

The relevant post: Get rid of Nokogiri LibXML warning on OSX Lion

Heroku with Unicorn backlog settings and performance

In light of this post from Rap Genius and subsequent blow up on Hacker News I decided to share what we did to play with the request queue on Unicorn & Heroku.

In the app/config/unicorn.rb we changed the backlog line to this:

:backlog => Integer(ENV['UNICORN_BACKLOG'] || 200)

And then we can alter the backlog as necessary via:

heroku config:set UNICORN_BACKLOG=25 -a <app_name>

We found 25 to be a sweet spot for two Unicorn Workers but it may be different with four (which we are experimenting with now). Again this is also relative to the app so your results may (and probably will) be different.

Update:

I neglected to mention that you have to move the port declaration from config.ru and put it in the unicorn.rb too. The full line should be something like this:

listen ENV['PORT'], :backlog => Integer(ENV['UNICORN_BACKLOG'] || 200)

Formatting Source for a blog post

I found this handy blog post for formatting my source code to place here in my last blog post. I encourage you to check it out!

Heroku: Scaling Dynos at night and in the morning

I needed a rake task to auto scale Dynos on Heroku; wind them down at night and wind them up in the morning. I found this helpful post on Stackoverflow. I made a few modifications and would up with this:

namespace :scale_dynos do
  require 'heroku-api'
  desc "scales up dynos"
  task :up do
    dyno_max = [6,0].include?(Time.now.wday) ? ENV['WEEKEND_DYNO_MAX'] : ENV['DYNO_MAX']
    heroku = Heroku::API.new(:api_key => ENV['HEROKU_API_KEY'])
    heroku.post_ps_scale(ENV['APP_NAME'], 'web', dyno_max.to_i)
  end

  desc "scales down dynos"
  task :down do
    dyno_min = ENV['DYNO_MIN']
    heroku = Heroku::API.new(:api_key =>  ENV['HEROKU_API_KEY'])
    heroku.post_ps_scale(ENV['APP_NAME'], 'web', dyno_min.to_i)
  end
end

It adjusts for the weekend to scale to a different level if you set the ENV varaible. Additionally here is how I set the Heroku config vars. The add command does them all at once but the remove goes through one by one, removes the VAR and restarts the app.

heroku config:add DYNO_MIN=1 DYNO_MAX=25 WEEKEND_DYNO_MAX=1 APP_NAME=<your app name> HEROKU_API_KEY=<your api key> -a <your app name>
heroku config:remove DYNO_MIN DYNO_MAX WEEKEND_DYNO_MAX APP_NAME HEROKU_API_KEY -a <your app name>

You will need the heroku-api gem. I also set the Gemfile entry to this:

gem 'heroku-api', :require => false

Since the rake task is called twice a day there was no need to load the gem into memory for the rest of the time.

Current Rails Project

Currently I am working with a company that is upgrading their Rails 1.2.3 app to Rails 3.0.13. We decided not to tackle the asset pipeline in this go around.

In production their app resides on a Windows server hosted inside the clients walls and operates off of MS-SQL. I have it up and running on my Windows box at home while I work on an Ubuntu laptop. I am trying to make it as agnostic as possible so that should a client want the Rails app on a Linux server there is little to change. Who knows how successful this will be.

Customary Introduction Post

The title of my blog comes from the fact that I have a cat who will sit on my lap staring at the screen while I try to write code. I am taking the stance that he wants to learn what I am doing rather than he is really saying "Srsly? You should read a best practices book."

I am a mid-range Ruby/Rails developer at this stage. Proficient in some areas, woefully lacking in others. Also, because I have only had one 'longish' term assignment I have had to hop around from Windows to Linux to Mac and from Rails 1.2.3 to 2.3.n to 3.n.

I find myself learning a lot of things... sometimes repeatedly... so I decided to start blogging about it as a way to take notes and possibly help someone else-- hopefully more than just providing a "don't do this!" example.