By josh

We use Google Apps for all of our emailing… and for the most part it works great. However, dealing with their outbound smtp relay can be a real pain. The biggest problem is that it only allows you to send an email if the From header matches the account that is used for authentication. (Note: I’m not suggesting that this is a bad practice, or something wrong with Google… it makes total sense.)

Because of this, I needed to set up different smtp account settings for one of our ActionMailer subclasses than those used by the rest of our site. Unfortunately, because of the way ActionMailer is designed… this isn’t exactly an easy task!

I found this article, which gave me some clues about how to do this. Here’s the solution I came up with:

class MailerWithCustomSmtp < ActionMailer::Base
    SMTP_SETTINGS = {
      :address => "smtp.gmail.com",
      :port => 587,
      :authentication => :plain,
      :user_name => "custom_account@transfs.com",
      :password => 'password',
   }
 
   def awesome_email(bidder, options={})
      with_custom_smtp_settings do
         subject       'Awesome Email D00D!'
         recipients    'someone@test.com'
         from          'custom_reply_to@transfs.com'
         body          'Hope this works...'
      end
   end
 
  # Override the deliver! method so that we can reset our custom smtp server settings
  def deliver!(mail = @mail)
    out = super
    reset_smtp_settings if @_temp_smtp_settings
    out
  end
 
  private
 
  def with_custom_smtp_settings(&block)
    @_temp_smtp_settings = @@smtp_settings
    @@smtp_settings = SMTP_SETTINGS
    yield
  end
 
  def reset_smtp_settings
    @@smtp_settings = @_temp_smtp_settings
    @_temp_smtp_settings = nil
  end
 
end

The idea here is simple… before sending the mail, set the @@smtp_settings to whatever it needs to be. Then, after delivering the email, set it back to whatever it was. Hope this helps someone else!

By josh

In our recent switch to EngineYard Cloud… one of the most annoying problems I ran into was getting our Wordpress blog configured properly so that it would play nicely with our Rails app. In our previous hosting environment, we ran with an Apache setup… and we used a simple virtualhost config that looked something like this:

  Alias /blog /var/www/apps/tfs_blog
  <Directory /var/www/apps/tfs_blog>
  PassengerEnabled off
  allow from all
  AllowOverride All
  </Directory>

With the move to EngineYard, we have made the switch to Nginx… because, after all, that’s what all the cool kids are using these days. I admit that one reason I was looking forward to Nginx was to rid myself of the awful apache config file syntax. However, I quickly learned that Nginx (or at least the version that is installed by default on the EY Cloud images) requires its own voodoo tricks in order to do seemingly simple things.

In this case, all we want to do is host our blog, alongside our rails app, at /blog. (and of course, this blog is hosted at /devblog) Unfortunately, getting this working properly required endless hour of scouring google for nginx config snippets… until I finally landed on the following, working setup:

location /blog/ {
 alias /data/tfs_blog/;
 index index.php index.html index.htm;
 if (-e $request_filename) {
 break;
 }
 rewrite ^/blog/(.+)$ /blog/index.php?q=$1;
}
location = /blog {
 fastcgi_pass 127.0.0.1:1027;
 fastcgi_param SCRIPT_FILENAME /data/tfs_blog/index.php;
 include /etc/nginx/common/fcgi.conf;
}
location ~ /blog/?.*\.php$ {
 if ($fastcgi_script_name ~ /blog/?(.*)$) {
 set $valid_fastcgi_script_name /$1;
 }
 fastcgi_pass 127.0.0.1:1027;
 fastcgi_index index.php;
 fastcgi_param SCRIPT_FILENAME /data/tfs_blog$valid_fastcgi_script_name;
 include /etc/nginx/common/fcgi.conf;
}

The worst thing about this, aside from the complexity required to get such a simple thing working, is that the error condition you get when the nginx fastcgi setup is misconfigured is incredibly obscure: “No input file specified.” This apparently means that fastcgi is not receiving the SCRIPT_FILENAME parameter properly… but the actual cause of this can be anything from misconfigured permissions to location paths that aren’t grabbing the proper filename from the request uri. As you can see, the final result requires manually capturing the php filename from the uri path using a regex, and then setting the SCRIPT_FILENAME env variable accordingly.

It’s also worth mentioning that this snippet:

 if (-e $request_filename) {
 break;
 }
 rewrite ^/blog/(.+)$ /blog/index.php?q=$1;

… from the first block is required to make wordpress’s pretty urls map correctly to the cgi handler.

That sure was a lot of work! Hopefully this post will save someone else all the pain and headache… or maybe someone will stumble across this and show me how it can all be done in a couple of simple config lines.

By josh

We recently underwent a pretty big change here at TransFS.com… all entirely “under the hood”. We decided to migrate from our old hosting setup at Joyent, to our new home at EngineYard Cloud. We’re now running on Amazon EC2 instances, all transparently and easily managed by the excellent admin tools provided by Engine Yard.

Why the switch?

I had several reasons for deciding that Joyent was no longer working for us. First, the Solaris architecture that Joyent uses provided no obvious benefits, and a surprising number of headaches. Solaris has most of the same tools as linux… but many of them take different command-line arguments or are annoyingly lacking in features. For example: “grep -R” for recursive search through files, doesn’t exist on Solaris. Why? I have no idea. But these types of things can become a real pain over time, and while they seem small, they really do matter.

Secondly, Joyent provides no automatic backup solution for your data. I was very surprised when we first switched to Joyent that I had to roll my own backup solution with mysqldump, rsync, and a script that uploaded to Amazon S3. I never felt good about this, because backups are so important, and because it isn’t an area that I’m very confident in… and I was never able to properly test that my backups would save us in the case of massive data loss. When selecting a hosting provider, I really want someone to hand me a pre-built backup solution that I can trust uses industry best-practices and has been carefully tested. EngineYard provided me with much more confidence on this front, and automated daily or hourly backups are dead-simple to set up (no hacking of my own scripts!)

Finally, we have been having some real problems with the memory footprint of our app. It has ballooned to over 150MB per rails instance, sometimes as high as 250MB. Between our Passenger children, background workers, mysql, etc… we kept bumping up against the RAM limit on our Joyent slice. This was obviously our fault, but the process for dealing with it with Joyent made it a real pain to solve. Moving to a two-slice setup, where our mysql db lived on a separate server was an obvious solution… but provisioning and setting up a whole new server was something that I didn’t have time for, and all Joyent really offered was to clone my existing setup. I would have had to deal with everything else myself, and suffer through a lot of downtime while I figured it out. Even switching to a Joyent slice with more RAM was not an instantaneous process. I would have needed to create a support ticket, and then wait for someone to get back to me and clone us over to a new “accelerator” slice. I’m sure this would have worked… but it isn’t nearly as convenient as it should be, in the age of push-button admin interfaces like those provided by EngineYard Cloud. In fact, with EY Cloud, I was able to switch to running our db on a separate server with a single checkbox and re-deploy. Very cool. We’re now running safely under the RAM threshold of the smallest EC2 instance, and I can very easily scale up to more servers or larger server by selecting a different server type in an EngineYard dropdown.

Custom Chef Recipes

It actually took us a lot longer to switch from Joyent than I had hoped. I began the process of investigating EY Cloud a few months ago… and had delayed the process for a while because I had some real concerns about some parts of our setup. Gems, packages, etc are all super-simple to set up in EY Cloud… but certain things are not as easy as a “checkbox”. For instance, we run Starling & Workling for background processing at TransFS.com. Getting these set up in EY Cloud was non-trivial, because it required writing a custom chef recipe. Chef is a really amazing server configuration tool that uses a ruby DSL to define changes to your server config. EngineYard has some decent chef documentation, but it still took some time to learn how Chef works before I was comfortable enough with it to get our servers set up properly.

However, once you learn to be a master Chef… it gives you amazing power to set up your server exactly the way you want, and in a totally repeatable way. In fact, when I need to add a new application server to our cluster… it should be as simple as flipping a switch, and all of my custom configuration settings will be applied automatically to bring that new server into the proper state for TransFS.com. Very cool. Learning Chef is a necessity to getting up and running with EngineYard Cloud… but it is worth the effort.

There are a few other quirks to EY Cloud that took some work to figure out, such as migrating capistrano hooks… but overall I was very pleased with the process. I had us up-and-running on the new servers in about 30 hours, start to finish, and we only had about 30 minutes of real downtime while I rsync’d our data over.

Anyway, thanks EngineYard for a great experience so far. And thanks as well to Joyent for hosting us through our early days… we’ll miss you, but we’ve found a new home that is much better suited to our needs!

Coming up in the next post: some lessons learned about Nginx, Rails, and Wordpress…

By josh

I recently discovered a simple, but really great, ruby idiom… that I’m not sure how I’ve lived without for all this time. You can query strings with a regexp using the [] array operator:

"Grab the number 555 from this string"[/\d+/]
"Grab the number 555 from this string"[/number (\d+)/, 1]

This will return nil, if the regexp is not matched, and the matched text otherwise. You can even supply an index number as a second argument, to return a particular matched substring.

So cool. It is this kindof amazing syntactic sugar that makes Ruby a joy to work with.

By josh

We’ve recently started using HighRise as our key CRM tool, and so far it looks like it will work very well for our needs. However, hooking our existing processes and our customer data into the web service takes a little bit of time & effort.

First of all, while HighRise has an excellent API… the “official” ruby code on the developers’ site is pretty minimal. Fortunately, some community efforts on GitHub have cleaned up this code and improved it dramatically. I’ve picked up this work and updated it slightly, and hopefully the lessons I learn can be rolled back into my GitHub repository.

One particularly tricky problem, even with a nice ActiveResource library like the one hosted on GitHub, is creating/updating contacts with associated “contact-data”. This includes adding email addresses, phone numbers, etc to contacts. I looked around on the developers forum, and there seem to be a number of people who have had questions about how to accomplish this… but no solid answers. After fiddling around with it for a while, I came up with the following solution:

Highrise::Person.create 'first-name'=>'Test', 
                                    'last-name'=>'API', 
                                    'contact_data'=>{ 
                                         'email_addresses'=>[ {
                                              'address'=>'test@test.com', 
                                              'location'=>'Work' } ]
                                    }

This code creates the following POST xml:

<?xml version="1.0" encoding="UTF-8"?>
<person>
  <contact-data>
    <email-addresses type="array">
      <email-address>
        <address>test@test.com</address>
        <location>Work</location>
      </email-address>
    </email-addresses>
  </contact-data>
  <last-name>API</last-name>
  <first-name>Test</first-name>
</person>

… And it works great! So hopefully this will help anyone out there who is struggling to get the HighRise API to do what they want via Ruby.