APM for CherryPy via New Relic

Being a Python guy I’m doing most of my web development these days using CherryPy. If you are not familiar it’s a  Python web framework that to me strikes just the right balance of simplicity and power.

Behind the scenes at Key Ring we also have a large Rails footprint. For the past year we’ve been using New Relic to monitor application performance up and down the stack. When it comes to monitoring and troubleshooting a web app I can’t recommend these guys enough. They’re constantly making improvements, adding features, and the product has become indispensable to our team.

So I just finished building some data services in CherryPy and was psyched to hook up the New Relic agent to this app. The docs on the New Relic site are pretty thorough but I wasn’t able to piece together how to hook the agent into the CherryPy app. Also, could’t find anything on the web. Through trial and error I was able to get it working…. it’s actually extremely easy if you just follow these steps.


1. Download the latest Python agent. Unpack.

2. Run the setup

3. Generate a New Relic agent config file using your API key.

4. Change your app name in the New Relic config. There are some other logging bells and whistles which are all self-explanatory

5.  Add these 2 lines to your sites startup script. Make sure you do it early in the script before any database calls, etc…


And that’s it. In the next minute or so you’ll begin to see your stats streaming into your New Relic dashboard.

urllib2 With Multiple Network Interfaces

Normally if I have an issue which is answered in the first 2-3 results of a Google search I won’t create a post. On the other hand when I spend 2-3 hours trying to solve something which should be simple I like to take the opportunity to describe the issue & resolution in hopes that someone will find it quickly in the future.

So the task here was to find a way to specify the IP address, aka socket, aka network interface when making an http request using Python’s urllib2. Why would you want to do this you ask? Well for many web API’s the request rate is limited by whitelisting the IP address – such is the case with Twitter. In the event that you want to be able to use the same machine (with multiple network interfaces) to run jobs in parallel you need to be able to specify where the requests should be routed.

The problem is Python’s urllib2 is based on the httplib library which doesn’t let you specify which address to bind to. This person tried to get around the problem in 2005 without any luck, another guy created a patch for httplib in 2008 which hasn’t been accepted, and finally someone else created a subclass for httplib which unfortunately I couldn’t get hooked up to the urllib2 class.

The best solution I found was this “monkey patch” from Alex Martelli over on Stack Overflow.  In his example he attacks the problem using the socket library instead of the httplib. By his own admission stuff like this is not ideal, but the solution is actually very simple and elegant. I like it.

I wrapped the snippet up into a function which can be called in a Python script anytime before you invoke a urllib2 request.

Hope this can be of help to someone in the future who’s searching for the same thing I was.

All Programming is Programming

Over on Coding Horror Jeff Atwood wrote a post which claimed that “All Programming is Web Programming“. Jeff made some good points about how the web provides programmers with the ability to reach an audience of a previously unimaginable size. Definitely agree. Also mentioned was that for better or for worse JavaScript is becoming the most important language in the world of software development. I agree with this as well, though I would add that the significance of JavaScript is in user facing applications only at this point.

It was unfortunate that the post was written in such a polarizing manner and that the comment thread quickly eroded into a shouting match because there is another important point to be made here.

Something else I want to put out there to developers is that the evolution of programming should be focused less on desktop vs web vs embedded or choice of language/platform, and more on how lowering the barriers to entry for new programmers is a positive and not a negative.

To someone writing device drivers or kernel patches the idea of writing a JavaScript function to manipulate the DOM may seem “uninteresting”, but the fact is that more and more people are getting started with programming this way. All they need is a text editor and a web browser and they are on their way. This is a very good thing.

Programming is about automation and automation is about improving efficiency. The more people we can somehow involve in this process the better because in the end the web provides not only the largest number of potential users, but also the largest number of potential programmers. The exciting thing is we are just getting started.

© 2009 - 2019 Ross Bates