Farm Development

Thoughts on Django

back to all thoughts

Web Frameworks Do Not Make DBAs Happy

posted in Django, JavaScript, Python, Ruby by kumar on Tuesday Sep 16th, 2008 at 10:18a.m.

A colleague of mine, Shaun Thomas, is one of a few database administrators who manage all our company's databases by monitoring, optimizing, partitioning, building star schemas, etc. The DBAs also maintain standard operating procedures for how to name a column or how to refer to an external identifier. Most importantly, they conduct reviews of your horrid schema changes before you break stuff.

Most web frameworks (Django, Rails, etc) out there abstract away a lot of low level database details since they focus on making life easier for web developers. This is great but it's important to have a way to easily tweak the low level stuff when you need to. In fact, most frameworks kinda leave DBAs in the dust. It looks like Shaun reached his breaking point on this a few weeks ago and the result was a hilarious rant. He does have a good point. The only database abstraction layer I've used that truly keeps the DBA in mind is SQLAlchemy. It adds more complexity to the tool but not in a way that makes your life difficult.

Leapfrog Online is looking for some Django developers (Chicago area)

posted in Chicago, Django, JavaScript, Python, Testing by kumar on Thursday Dec 13th, 2007 at 4:45p.m.

Leapfrog Online is looking to hire several Python developers to work on a Django site. If you know Python but not Django, this is an excellent opportunity to learn. If you know Django but want to learn how to use Python in other contexts, you'll get to do that, too. You'll be working on a high traffic website that hooks into several web services to help customers find Broadband Internet connectivity based on geo location (just US and Canada at the moment). Surrounding that basic function are all kinds of front-end and back-end features, services and systems.

The Software Engineer position is outlined in detail here.

You can send your resume to kumar.mcmillan@gmail.com or send it through the site above. These are full-time positions but if you'd rather work with us as a contractor that may be possible.

What Do We Do?

Leapfrog Online does performance-based customer acquisition, which translates to "we don't make money unless our clients make money." Because of this our software has to work well and we need to collect lots of structured, sensical data so our analysts can build the right marketing strategies. In a more abstract sense, the interesting challenges we face are building high-availability websites, fault-tolerant web services, pushing and pulling at hundreds of gigs of data, and accounting for tight security all along the way. As for the atmosphere, we're still a small company but we're not a struggling startup.

We Care About Open Source

We use open source tools that are right for the job. Currently we use Python or Ruby for websites / web services (Django, Pylons, Paste, Ruby on Rails), Python for backend tools, PostgreSQL for databases, and Trac for our projects. We use rich web interface libraries like Ext JS and we even wrote a distributed content system in Erlang because it was a good fit.

We have contributed patches to most of the projects listed above and maintain our own projects like nose, a few nose plugins, fixture, wsgi_intercept, and sv. We give talks at conferences like Pycon (see #24, #85 and #127). Also, Jeff Cohen (one of our senior developers) runs a popular blog called Softies on Rails and teaches and writes books about Rails.

Scrum: You'll Like It

We started with Extreme Programming a few years ago and have moved towards Scrum and other Agile methods as our approach to software development. We are constantly refining our process, keeping what works, discarding what doesn't. The company is on board with Scrum all the way up to the principles and we are always working to improve how Scrum is integrated holistically (a training program is in the works).

We think you'll like Agile for development. We have several teams of no more than three developers who work in two week "sprints." The sprints are planned out by product owners, developers, and project managers with user stories estimated in "story points" so that the business gets what it needs in order of priority. A sprint is exactly what it sounds like -- you just work! At the end of each sprint the work is released and you attend a retrospective meeting to see what was good, bad, and ugly, and how much work you did. Nothing is perfect so, of course, there are emergencies and derailments here and there but for the most part Scrum keeps things moving at a productive pace. As a developer, I find this discipline empowering and highly motivational.

You Must Test It

We are nutty about automated testing (in case you didn't notice). All code must have automated regression tests so if you're not familiar with this way of writing software, you will learn! We have a fairly involved continuous integration process running in buildbot (though probably moving to Bamboo soon) that performs several builds of each app, one with stable 3rd party libs, the others with trunk versions of 3rd party libs. As well as getting immediate feedback when a bad change is checked in, this also helps us pinpoint bugs in our dependencies before they are released. Our QA department is also different than most in that it consists of developers who are writing functional and/or integration tests in code and adding these to the build process. They are essentially software engineers like the rest of us.

Your Time Is Valuable

No one has a sleeping bad under their desk here; we work until 5 or 6 (weekdays only) to achieve a "sustainable pace." Most of us have been through the "death march" routine at other companies so we know it doesn't work long term. Scrum helps us maintain this ethic.

No Pigeonholes

While we are currently looking for Python/Django programmers, we are always interested in meeting people who think in Ruby/Rails, PHP 5 and other open source web technologies, too. We're especially interested if you're feeling ecumenical and want to learn about and work with, say, both Python and Ruby. You might only work in one language most of the time, but we think it is important for developers to stretch themselves and understand what tools are best for the job.

Blogging, Blogosphere, or something

posted in Django, Whatever by kumar on Tuesday Nov 28th, 2006 at 9:25a.m.

So ... I have a blog now. Dunno about Bonde but I always kinda told myself I'd never have a blog because I don't read them much and think the whole phenomenon is a little weird and self-indulgent.

On the flipside, I find myself googling a lot for things like "make it work damnit!" (more specifically of course) and usually that pops up a blog where someone kindly posted his/her instructions for making it work. So hopefully this will be a useful blog for technical nerd stuff at best.

I also find myself perusing Planet Python via RSS on my lunch breaks these days. I always seem to find some useful python tidbit in there. Then again, being a nerd is hard work so I'll probably post my handful of useless, self-indulgent rants and add to the clutter we call the Internet. Oh, what beautiful clutter it is.

Speaking of blogs, can someone post a comment if they know of a good blogging app for python? I say python because I would be interested to contribute if it was well designed, open source, etc. I ended up installing Wordpress for my sister, even though I loathe PHP for all the usual reasons. Wordpress pretty much rocks, but as a wounded PHP veteran, I have no desire to touch the language ever again.

Typo seemed promising at first but looks a little dead. I tried installing Mephisto and, although their install instructions seem more reasonable now, when I tried a month ago it required subversion and rails edge. Both of which caused the tests to break. And pass. Then break again. You get the picture. I couldn't even get the thing to run without a 500 error and gave up after a couple of weeks. Woodlog looked promising too since I like Django but that too seems a little dead. Sigh.

So I built this thing with Django in about 4 sittings and it's super basic but seems to work!

Recent Projects

  • fixture

    Python module for loading and referencing test data.

  • pypione

    A mirror of PyPi running on Google App Engine

  • Wikir

    converts reStructuredText documents to various Wiki formats.