May 21, 2012

Django Formatter Mixin Class

guitar Django Formatter Mixin ClassI’ve always disliked having to write __repr__ methods for my classes. It isn’t hard, it is just tedious. In Django, it is especially important to make them, since the admin pages use that method when listing objects from the data model.

However, since Python allows classes to have multiple inheritance, I have long since made a helper “Mixin” class to remove most of the pain from writing my __repr__ methods. If you haven’t heard the term, a “Mixin” is a class which adds functionality to classes which inherit from it. The philosophy is that it shouldn’t change base functionality, thus avoiding many of the problems with multiple inheritance.

Philosophy aside, mixins can be extremely useful, and I love this one. [Read more...]

Stats on a larger Django project

diamond1 Stats on a larger Django projectA Django website that took (a lot) more than 20 minutes. Interesting writeup of a meta-forum app, including code samples and time breakdown.

Found via Django Forums.

[tags]django[/tags]

Zyons Forum for Django

handshake 2 Zyons Forum for DjangoThis summer, I’ve been converting my biggest site, Invisible Castle, from Cheetah to straight Django. At the moment, Invisible Castle is run on straight CGI, using Cheetah to render the pages, so this is a rather big conversion.

I’m not just changing the template, that part is surprisingly simple, but the whole page flow logic. From stateless to a full web app. It is interesting, going quickly and the site will be much stronger when I’m done. It already gets 20-50K hits a month, and with my changes it will be able to take a lot more hits and handle them more quickly.

What’s the point?

Ah, but what’s the point? Why bother? Well, partially I just wanted to see how easy it would be, and because I find Cheetah templates annoying to maintain. But more importantly, after conversion, I’ll be able to use Zyons forums, and integrate them into the site. Zyons is a nice, full featured forum app, written using Django. So, I’ll be able to integrate my current app into a forum. How great is that?

Just try that with some hacky PHP forum. I dare you. I am absolutely certain it will work, and work smoothly. I can hardly wait to integrate and bring this dream to life.

[tags]django,cheetah,forum,zyons[/tags]

Google Checkout – Initial Impressions from a developer

shopping cart Google Checkout   Initial Impressions from a developerThis weekend I wrote my first Google Checkout integration module in my favorite language, Python. I didn’t expect much, largely because I am used to the more simple and limited offering from PayPal. I thought the initial offering from Google would be simple, limited, and straightforward.

Nope.

It goes about as deep as you’d want to go, but almost all the “deep features” are optional. At the easiest, you can simply post a “buy now” button, much like you can do with PayPal. But at the most advanced, you can have complete order flow tracking with multiple order states, split and recurring payments, with tax-tables, zip-code based shipping tables, and a complete coupon system.

Two Levels

At the basic level, you can simply have Google handle taking payment for your shopping cart. You can have line items for each cart item, and a shipping charge. Once the customer buys, you get an email, and you manage the charge request and the shipment status on Google’s site.

That’s what I am doing on eBookTribe for now. Here’s the Django template for an order from that site:

<?xml version="1.0" encoding="UTF-8"?>
<checkout-shopping-cart xmlns="http://checkout.google.com/schema/2">
<shopping-cart>
<items>
{% for orderitem in cart.orderitems.all %}<item>
<item-name>{{ orderitem.title }}</item-name>
<item-description>{{ orderitem.description }}</item-description>
<unit-price currency="USD">{{ orderitem.price }}</unit-price>
<quantity>{{ orderitem.quantity }}</quantity>
</item>{% endfor %}
</items>
</shopping-cart>
<checkout-flow-support>
<merchant-checkout-flow-support>
{% if edit_url %}<edit-cart-url>{{ edit_url }}</edit-cart-url>{% endif %}
{% if continue_url %}<continue-shopping-url>{{ continue_url }}</continue-shopping-url>{% endif %}
</merchant-checkout-flow-support>
</checkout-flow-support>
</checkout-shopping-cart>

I think that is so clean, almost elegant. The bottom part is support for “continue shopping” and “edit cart” links from Google’s site.

“Level 2″ integration with Google occurs when you give Google a callback url, which must be SSL enabled. Using that callback, Google and your website can communicate regarding status changes, and your site can request card capture, refunds, set shipping status, etc. I’m looking forward to coding this part, but for now, I’m going to launch with just the level 1 integration.

Security

I’ve never been fond of how PayPal forms are so easily modified by customers. All the information is right there in the form, to be manipulated at will. That means that the merchant, me, has to manually verify totals and match it to the order before shipping. Google has a much better solution.

With Google Checkout, you get a merchant ID and key. You convert the shopping cart XML into a base64 representation and post that as one form element, and you post the signed hash for it in another form element. Since your key is private, and visible only on the server side, your customers cannot modify the form prior to submitting it.

An encoded google shopping cart form looks something like this:

<form name="google_form" action="https://sandbox.google.com/cws/v2/Merchant/YOURMERCHANTIDHERE/checkout" method="post">
<input type="hidden" name="cart"
value="PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz4KPGNoZWNrb3V0LXNob3BwaW5n
LWNhcnQgeG1sbnM9Imh0dHA6Ly9jaGVja291dC5nb29nbGUuY29tL3NjaGVtYS8yIj4KICAgIDxz
aG9wcGluZy1jYXJ0PgogICAgICAgIDxpdGVtcz4KICAgICAgICAgICAgPGl0ZW0+CiAgICAgICAg
ICAgICAgICA8aXRlbS1uYW1lPlF1aWNrIEJlYXV0eTwvaXRlbS1uYW1lPgogICAgICAgICAgICAg
ICAgPGl0ZW0tZGVzY3JpcHRpb24+VGl0bGU6IHRlc3Qgb3JkZXIgMTwvaXRlbS1kZXNjcmlwdGlv
bj4KICAgICAgICAgICAgICAgIDx1bml0LXByaWNlIGN1cnJlbmN5PSJVU0QiPjE0Ny4wMDwvdW5p
dC1wcmljZT4KICAgICAgICAgICAgICAgIDxxdWFudGl0eT4xPC9xdWFudGl0eT4KICAgICAgICAg
ICAgPC9pdGVtPgogICAgICAgIDwvaXRlbXM+CiAgICA8L3Nob3BwaW5nLWNhcnQ+CiAgICA8Y2hl
Y2tvdXQtZmxvdy1zdXBwb3J0PgogICAgICAgIDxtZXJjaGFudC1jaGVja291dC1mbG93LXN1cHBv
cnQ+CiAgICAgICAgICAgIAogICAgICAgICAgICAKICAgICAgICA8L21lcmNoYW50LWNoZWNrb3V0
LWZsb3ctc3VwcG9ydD4KICAgIDwvY2hlY2tvdXQtZmxvdy1zdXBwb3J0Pgo8L2NoZWNrb3V0LXNo
b3BwaW5nLWNhcnQ+Cg=="/>
<input type="hidden" name="signature" value="XvnIeUBLq2loVK78RU/+PaqRLXo="/>

<input type="image" name="Google Checkout"
src="http://checkout.google.com/buttons/checkout.gif?merchant_id=YOURMERCHANTIDHERE&w=168&h=44&style=white&variant=text&loc=en_US"
alt="Fast checkout through Google"/>
</form>

Conclusion

Google Checkout is quite a bit more difficult to get working initially than PayPal, but much more rewarding. After working with it for only a weekend, I think that PayPal should be quaking in its books. This is one excellent solution, and one that I really look forward to continuing to explore.

[tags]python, google checkout, gcheckout, gbuy, paypal[/tags]

Speedy Django Development

guitar Speedy Django DevelopmentAs I’ve mentioned before, I am building my own web store using Django. The coding is going extremely quickly, thanks to the excellence of the framework.

I find myself marvelling at how quickly and simply various elements come together, especially when I contrast this to my previous experience doing the same thing with Java for an employer.

For example, the Login/User framework. With Django, I simply include the auth middleware, and create a couple templates to “skin” the login, profile, password change, and password reset pages. In return, I get a full-featured, working login system with sessions, cookies, and all that tedious plumbing done for me. Total time from decision to use the built-in framework to implementation? Two hours, including learning the system. Unbelievable.

Contrast that with my use of Acegi/Spring to handle authentication in a Java app. I was shockingly fast implementing this, taking just two full working days. The documentation directly states that you should plan on at the very least one full week. The system was huge, slow, and baroque. Modifying the system was clumsy and frequently required I relearn things from the reams of documentation. Yes, overall it offered more functionality. No, there was no stripped-path to just get the most needed set of features for a website. The craziest thing is that the config files for Acegi
alone were larger than all the code and templates I wrote for django login
.

Another example is my need to generate thumbnails for some graphics. I want to generate them “on the fly”, when needed. Total lines of code to do this in Python with PIL, integrated with Django? 7. Total time? 15 minutes, including all the logic needed to make sure that I only create the thumbnail once.

Contrast this with the nightmare that is the Java image API. I seriously spent days making a caching thumbnailing system in Java for an Auction site I worked on. Even after that, it was much slower than my Python solution.

Now that I am actually developing a serious app for Django, I find myself feeling the love more and more. Thanks Django team!
[tags]django,python,spring,acegi,pil[/tags]

The best python url_join routine ever

thumbs up The best python url join routine everI don’t know how many times I’ve had to write a function to safely join url path components. It seems so simple, “Just join the strings together with forward slashes.”

But, then I think, “Oh, and don’t strip the leading forward slash, if any”. And “oh, also don’t strip the trailing one”. Also, “make sure not to double up any slashes.”

It is an annoying routine to write, and to be honest, I’ve never really liked any of my implementations. They always failed on some edge case. But tonight, I conquered.

So, without further delay, here is my perfect url_join routine. It hasn’t failed any reasonable edge case I’ve thrown at it yet! The backslash replacement parts are to make it windows compatible, but of course I haven’t tested those.


def url_join(*args):
    """Join any arbitrary strings into a forward-slash delimited list.
    Do not strip leading / from first element, nor trailing / from last element."""
    if len(args) == 0:
        return ""

    if len(args) == 1:
        return str(args[0])

    else:
        args = [str(arg).replace("\\", "/") for arg in args]

        work = [args[0]]
        for arg in args[1:]:
            if arg.startswith("/"):
                work.append(arg[1:])
            else:
                work.append(arg)

        joined = reduce(os.path.join, work)

    return joined.replace("\\", "/")

[tags]python[/tags]

SQLAlchemy = Another great SQL Frameword

surgical tools SQLAlchemy = Another great SQL FramewordJust a few days ago, I was talking about how much I enjoy PySqlite, and now I’ve found another great tool for my SQL war chest. SQLAlchemy

I am already using it for another part of a big multi-database reporting project. The fun part for me is that it is more modular and less demanding of structure than SQLObject, so I can actually use its superior raw SQL connection and marshalling capabilities for manual report queries, while continuing to utilize my SQLObject enabled object model.

Very sweet, and very quick to get going on my Mac development and Linux deployment environments. I’m currently collecting data from three different databases (SQLite, PostgreSQL, and MySql) on five different servers. This is a killer app for me.

I’ll be exploring its object-relational capabilities soon, but I am already very fond of the framework, and how lightweight it feels.
[tags]python,sql,sqlobject,sqlalchemy[/tags]

Discovered the killer combo of Pysqlite and SQLObject

light bulb yellow Discovered the killer combo of Pysqlite and SQLObjectAt work, I’m doing a bunch of data acquisition on various machines outside the firewall. I think I’ve hit on an absolutely great combination of Python technologies to do this in a really manageable and scalable manner.

The design constraints:

  • No long-running processes. These are heavily loaded boxes. The data acquisition will run via scheduled cron jobs.
  • No new ports open. I can’t run Mysql on these boxes.
  • No drilling back from these servers into the corporate network. So I can’t run mysql inside and connect from the outside.

The plan

  • Each remote box will run a small python app on a scheduled basis, writing its data to a single file.
  • A "gatherer" system inside the firewall will retrieve the files from all remote machines on an hourly basis.
  • On acquisition, the remote files will be removed.
  • The gatherer process will then merge the remote data into a master Mysql database for reporting.

The technologies

  • PySqlite: will provide access to an on-the-fly generated database file on remote machines.
  • SQLObject: can talk to both PySqlite and Mysql. I simply define the object model using its scheme, and it will happily generate the tables for me in both Sqlite and Mysql. Writing the merge process was the trickiest bit, but it only took a few hours to write and test.

Conclusion

This was an extremely fast development path. I was all the way from concept to deployment in just three days, and I didn’t even know anything about SQLObject or Sqlite at the start.

Contrast that productivity to Java, where a similar project would take at the minimum a couple weeks, and you start to see why I think that Python has become the secret weapon in my developer toolbox.

[tags]python,sqlite,pysqlite,sqlobject[/tags]

Magic Removal Success!

classical guitar Magic Removal Success!I’m pleased to report that the Django magic removal branch is now much easier to use. It feels stable, the docs are getting complete, and I see now that the branch has now been officially merged to the head.

My magic-removed code works great, and is passing all its tests with flying colors. Total time taken for 3 apps was approximately 4 hours. The only real pain was in updating the database. I feel certain I could have made it work, but I took a shortcut and simply reset the whole thing.

manage.py syncdb is sweet!

[tags]Django,magic-removal,python[/tags]

Django Magic Removal, this time for real

guitar Django Magic Removal, this time for realDespite my earlier initial successes using the Django Magic-Removal Branch, I had to roll back. The branch code was stuck in an unusable state for a couple weeks when I was really wanting to crank out some functionality on my storefront.

I’ve long since learned that when you have a strong urge to code, you should do it right then, not put it off for when it would be more “convenient”. It won’t be, and you’ll lose precious hours of probable “flow programming.” So, I rolled back and worked off the .91 codebase.

This time there’s no looking back

Because the branch is supposed to merge to the head of the tree today, I have no choice remaining. Ah well. I spent three hours last night merging my three Django apps. I’m not quite done, but all that’s left is the database API changes, and my DB calls are all well isolated.

Not only will I be “on the mainline” of the Django code, but when the code is all merged I’ll be able to use the far more intuitive new API. I’m honestly quite happy to see them break with their old API before they release the 1.0 version. That way they don’t have to support cruft, which slows down development. Possibly more importantly, they obviously have applied “lessons learned” when refactoring.

This isn’t some little change. It is “shoring up the foundation” before major new functionality is built on it. I can’t wait to see what happens next.

Up next, more ECommerce

For my part, I’m getting more involved with the Django yet-to-be-named ecommerce shopping module. It is amazing what just a few Django programmers can throw together in a few days, really a testament to the productivity of the framework.

[tags]django,magic-removal,python[/tags]