Generating SVG Output (from Graphviz) in your Django App

So accomplishing new coding tasks can be a challenge with interruptions and I've had a lot of interruptions this week but I finally got there. And I'm thankful!

I have an app that is storing data, meaning Django models for the uninitiated. What is in there doesn't matter, but it is something that is conducive to plotting with graphviz. So the starting point is a string that is in the .dot format. I have some code that makes queries to the database and I end up with a string.

So there is a utility function that creates this string...

def make_svg_str:
#blah blah blah snip
dot_string += "}"
p = subprocess.Popen('/usr/bin/dot -Tsvg', shell=True,\
stdin=subprocess.PIPE, stdout=subprocess.PIPE)
(stdout,stderr) = p.communicate(dot_string)
return stdout

So I almost got this right the first time except that I forgot the stdout in Popen() which caused the output to go to stdout (and not be assigned to the string) so I saw the .xml in the dev web server logs.

The graphviz string (dot_string) is being piped to the dot executable and then the function is returning the XML SVG as a string, and is obviously assigned to the stdout variable in the tuple.

Now the tricky part within my views.py.

My first mistake was using the Django CSV docs instead of the PDF docs because the latter is what we need. I also didn't remember that HttpResponse is a file-like object so we can can just write to it once we have the SVG text.

def svg(request):
f = Foo.objects.all()
response = HttpResponse(mimetype='image/svg+xml')
response['Content-Disposition'] = 'filename=somefilename.svg'
response.write(make_svg_str(f))
return response

So this will display your image within your browser (which is what I wanted) instead of downloading file if you the use the "attachment" in the Content-Disposition key.

The name of the game is taking shortcuts that get the job done. I'm using the admin interface to provide a good-enough UI to enter the data and now I'm using Graphviz to visualize that data without having to spend a lot of time writing UIs or nasty JavaScript.

Google Protocol Buffers

Google Protocol Buffers was mentioned on the Python LinkedIn Community.

Protocol buffers are a flexible, efficient, automated mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages. You can even update your data structure without breaking deployed programs that are compiled against the "old" format.

Here is the Link to the Python API.

Talking 3.0

I'm embarrassed to say I've hardly touched 2.6 let alone 3.0 but Let's talk about Python 3.0 has some interesting background on the limitations of Python 2.x as well as thoughts on process improvement

This is a startlingly good analogy for the way lots of corporations do things; once a particular process is entrenched (and especially after a couple rounds of employee turnover), there’s nobody left who remembers why the company does things this way. There’s nobody who stops to think about whether this is still a good way to do things, or whether it was even a good idea way back at the beginning. The process continues through nothing more than inertia, and anyone who suggests a change is likely to end up viciously attacked by monkeys.

Of course CentOS5 still only comes with Python 2.4.3, which is what I'm stuck for much of my development unless is compile from scratch.

Back to my Labor Day App

One of the best rules of programming is to leave your code alone if you are banging your head against a problem. When upgrading to 1.0 Well I finally fixed the issue with my admin interface. It helps if you put admin.py in the right directory!

Within the app instead of the project.

DOH!

Func: Python Systems Management

Func: Fedora Unified Network Controller looks pretty interesting.

A lot of programs, systems, and tools need some way to communicate. Func provides a two-way authenticated system for generically doing these sort of things. You can build your own applications on top of it, and easily expand func by adding in additional modules, whether you want these to work through the func command line or by means of some other application. If you just want to use the func command line, that's great. If you want to build apps on the func tools, that's great too. If this sounds generically vague, it's only because it really is that expandable.

With some examples

# func target.example.org call hardware info
# func "*.example.org" call yumcmd update
# func "*" call moduleyouwrote functionname 1234 5678 firetruck acmelabs hike! 

Converting Subnet to CIDR in Python

A nice recipe, here is the meat of an even faster version in the comments.

def calcDottedNetmask(mask):
    bits = 0xffffffff ^ (1 << 32 - mask) - 1
    return inet_ntoa(pack('>I', bits))

Which also led me to the Python netaddr project. Very cool!

Rundown on Django 1.0 Upgrade Problems (and Solutions)

I've been too swamped with my teaching lately to make much progress on a little app I initially wrote in 0.96.2 and have been trying to port to 1.0.

Besides some bonehead typos that wasted a lot of time, here are the summary of issues I've run across:

1. Creation of the new admin.py
2. Modifications of urls.py
3. Conversion between max_length and maxlength in your modles

I still don't have my admin interfaces working even though I've followed the exact steps, so I'm sure it is something else.

The main problems related to the admin interface, but there are a number of resources I've run across including Porting your apps from Django 0.96 to 1.0 and the complete list of backward incompatible changes and a conversion script to create the new admin interface conversion.