Post tagged: python

The "verbose" Python

PYTHONPATH, PYTHONVERBOSE and other infuriations

The symptoms

“Suddenly” (as is traditional to commence these sort of stories), I started having problems with some of the Python installations on my local machine. As is the way of things, I had three distinct sources of Python:

  • The native / builtin Python that came with my Mac
  • The Python ...

simpleredcapbuilder

A builder for more compact REDCap schema

Background

REDCap is a fabulously useful tool for creating web databases, including a useful scheme for defining the database schema ("data dictionaries") in plain CSV files. However, difficulties arise when designing complex schema. It is cumbersome to code multiuple occurences of essentially the same field (e.g. sample_1 ... sample_2 ...). If ...

Porting this site to Pelican

On making a simpler website in a slightly complicated way.

Background

Way, way back, this site ran on various permutations of PHP-based frameworks. Getting sick of ugly URLs and having to vigilantly update the software so as to avoid hacks (and still periodically falling victim to various hacks), I went in search of a Python-based framework that I could customize ...

Unicode and HTML entities

In which we struggle with a cacophony of characters for the web.

Buried in the Python standard library, unicodedata contains most the information needed to interrogate and translate unicode characters. Unfortunately, it's underdocumented. More accurately, the docs are a terse list of what it does, but not why you might want to use it or how you use it. Unfortunately it ...

008 - paths across grid

Project Euler problem #8.

See Problem 8: Find the greatest product of five consecutive digits in a 1000-digit number.

This is fairly straightforward. First we define the number, using Python's implicit string continuation:

numstr = '73167176531330624919225119674426574742355349194934'
'96983520312774506326239578318016984801869478851843'
'85861560789112949495459501737958331952853208805511'
'12540698747158523863050715693290963295227443043557'
'66896648950445244523161731856403098711121722383113'
'62229893423380308135336276614282806444486645238749'
'30358907296290491560440772390713810515859307960866'
'70172427121883998797908792274921901699720888093776'
'65727333001053367881220235421809751254540594752243'
'52584907711670556013604839586446706324415722155397'
'53697817977846174064955149290862569321978468622482'
'83972241375657056057490261407972968652414535100474'
'82166370484403199890008895243450658541227588666881'
'16427171479924442928230863465674813919123162824586'
'17866458359124566529476545682848912883142607690042'
'24219022671055626321111109370544217506941658960408'
'07198403850962455444362981230987879927244284909188'
'84580156166097919133875499200524063689912560717606'
'05886116467109405077541002256983155200055935729725 ...

Euler 035 - counting circular primes

How many circular primes (primes that when rotated are still primes) are there below one million?

See Problem 20.

Again, this rescues a lot of code from previous problems, but the solution is less than perfect. The Euler guidelines say that every problem should be computable within a minute, but despite much tweaking, this solution takes significantly longer.

For speed, primes should be generated only once ...

Language Wars

"What language would you recommend to introduce programming to an audience of life science students at a bachelor level?"

(Originally published on BiocodersHub)

Following several lengthy and passionate discussions in different venues on what language to use for teaching bioinformatics, I've started cutting and pasting my reply. And here it is.


You'll get a lot of different opinions on this because:

  • It's a religious issue. That ...

Py-konval

Validation and conversion of data.

Note

This package has been taken over by TODO, who has substantially re-engineered it and chnaged some of the API. This page is maintained for largely historical purposes.

Background

The problem of sanitizing data (checking correctness and transforming to a useful form) is widespread throughout programming:

  • How do I verify ...

Py-qanda

Simple commandline questions

Background

Interactive command-line programs need to query users for information, be it text, choices from a list, or simple yes-or-no answers. qanda is a module of simple functions to prompt users for such information, allowing validation and cleanup of answers, default responses, consistent formatting and presentation of help text, hints ...

Surprises

In a way, tuples are one of the most magical parts of Python. How does the interpreter distinguish between a method call, a bracketed portion of a mathematical expression and an actual tuple?:

calc_midpoint (1, None, [2, 3], False)
x = w + (x * y)
a = (b, c)

The really smart thing ...

write argument must be string

When wsgi won't work.

So I was writing a web application using WSGI and - after working for a long time - it started erroring out with:

AssertionError: write() argument must be string

from deep inside the WSGI handler. Now, this usually means that you're returning unicode, which WSGI can't handle. However, it's ...

Can't use print

Who would of thunk it? print is a reserved word everywhere.

Perhaps this is buried in some specification, but it seems that you can't use print as the name of a method of a class. (It makes sense you can't override the global name, but in an object ...

psicons.core

A tool for documentable and reproducible analysis and research.

Note

Although I used psicons heavily for a while, better and more humane tools have come along. In particular, I'd recommend snakemake. This page is left here for historical interest

Background

Scientific analysis can be problematic:

  • It may involve multiple steps, each using the results of the previous stage ...

Getting the name of something

From the Department of the Blindingly Obvious.

Recently, I had the need to get an informative name from a range of Python objects for generating a useful error message. Where problems started was that these objects could include classes (new-style and old), functions, lambdas and built-in types. And here the logic started getting tricky.

  • To get the ...

osgb

A python module for interpreting Ordnance Survey grid references.

Background

Many geospatial locations within the UK are given with the Ordnance Survey system. While common and highly accurate, this coordinate system is peculiar to the UK and incompatible with many modern cartography tools. This module presents functions for converting between these and the most widely spread longitude-latitude system.

Installation ...

Epydoc go boom

Making a good but abandoned documentation tool work.

So, epydoc used to be the neatest and bestest documentation tool available for Python. It produced documentation from introspecting code and thus was the easiest way to the most accurate API documentation. However, it looks like the project has been abandoned. Attempts to run it over code result in sometimes ...

How to stop output and printing

The essential problem is how to not get output from a program.

Let me explain: Ete2, a Python module for representing phylogenies, has a number of dependencies (MySQLdb, Numpy, PyQt, etc.) that it doesn’t necessarily need and it can be installed without them. If you don’t use the associated functionality, you won’t need these dependencies. But, irritatingly, ete2 tries ...

Applescript via Python

Controlling Mac applications with appscript

Over the years, Apple has fallen in and out of love with Applescript, its "official" scripting language for MacOS. True, Applescript isn't going to go away any day now and true, it is a very simple language and easy to use. But if you don't want to have ...

Matplotlib

Notes on installing and using.

Matplotlib is cool. Did I say it was cool? I meant very cool. While principally a plotting library, it can be used for image manipulation and drawing.

draw_lines

Which is where our first problem occurs (as of the 0.8.4 version of matplotlib and possibly earlier). Should you try ...

biblio.webquery

Retrieving book data from online sources.

Background

This package presents a number of methods for querying webservices for bibliographic information, and includes two scripts for querying and renaming files by ISBN.

Installation

This package can be installed by the usual Pythonic methods:

  1. use your favourite installation tool:

    % easy_install biblio.webquery
    
  2. or download the source, unpack it ...

Encoding types in etree

Setting encoding in the header of ElementTree generated XML.

Not that I'm picking on ElementTree or anything but ...

Normally XML documents will declare their character encoding in their opening tag. And, normally, this will be utf8. (This may be a standard.) So this is a common sight:

xml version='1.0' encoding='utf8'?>

An oddity in ElementTree is ...

Installing MySQLdb

When unresolved linker symbols attack.

Good news: you can get reliable and easy access to a proper, fully-featured database from within Python.

Bad news: you may have to tinker to get it to (a) install properly. And if not done properly, this may cause it to (b) not work properly.

Good news: Download a .dmg ...

Installing Pydee on OSX

Short notes on getting Pydee (and PyQt) to run on a Mac.

Pydee is new and nifty graphical frontend for using Ipython. Unfortunately, there's no monolithic binary package available for installing (at least not on the Mac), and so users have to install and compile all the prerequisites. Given the the number of steps and small but important details, it's ...

Parsing dates in Python

One of those weird things that always slips my mind and always seems less obvious than it should be.

Parsing a date (object) from a string representation in Python seems oddly neglected or cumbersome as compared to the rest of the standard library that surrounds it. (Witness the number of ...

rst2beamer

A restructured text-based tool for preparing presentations.

Note

rst2beamer has been wildly successful and used far beyond what I imagined. Further development has been taken on by Ryan Krauss and subsequently others. The source has been moved to its own repo on github and that (and the PyPi page) should be used as the primary sources. No ...

TreeMaker

An application for the interactive building of taxonomies.

Background

Biodiversity assessment demands objective measures, because ultimately conservation is an issue of economics, prioritizing the use of limited resources for preserving taxa. The most general framework for such metrics are those that assess evolutionary distinctiveness as judged by how much of a phylogeny is conserved. However, their applicability is ...

Installing and using PIL

Overcoming the sometimes tricky installation of the Python Imaging Library and improving image quality.

While some people have reporting installing PIL on MacOSX without problems, there are legion reports that say otherwise. Frequently this revolve around PIL installing without JPEG support. There are a myriad of solutions (involving editing make files, configure flags, using fink to install dependencies, using package managers, casting goat entrails ...

Creating attributes on classes

Programmatic addition of methods.

I've gotten used to manipulating members of Python objects with getattr, setattr etc. But a recent similar problem had me stumped. I had a class that I wanted to create a large number of similar behaving properties on (they would all manipulate an internal dictionary):

>>> f = Foo()
>>> f.contributor ...

Slots

A new, and poorly explained, feature of Python classes.

Around version 2.2, Python rejigged its classes with some useful extensions. Unfortunately these enhancements have been explained so poorly that they appear in little published code.

One such enhancement is __slots__. An attribute of this name in a class restricts what attributes can be created in objects of that ...

MeSA

Macroevolutionary Analysis & Simulation.

Background

Phylogenies are smoking guns. An evolutionary history carrys in its very shape the fingerprints of every process that effected its creation: extinctions, key adaptations, radiations, colonizations, changes in the tempo and mode of macroevolution.

Unfortunately analysing phylogenies is difficult. The calculations of interest to any investigator are sometimes intricate ...