Manipulating Dictionaries - nmbooker/python-funbox GitHub Wiki

The module funbox.mappings has tools for transforming and generating dictionaries in various ways.

The docstrings describe how its functions work, but here's a worked example.

Example

Say you have a series of records that look like this:

recs = [
    {'id': 1, 'forename': 'Fred', 'surname': 'Bloggs'},
    {'id': 2, 'forename': 'Jane', 'surname': 'Doe'},
]

You want to end up with a dictionary mapping each record's id to the remainder of the record (with the 'id' field removed) but with a new 'fullname' field calculated from 'forename' and 'surname'. We assume that 'forename' and 'surname' will always be a string, for simplicity's sake.

We have a function in funbox.mappings that can do each step of this.

This will calculate our forename field and append it to a function.

We already have imap in itertools in the standard library to help us do stuff over lists lazily.

So step one is to add the extra field to each record using mappings.with_calculated:

from itertools import imap
import funbox.mappings as mp

def fullname(record):
    return '%(forename)s %(surname)s' % record

calculated_recs = imap(mp.with_calculated({'fullname': fullname}), recs)

Step two is to make the dictionary with the keys, using mappings.pull_key:

from operator import itemgetter

# note syntax of (...)(...)   - that's because pull_key is curried
indexed_recs = mp.pull_key(itemgetter('id'))(calculated_recs)

Step three is to take our new dictionary, and remove the 'id' key from each value (each value is a dict):

final_records = mp.map_values(mp.without_keys(['id']), indexed_recs)

That gives us what we're after.

So putting it all together into a runnable script, we have:

#! /usr/bin/env python

recs = [
    {'id': 1, 'forename': 'Fred', 'surname': 'Bloggs'},
    {'id': 2, 'forename': 'Jane', 'surname': 'Doe'},
]

from itertools import imap
import funbox.mappings as mp
from operator import itemgetter
import pprint

def fullname(record):
    return '%(forename)s %(surname)s' % record

calculated_recs = imap(mp.with_calculated({'fullname': fullname}), recs)
indexed_recs = mp.pull_key(itemgetter('id'))( calculated_recs)
final_records = mp.map_values(mp.without_keys(['id']), indexed_recs)

pprint.pprint(final_records)

Save the above program in a file, and run it, and it will print something like this (remembering ordering in dicts is nondeterministic):

{1: {'forename': 'Fred', 'fullname': 'Fred Bloggs', 'surname': 'Bloggs'},
 2: {'forename': 'Jane', 'fullname': 'Jane Doe', 'surname': 'Doe'}}

Other Problems

The above example is remarkably similar to the kind of transformations required to implement function fields in OpenERP for multi=something. This is no coincidence - I wrote this module in reaction to having to write the same loop out three times to do just that.

It's not so easy to produce a standalone runnable example for OpenERP without becoming really contrived, but believe me if I say you just need to replace this in the above finished example:

def fullname(record):
    return '%(forename)s %(surname)s' % record

calculated_recs = imap(mp.with_calculated({'fullname': fullname}), recs)

with this:

from operator import attrgetter

def fullname(obj):
    return '%s %s' % (obj.forename, obj.surname)

key_functions = {'id': attrgetter('id'), 'fullname': fullname}
calculated_recs = imap(mp.to_dict(key_functions), recs)

That's not far off a generic function for that task.