Method Chaining in Python with Iterators

I have been reading about Scala and learning improved coding practices along the way.  In additional to wishing that I could use such a cool language at work, I’ve also been trying to apply what I’m learning towards improving my Python code.  One of these “scalaisms” is the encouragement of method chaining.  I think I must have considered method chaining a bit too smelly to really give it a shot.  Once I saw some examples in a couple of Scala books, though, my mind opened up to the idea.  Thinking about method chaining in Python while reading about iterators and generators (reading about functional concepts in Python) led me to consider applying both concepts.  I came up with the idea of chaining iterators as a way to iterate over and filter large lists.  A nice thing about iterators is that they are lazy.  They only return their contents one item at a time.  I wanted to chain iterators that could be dynamically plugged together to filter the contents of a list lazily.  The rest of this blog post will be about my proof-of-concept code.

First, I needed something to iterate over. I decided to work with objects that represented pets in a pet store (the simplest idea I could come up with).

class Pet(object):
    @staticmethod
    def from_list(args_list):
        kwargs = dict()
        type_str = args_list[0]
        if type_str == "Dog":
            pet_clss = Dog
        elif type_str == "Cat":
            pet_clss = Cat
        else:
            raise ValueError("Unknown Pet type '%s'" % type_str)
        return pet_clss(**dict([[args_list[x], args_list[x+1]] for x in range(1, len(args_list)-1, 2)]))

    def __init__(self, **kwargs):
        self.name = kwargs['name']
        self.breed = kwargs['breed']
        self.owner = kwargs.get('owner', None)
        self.color = kwargs['color']

    def was_sold(self):
        if self.owner is not None:
            return True
        else:
            return False

class Cat(Pet):
    def __init__(self, **kwargs):
        Pet.__init__(self, **kwargs)
        self.meows = kwargs.get("meows", True)

class Dog(Pet):
    def __init__(self, **kwargs):
        Pet.__init__(self, **kwargs)
        self.barks = kwargs.get("barks", True)

I wanted to be able to load data from a CSV file, hence the from_list() constructor.  Here is a parent class called Pet which is inherited by Cat and Dog.

The rest of the code is based on FilterablePetStore.  This is an abstract class that I used to test two different approaches to implementing a filterable PetStore class.

class FilterablePetStore(object):
    def __iter__(self):
        raise NotImplementedError()

    def filter(self, filter_func):
        return PetStoreView(self, filter_func)

    def was_sold(self):
        return self.filter(lambda pet: pet.was_sold())

    def is_type(self, test_type):
        return self.filter(lambda pet: isinstance(pet, test_type))

    def is_color(self, color):
        return self.filter(lambda pet: pet.color.upper() == color.upper())

    def is_tabby(self):
        return self.filter(lambda pet: pet.color.upper().endswith("TABBY"))

    def as_list(self):
        return [pet for pet in self]

FilterablePetStore just contains some ideas of how a pet store might want to filter their list of pets.  Notice also that filter() method takes a function as its argument, and all of the other methods simple reuse the filter() method by supplying a function.

First I’ll show the simple way to implement method chaining in Python.  This does not use iterators.  Each method in the chain returns a new object which contains its own list of pets.  (I actually coded this one second, hence the ‘2’ in the name).

class PetStore2(FilterablePetStore):
    def __init__(self, list_of_pets):
        self.pets = list_of_pets

    def __iter__(self):
        return iter(self.pets)

    def filter(self, filter_func):
        return PetStore2(filter(filter_func, self.pets))

    def as_list(self):
        return self.pets

This should be pretty straight forward.  The filter() method returns a new PetStore2 instance with a new (possibly smaller) list of pets.  Additional methods are inherited from FilterablePetStore.

The more interesting implementation uses iterators.  The goal of this code was to lazily evaluate each call to next() (see iterators) all the way through the chain of methods.

class PetStore(FilterablePetStore):
    def __init__(self, list_of_pets):
        self.pets = list_of_pets

    def __getitem__(self, index):
        assert isinstance(index, int), "Index must be an integer"
        return self.pets[index]

    def __iter__(self):
        return PetStoreView(self, lambda pet: True)

class PetStoreView(FilterablePetStore):
    def __init__(self, pets, filter_func):
        self._pets = pets
        self._location = 0
        self._filter_function = filter_func

    def __iter__(self):
        return self

    def next(self):
        try:
            try:
                next_item = self._pets.next()
            except AttributeError:
                next_item = self._pets[self._location]
                self._location += 1
        except IndexError:
            raise StopIteration()
        if self._filter_function(next_item):
            return next_item
        else:
            return self.next()

Here the iterator is implemented by the PetStoreView class.  A potential drawback of this approach is that you don’t end up with a PetStore object, but since the goal was to create an iterator chain, this isn’t a big deal.  Each element is passed down the filter chain and, if it passes all of the filters, is iterated over.  Each element waits until the next iteration before being filtered; there are no intermediary lists.

So what is my conclusion?  That will have to come in another post!

Here is the code for anyone who wants to play with it.