Marco Islas

Python trick: How to make lazy objects?

This time I'm about to talk about lazy object, I called like that because they are more or less similar to what we use in lazy treeviews (In Gtk) where the Tree shows expanders but didn't load the child nodes until you expand the node). What I want to do is create objects and count them, but don't load any data until I need it.

Why would I need this?. Well. I'm currently writing a program for my employer, this program load a list of customers and shows how many accounts it has. As this accounts are kind of complex I create an object for them. This object is responsible for load/save the data for it, and any data you need is accessed via properties.

In my first approach, the object was loading the data as soon as the constructor was called, but sometimes I never access to any info in that object, remember, a customer may have many (and I mean really many) accounts, then why load all the data if there is a chance that I'll never use it?. Well, that is because there is a chance that I have to use that data.

Then how could I create the object and then load the data when somebody require it?. That may be easy, may not. :-). One approach is making use of python properties. Just create the object and when someone want to use a "value" from the object use the property to retrieve the data, process it and then return a value. A simple property may be defined like this:

 
 
class demo(object):
    def get_some_property(self):
        #Here you can make use of whatever you want to retrieve the data
        pass
    def set_some_property(self, value):
        #Here you get a value and save it.
        pass
    some_property = property(get_some_property, set_some_property, None, 'Doc')
 

I just create a class "demo" and the property "some_property", everytime you use this:

 
d = demo()
d.some_property
 

Python will call the method get_some_property allowing you to do your trick. But if you have an object with more and more properties is better to get all data at one time and then just use properties to return a piece of it. You could call a method on every property if the data container is empty but for me it is not the best.

What you could do, is get the data the first time you need it. But, how could I know when a property/variable is requested on my object?

Well, Python classes have several reserved methods, and one of them is __getattribute__. This method let you handle every request to any property/class variable in your object.

With this you can digg in your object to know if the container is empty, and if it is empty fill it with data before returning the property value :-).

Lets say that the demo object have a __data dictionary containing the data. and propeties look into it for the right value, it also have a reference to the storage layer called storage.

 
class demo(object):
        def __init__(self):
                self.__data = {}
                self.storage = storage()
 
        def __getattribute__(self, name):
                obj = object.__getattribute__(self,name)
                #Check if the requested property is __data
                if name in("__data", '_account__data'):
                        #Get the reference of __data
                        d = object.__getattribute__(self,'_account__data')
                        if not d: #Get in only if __data is empty.
                                #Get your
                                s = object.__getattribute__(self,'storage')
                                #We can assign without using object.__getattribute__
                                self.__data = s.get_data()
                                #Get the value again
                                obj = object.__getattribute__(self,name)
                return obj
 

Pay attention that I used object.__getattribute__ instead self.property, if I use self.property will call self.__getattribute__ and will cause a recursion exception.

With this, we can create a great number of objects quickly and reducing the memory footprint.

blog comments powered by Disqus