2014-03-24

Python has an intriguing feature called "slots".

Ordinarily, every instance of a user-defined Python class has a __dict__ attribute. This is a dictionary used to dynamically store attributes as key-value pairs.

class A(object):
    pass

a = A()
a.attr = 5
a.__dict__ == {'attr': 5}

The __slots__ class attribute can be used to statically define available instance attributes, which by default removes the instance __dict__ altogether.

class B(object):
    __slots__ = ['b']

b = B()
b.b = 3
b.a = 1    # => raises an AttributeError
b.__dict__  # => raises an AttributeError

The main reason to define __slots__ is to save memory. Every instance __dict__ takes up space; if all instance attribute names are known at class creation time, then there's no need to have an instance __dict__.

One twist on __slots__ is that it's possible to explicitly define a __dict__ slot, which will lead to the instance dict being created after all.

class C(object):
    __slots__ = ['c', '__dict__']

c = C()
c.c = 1
c.d = 2
c.__dict__ == {'d': 2}

Note that the "d" attribute appears in the instance __dict__, but the "c" attribute does not.

A final piece of background information is that slots accumulate through inheritance. A slot should be defined once per class hierarchy.

Don't believe everything you read

The Python docs say, "When inheriting from a class without __slots__, the __dict__ attribute of that class will always be accessible, so a __slots__ definition in the subclass is meaningless." The last statement is actually incorrect.

I noticed this when using the pympler memory measurement tool. Python's standard library includes sys.getsizeof, but that doesn't count nested objects. We can see that Pympler provides more intuitive numbers (measurements taken on Python 2.7.5).

from sys import getsizeof
from pympler.asizeof import asizeof

getsizeof([]) == 72
asizeof([]) == 72

getsizeof([[]]) == 80
asizeof([[]]) == 152

Now, the real test. A is a class without __slots__, and B is a subclass with a __slots__ definition. Do the __slots__ matter?

from pympler.asizeof import asizeof

class A(object):
    pass

class B(A):
    __slots__ = ['b']

a = A()
b = B()
asizeof(a) == 344
asizeof(b) == 408

attrs = ['a', 'b', 'c', 'd', 'e', 'f']
for name in attrs:
    setattr(a, name, 0)
    setattr(b, name, 0)

asizeof(a) == 1376
asizeof(b) == 640

An instance of the class with slots defined starts out slightly larger, presumably because it has storage allocated for attributes defined as slots. But after we set six attributes on each instance, the instance without __slots__ uses substantially more memory. Why's that?

len(a.__dict__) == 6
len(b.__dict__) == 5

asizeof(a.__dict__) == 1312
asizeof(b.__dict__) == 504

Python's dict objects resize after a certain number of key-value pairs are added. The first five resizes occur at 6, 22, 86, 342, and 1366 elements (increase by alternating powers of 2: +16, +64, +256, +1024).

An instance of class B has only five elements in its dict, with one in special slots storage. The instance of A has six key-value pairs in its __dict__, which explains the large leap in memory consumption.

Conclusion

So, perhaps there's a minor inaccuracy in the stellar Python documentation. In any case, I learned a lot in the lead-up to this post.