How do I write good/correct package __init__.py files

Question

My package has the following structure:

mobilescouter/
    __init__.py #1
    mapper/
        __init__.py  #2
        lxml/
            __init__.py #3
            vehiclemapper.py
            vehiclefeaturemapper.py
            vehiclefeaturesetmapper.py
        ...
        basemapper.py
   vehicle/
        __init__.py #4
        vehicle.py
        vehiclefeature.py
        vehiclefeaturemapper.py
   ...

I'm not sure how the __init__.py files should be correctly written.
The __init__.py #1 looks like:

__all__ = ['mapper', 'vehicle']
import mapper
import vehicle

But how should for example __init__.py #2 look like? Mine is:

__all__ = ['basemapper', 'lxml']
from basemaper import *
import lxml

When should be __all__ used?

Answer 1

__all__ is very good - it helps guide import statements without automatically importing modules http://docs.python.org/tutorial/modules.html#importing-from-a-package

using __all__ and import * is redundant, only __all__ is needed

I think one of the most powerful reasons to use import * in an __init__.py to import packages is to be able to refactor a script that has grown into multiple scripts without breaking an existing application. But if you're designing a package from the start. I think it's best to leave __init__.py files empty.

for example:

foo.py - contains classes related to foo such as fooFactory, tallFoo, shortFoo

then the app grows and now it's a whole folder

foo/
    __init__.py
    foofactories.py
    tallFoos.py
    shortfoos.py
    mediumfoos.py
    santaslittlehelperfoo.py
    superawsomefoo.py
    anotherfoo.py

then the init script can say

__all__ = ['foofactories', 'tallFoos', 'shortfoos', 'medumfoos',
           'santaslittlehelperfoo', 'superawsomefoo', 'anotherfoo']
# deprecated to keep older scripts who import this from breaking
from foo.foofactories import fooFactory
from foo.tallfoos import tallFoo
from foo.shortfoos import shortFoo

so that a script written to do the following does not break during the change:

from foo import fooFactory, tallFoo, shortFoo

Answer 2

My own __init__.py files are empty more often than not. In particular, I never have a from blah import * as part of __init__.py -- if "importing the package" means getting all sort of classes, functions etc defined directly as part of the package, then I would lexically copy the contents of blah.py into the package's __init__.py instead and remove blah.py (the multiplication of source files does no good here).

If you do insist on supporting the import * idioms (eek), then using __all__ (with as miniscule a list of names as you can bring yourself to have in it) may help for damage control. In general, namespaces and explicit imports are good things, and I strongly suggest reconsidering any approach based on systematically bypassing either or both concepts!-)

Answer 3

Your __init__.py should have a docstring.

Although all the functionality is implemented in modules and subpackages, your package docstring is the place to document where to start. For example, consider the python email package. The package documentation is an introduction describing the purpose, background, and how the various components within the package work together. If you automatically generate documentation from docstrings using sphinx or another package, the package docstring is exactly the right place to describe such an introduction.

For any other content, see the excellent answers by firecrow and Alex Martelli.