What is the purpose of the -m switch?

Question

Could you explain to me what the difference is between calling

python -m mymod1 mymod2.py args

and

python mymod1.py mymod2.py args

It seems in both cases mymod1.py is called and sys.argv is

['mymod1.py', 'mymod2.py', 'args']

So what is the -m switch for?

Answer 1

Despite this question having been asked and answered several times (e.g., here, here, here, and here), in my opinion no existing answer fully or concisely captures all the implications of the -m flag. Therefore, the following will attempt to improve on what has come before.

Introduction (TLDR)

The -m flag does a lot of things, not all of which will be needed all the time. In short, it can be used to: (1) execute python code from the command line via modulename rather than filename (2) add a directory to sys.path for use in import resolution and (3) execute python code that contains relative imports from the command line.

Preliminaries

To explain the -m flag we first need to explain a little terminology.

Python's primary organizational unit is known as a module. Modules come in one of two flavors: code modules and package modules. A code module is any file that contains python executable code. A package module is any directory that contains other modules (either code modules or package modules). The most common type of code module is a *.py file while the most common type of package module is a directory containing an __init__.py file.

Python allows modules to be uniquely identified in two ways: modulename and filename. In general, modules are identified by modulename in Python code (e.g., import <modulename>) and by filename on the command line (e.g., python <filename>). All Python interpreters are able to convert modulenames to filenames by following the same few, well-defined rules. These rules hinge on the sys.path variable. By altering this variable one can change how Python resolves modulenames into filenames (for more on how this is done see PEP 302).

All modules (both code and package) can be executed (i.e., code associated with the module will be evaluated by the Python interpreter). Depending on the execution method (and module type) what code gets evaluated, and when, can change quite a bit. For example, if one executes a package module via python <filename> then <filename>/__main__.py will be executed. On the other hand, if one executes that same package module via import <modulename> then only the package's __init__.py will be executed.

Historical Development of -m

The -m flag was first introduced in Python 2.4.1. Initially its only purpose was to provide an alternative means of identifying the python module to execute from the command line. That is, if we knew both the <filename> and <modulename> for a module then the following two commands were equivalent: python <filename> <args> and python -m <modulename> <args>. One constraint with this iteration, according to PEP 338, was that -m only worked with top level modulenames (i.e., modules that could be found directly on sys.path without any intervening package modules).

With the completion of PEP 338 the -m feature was extended to support <modulename> representations beyond the top level. This meant names such as http.server were now fully supported. This extension also meant that each parent package in modulename was now evaluated (i.e., all parent package __init__.py files were evaluated) in addition to the module referenced by the modulename itself.

The final major feature enhancement for -m came with PEP 366. With this upgrade -m gained the ability to support not only absolute imports but also explicit relative imports when executing modules. This was achieved by changing -m so that it set the __package__ variable to the parent module of the given modulename (in addition to everything else it already did).

Use Cases

There are two notable use cases for the -m flag:

1. To execute modules from the command line for which one may not know their filename. This use case takes advantage of the fact that the Python interpreter knows how to convert modulenames to filenames. This is particularly advantageous when one wants to run stdlib modules or 3rd-party module from the command line. For example, very few people know the filename for the http.server module but most people do know its modulename so we can execute it from the command line using python -m http.server.

2. To execute a local package containing absolute or relative imports without needing to install it. This use case is detailed in PEP 338 and leverages the fact that the current working directory is added to sys.path rather than the module's directory. This use case is very similar to using pip install -e . to install a package in develop/edit mode.

Shortcomings

With all the enhancements made to -m over the years it still has one major shortcoming -- it can only execute modules written in Python (i.e., *.py). For example, if -m is used to execute a C compiled code module the following error will be produced, No code object available for <modulename> (see here for more details).

Detailed Comparisons

Module execution via import statement (i.e., import <modulename>):

• sys.path is not modified in any way
• __name__ is set to the absolute form of <modulename>
• __package__ is set to the immediate parent package in <modulename>
• __init__.py is evaluated for all packages (including its own for package modules)
• __main__.py is not evaluated for package modules; the code is evaluated for code modules

Module execution via command line with filename (i.e., python <filename>):

• sys.path is modified to include the final directory in <filename>
• __name__ is set to '__main__'
• __package__ is set to None
• __init__.py is not evaluated for any package (including its own for package modules)
• __main__.py is evaluated for package modules; the code is evaluated for code modules.

Module execution via command line with modulename (i.e., python -m <modulename>):

• sys.path is modified to include the current directory
• __name__ is set to '__main__'
• __package__ is set to the immediate parent package in <modulename>
• __init__.py is evaluated for all packages (including its own for package modules)
• __main__.py is evaluated for package modules; the code is evaluated for code modules

Conclusion

The -m flag is, at its simplest, a means to execute python scripts from the command line by using modulenames rather than filenames. The real power of -m, however, is in its ability to combine the power of import statements (e.g., support for explicit relative imports and automatic package __init__ evaluation) with the convenience of the command line.

Answer 2

The first line of the Rationale section of PEP 338 says:

Python 2.4 adds the command line switch -m to allow modules to be located using the Python module namespace for execution as scripts. The motivating examples were standard library modules such as pdb and profile, and the Python 2.4 implementation is fine for this limited purpose.

So you can specify any module in Python's search path this way, not just files in the current directory. You're correct that python mymod1.py mymod2.py args has exactly the same effect. The first line of the Scope of this proposal section states:

In Python 2.4, a module located using -m is executed just as if its filename had been provided on the command line.

With -m more is possible, like working with modules which are part of a package, etc. That's what the rest of PEP 338 is about. Read it for more info.

Answer 3

It's worth mentioning this only works if the package has a file __main__.py Otherwise, this package can not be executed directly.

python -m some_package some_arguments

The python interpreter will looking for a __main__.py file in the package path to execute. It's equivalent to:

python path_to_package/__main__.py somearguments

It will execute the content after:

if __name__ == "__main__":

Answer 4

I just want to mention one potentially confusing case.

Suppose you use pip3 to install a package foo, which contains a bar module. So this means you can execute python3 -m foo.bar from any directory. On the other hand, you have a directory structure like this:

src
|
+-- foo
    |
    +-- __init__.py
    |
    +-- bar.py

You are at src/. When you run python -m foo.bar, you are running the bar.py, instead of the installed module. However, if you are calling python -m foo.bar from any other directory, you are using the installed module.

This behavior certainly doesn't happen if you are using python instead of python -m, and can be confusing for beginners. The reason is the order how Python searches for modules.

Answer 5

In short, one of the best use case for 'python -m' switch is when you want to tell Python that you want to run a module instead of executing a .py file.

Consider this example: you have a Python script in a file named 'venv' (without '.py' file extension). If you issue this command:

python venv

then, Python will excute the 'venv' file in the current directory. However, if instead you want to create a new virtual environment using the 'python venv' module, you would run:

python -m venv

in which case, Python will run the 'venv' module, not the file 'venv'.

Another example, if you want to run Pyhton's built-in local http server and issue the command:

python http.server

you would get an error like:

python: can't open file '/home/user/http.server': [Errno 2] No such file or directory

That's because Python tried to execute a file called 'http.server' and didn't find it. So instead, you want to issue the same command but with '-m' switch:

python -m http.server

so that Python knows you want the module 'http.server' not the file.

Answer 6

Since this question comes up when you google Use of "python -m", I just wanted to add a quick reference for those who like to modularize code without creating full python packages or modifying PYTHONPATH or sys.path every time.

Setup

Let's setup the following file structure

.
├── f1
│   ├── f2
│   │   ├── __init__.py
│   │   └── test2.py
│   ├── __init__.py
│   └── test1.py
└── test.py

Let the present path be m1.

Using python -m instead of python ./*

1. Use . qualified module names for the files (because they're being treated as modules now). For example, to run the contents in ./f1/test1.py, we do

   python -m f1.test1
   

   and not

   python ./f1/test1.py
   

2. When using the module method, the sys.path in test1.py (when that is run) is m1. When using the ./ (relative file) method, the path is m1/f1.

   So we can access all files in m1 (and assume that it is a full python package) using -m. This is because the path to m1 is stored (as PYTHONPATH).

3. If we want to run deeply nested "modules", we can still use . (just as we do in import statements).

   # This can be done
   python -m f1.f2.test2
   

   And in test2.py, we can do from f1.test1 import do_something without using any path gimmicks in it.

4. Every time we do module imports this way, the __init__.py is automatically called. This is true even when we're nesting.

   python -m f1.f2.test2
   

   When we do that, the ./f1/__init__.py is called, followed by ./f1/f2/__init__.py.