How to fix "Attempted relative import in non-package" even with __init__.py

Question

I'm trying to follow PEP 328, with the following directory structure:

pkg/
  __init__.py
  components/
    core.py
    __init__.py
  tests/
    core_test.py
    __init__.py

In core_test.py I have the following import statement

from ..components.core import GameLoopEvents

However, when I run, I get the following error:

tests$ python core_test.py 
Traceback (most recent call last):
  File "core_test.py", line 3, in <module>
    from ..components.core import GameLoopEvents
ValueError: Attempted relative import in non-package

Searching around I found "relative path not working even with __init__.py" and "Import a module from a relative path" but they didn't help.

Is there anything I'm missing here?

Answer 1

To elaborate on Ignacio Vazquez-Abrams's answer:

The Python import mechanism works relative to the __name__ of the current file. When you execute a file directly, it doesn't have its usual name, but has "__main__" as its name instead. So relative imports don't work.

You can, as Igancio suggested, execute it using the -m option. If you have a part of your package that is meant to be run as a script, you can also use the __package__ attribute to tell that file what name it's supposed to have in the package hierarchy.

See http://www.python.org/dev/peps/pep-0366/ for details.

Answer 2

Yes. You're not using it as a package.

python -m pkg.tests.core_test

Answer 3

It depends on how you want to launch your script.

If you want to launch your UnitTest from the command line in a classic way, that is:

python tests/core_test.py

Then, since in this case 'components' and 'tests' are siblings folders, you can import the relative module either using the insert or the append method of the sys.path module. Something like:

import sys
from os import path
sys.path.append( path.dirname( path.dirname( path.abspath(__file__) ) ) )
from components.core import GameLoopEvents

Otherwise, you can launch your script with the '-m' argument (note that in this case, we are talking about a package, and thus you must not give the '.py' extension), that is:

python -m pkg.tests.core_test

In such a case, you can simply use the relative import as you were doing:

from ..components.core import GameLoopEvents

You can finally mix the two approaches, so that your script will work no matter how it is called. For example:

if __name__ == '__main__':
    if __package__ is None:
        import sys
        from os import path
        sys.path.append( path.dirname( path.dirname( path.abspath(__file__) ) ) )
        from components.core import GameLoopEvents
    else:
        from ..components.core import GameLoopEvents

Answer 4

You can use import components.core directly if you append the current directory to sys.path:

if __name__ == '__main__' and __package__ is None:
    from os import sys, path
    sys.path.append(path.dirname(path.dirname(path.abspath(__file__))))

Answer 5

In core_test.py, do the following:

import sys
sys.path.append('../components')
from core import GameLoopEvents

Answer 6

The issue is with your testing method.

You tried python core_test.py and then you will get this error:

ValueError: Attempted relative import in non-package

Reason: you are testing your packaging from a non-package source.

So test your module from the package source.

If this is your project structure,

pkg/
  __init__.py
  components/
    core.py
    __init__.py
  tests/
    core_test.py
    __init__.py

cd pkg

python -m tests.core_test # dont use .py

or from outside pkg/

python -m pkg.tests.core_test

single . if you want to import from a folder in the same directory. For each step back, add one more.

hi/
  hello.py
how.py

In how.py

from .hi import hello

in case if you want to import how from hello.py:

from .. import how

Answer 7

If your use case is for running tests, and it seams that it is, then you can do the following. Instead of running your test script as python core_test.py use a testing framework such as pytest. Then on the command line you can enter

$$ py.test

That will run the tests in your directory. This gets around the issue of __name__ being __main__ that was pointed out by @BrenBarn. Next, put an empty __init__.py file into your test directory, this will make the test directory part of your package. Then you will be able to do

from ..components.core import GameLoopEvents

However, if you run your test script as a main program then things will fail once again. So just use the test runner. Maybe this also works with other test runners such as nosetests but i haven't checked it. Hope this helps.

Answer 8

My quick-fix is to add the directory to the path:

import sys
sys.path.insert(0, '../components/')

Answer 9

As you have already marked everything as a module, there isn't any need to use the relative reference if you launch it as a Python module.

Instead of

from ..components.core import GameLoopEvents

simply

from pkg.components.core import GameLoopEvents

When you run from the parent of the package, use the following:

python -m pkg.tests.core_test

Answer 10

As Paolo said, we have two invocation methods:

1. python -m tests.core_test
2. python tests/core_test.py

One difference between them is the sys.path[0] string. Since the interpret will search sys.path when doing import, we can do with tests/core_test.py:

if __name__ == '__main__':
    import sys
    from pathlib import Path
    sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
    from components import core
    <other stuff>

And more after this, we can run core_test.py with other methods:

cd tests
python core_test.py
python -m core_test
...

Note, tested only on Python 3.6.

Answer 11

I found out that adding an __all__= ['submodule', ...] to the init.py file and then using the from <CURRENT_MODULE> import * in the target works fine.

Answer 12

You can use from pkg.components.core import GameLoopEvents. For example, I use PyCharm.

The below is my project structure image. I just import from the root package, and then it works:

Answer 13

This approach worked for me and is less cluttered than some solutions:

try:
  from ..components.core import GameLoopEvents
except ValueError:
  from components.core import GameLoopEvents

The parent directory is in my PYTHONPATH, and there are __init__.py files in the parent directory and this directory.

The above always worked in Python 2, but Python 3 sometimes hit an ImportError or ModuleNotFoundError (the latter is new in Python 3.6 and a subclass of ImportError), so the following tweak works for me in both Python 2 and Python 3:

try:
  from ..components.core import GameLoopEvents
except ( ValueError, ImportError):
  from components.core import GameLoopEvents

Answer 14

Try this

import components
from components import *

Answer 15

python <main module>.py does not work with relative import

The problem is relative import does not work when you run a __main__ module from the command line

python <main_module>.py

It is clearly stated in PEP 338.

The release of 2.5b1 showed a surprising (although obvious in retrospect) interaction between this PEP and PEP 328 - explicit relative imports don't work from a main module. This is due to the fact that relative imports rely on __name__ to determine the current module's position in the package hierarchy. In a main module, the value of __name__ is always '__main__', so explicit relative imports will always fail (as they only work for a module inside a package).

Cause

• Python Bug Tracker Issue1510172: Absolute/relative import not working?

The issue isn't actually unique to the -m switch. The problem is that relative imports are based on __name__, and in the main module, __name__ always has the value __main__. Hence, relative imports currently can't work properly from the main module of an application, because the main module doesn't know where it really fits in the Python module namespace (this is at least fixable in theory for the main modules executed through the -m switch, but directly executed files and the interactive interpreter are completely out of luck).

To understand further, see Relative imports in Python 3 for the detailed explanation and how to get it over.

Answer 16

Here is a workaround. Here's a bit of context. I wanted to test out one of the methods I've in a file. When I run it from within

if __name__ == "__main__":

it always complained of the relative imports. I tried to apply the above solutions, but failed to work, since there were many nested files, each with multiple imports.

Here's what I did. I just created a launcher, an external program that would import necessary methods and call them. Though, not a great solution, it works.

Answer 17

For me only this worked: I had to explicitly set the value of package to the parent directory, and add the parent directory to sys.path

from os import path
import sys
if __package__ is None:
    sys.path.append( path.dirname( path.dirname( path.abspath(__file__) ) ) )
    __package__= "myparent"

from .subdir import something # the . can now be resolved

I can now directly run my script with python myscript.py.

Answer 18

If your project structure would look like this:

   project
     |
     | --- module1
     |      |
     |      file1.py
     |
     |-----module2
     |     |
     |     file2.py

and you are going import file1.py from within file2.py, you can do this in file2.py:

import sys
sys.path.append('.')

import file2

I still don't know why and how, but it worked for me.

Answer 19

Here's one way which will piss off everyone, but it works pretty well. In tests, run:

ln -s ../components components

Then just import components like you normally would.

Answer 20

None of the answers cover Jupyter/ Jupyterlab notebooks so here goes:

Problem Statement

If your folder structure is

root
  +- foo
  |   +-- __init__.py
  |   +-- file1.py
  +- bar
      +-- __init__.py
      +-- file2.ipynb

If you start jupyter using the command:

root$ jupyter lab

In Jupyter, navigate to file2.ipynb. In the file2 notebook, try to import file1. It will fail. If you execute import sys followed by print(sys.path) you will see that the path to file1.py is not in the path.

Solution

Start jupyter lab using the command below to fix this:

root$ PYTHONPATH=<path to root>/root/foo/file1:$PYTHONPATH jupyter lab

Explanation

You are defining the path to file1: <path to root>/root/foo/file1

You are appending this to PYTHONPATH: <path to root>/root/foo/file1:$PYTHONPATH

You are updating PYTHONPATH: PYTHONPATH=<path to root>/root/foo/file1:$PYTHONPATH

However you did not update PYTHONPATH for the system, but just for the jupyter lab run: PYTHONPATH=<path to root>/root/foo/file1:$PYTHONPATH jupyter lab

You can confirm that file1.py is now in the path by examining sys.path.

Answer 21

This is very confusing and if you are using IDE like PyCharm, it's little more confusing.

What worked for me:

1. Make PyCharm project settings (if you are running Python from a venv or from Python directory)

2. There is nothing wrong with the way you defined. Sometime it works with:

   from folder1.file1 import class
   

   If it does not work, use:

   import folder1.file1
   

3. Your environment variable should be correctly mentioned in system or provide it in your command-line argument.

Answer 22

Let's say you DON'T want to do any of these:

• run with python -m pkg.tests.core_test
• edit __package__
• add if check in __main__
• edit sys.path by hand
• restructure the project

Here is what you can do instead:

$ pip install importmonkey [github] [pip] [docs]

├─ src
│   └─ project
│       ├─ __init__.py
│       └─ module.py
└─ test
    └─ test.py

# In test.py

from importmonkey import add_path
add_path("../src")  # relative to current __file__
import project

Answer 23

Because your code contains if __name__ == "__main__", which doesn't be imported as a package, you'd better use sys.path.append() to solve the problem.