A Python import is really a search of your filesystem dressed up as a language feature. When you write import os.path, the interpreter walks a list of directories called sys.path, opens the first match that it finds, runs that file once, and then binds the module object to the name os. That is the whole idea behind Python's import path system, and once you can see the path, the famous ModuleNotFoundError stops being mysterious.
This is the first lesson in the Python Modules and Packages module, and the next lesson in this module is about "Structuring a Package: __init__.py and Layouts".
What actually happens during an import statement?
Take import json as an example. Python does five things, always in the same order. It splits the dotted name into json. It searches sys.path for a file called json.py or for a package folder called json. It creates a module object to hold the namespace of that module. It runs the top-level code of the module inside that namespace. Finally, it binds the name json in the scope where you wrote the import.
The binding is the part that is worth understanding properly, because an import does not copy code into your file. It creates a second reference to one shared module object. That is why a module can hold state that every importer sees, and it is also why editing a module in the middle of a session does nothing at all until you reload it.
import math
import math as m
from math import sqrt, pi
from math import gcd
print(math.floor(3.7)) # 3
print(m.ceil(3.2)) # 4
print(sqrt(16)) # 4.0
print(round(pi, 4)) # 3.1416
print(gcd(12, 18)) # 6
All four spellings reach the very same module object. import math binds the module itself. from math import sqrt binds one name from inside it. import math as m binds the module under a different name. And a dotted from package.module import name walks into the package first and then picks the name out of the submodule. Choose a spelling for each situation instead of by habit, and then stay consistent inside a project.
Where does Python look for a module?
sys.path is an ordinary list of folders, and it is searched from left to right. The first entry is a special one. It is the folder of the script that you ran, or an empty string when you are in the interactive interpreter. After that come your own user folder, any folder that you added with the PYTHONPATH environment variable, and finally the standard library and the site-packages folder where pip installs third-party code.
import sys
import importlib.util
print('sys.path[0]:', sys.path[0] or '(empty - interactive mode)')
print('json found:', importlib.util.find_spec('json') is not None)
print('nope found:', importlib.util.find_spec('no_such_module_xyz') is None)
import json
print('json cached:', 'json' in sys.modules)
print('json file:', json.__file__)
The find_spec call is the diagnostic that you want when an import fails. It searches exactly the same path that the importer uses, and it gives you None instead of raising an error, which tells you straight away whether the problem is a missing file or a broken one. The last two lines also show where the module actually came from. A path inside your own project means that your code has taken the place of something else, and a path inside the Python installation means that you are looking at a library module.
Two facts about that list explain most of the surprises that people meet with imports. First, the folder of the script comes before the standard library, so a file called random.py in your project silently replaces the standard library's random module for every script that you run from that folder. Second, python script.py puts the script's own folder on the path, while python -m package.module puts your current working directory there instead. That single difference is the whole reason that -m works for package code and plain script invocation does not.
Why does a module's code run only once?
Python keeps every imported module in sys.modules, keyed by its name. The second time that you import it, the interpreter finds the cached object and simply rebinds the name, so the top-level code does not run a second time. That is why module-level code is a bad place for side effects. It happens exactly once, at a moment that you did not choose, and quite often during an import that you did not even expect.
The idiom that gives you control over this is the __main__ guard. __name__ is set to "__main__" only for the file that you ran directly, and for an imported file it is the module's own name. Putting your demo code behind that guard means that the same file can be a library and a runnable script at the same time.
# geometry.py
"""Small geometry helpers used by the demo."""
import math
def circle_area(radius):
"""Return the area of a circle with the given radius."""
return math.pi * radius ** 2
def hypotenuse(a, b):
"""Return the length of the hypotenuse of a right triangle."""
return math.hypot(a, b)
if __name__ == '__main__':
print('circle_area(2) =', circle_area(2))
print('hypotenuse(3, 4) =', hypotenuse(3, 4))
# runner.py - imports geometry, so the guard does not fire
from geometry import circle_area, hypotenuse
print('area of r=2:', circle_area(2))
print('hypotenuse(3, 4):', hypotenuse(3, 4))
Run python runner.py and you get two lines of output. Run python geometry.py and you get the same two values under different labels, because now the file is the program and not a library. The important part is what does not happen. Importing geometry from runner.py never prints the demo lines, because the guard is false for an imported module.
How do I fix a circular import error?
Two modules that import each other break in a very specific way. The first module starts running, reaches its import statement, and begins the second module. The second module then tries to import the first one, but the first module is only half built, so the name that it is trying to take does not exist yet.
# module_a.py
from module_b import b_function
def a_function():
return 'a' + b_function()
# module_b.py
from module_a import a_function
def b_function():
return 'b'
Running python module_a.py raises ImportError: cannot import name 'b_function' from 'module_b'. The exact wording changes a little between Python versions, and some of them add a hint about a partly initialised module, but the cause is always the same. A name was imported from a module that had not finished loading.
There are two clean repairs. The first one is to import the module instead of the name, so that the lookup happens later, at call time, when both modules are complete. The second one is to move the shared piece into a third module that both of them import.
# module_a.py - repaired by importing the module, not the name
import module_b
def a_function():
return 'a' + module_b.b_function()
if __name__ == '__main__':
print(a_function())
# module_b.py - repaired the same way
import module_a
def b_function():
return 'b'
With those two versions in place, python module_a.py prints a+b. The deeper repair is a question of design. A cycle between two modules is usually a sign that one of them is doing two different jobs, and splitting the shared data into a third module removes the cycle instead of hiding it.
Which import traps bite beginners most often?
- Shadowing the name of a standard library module. Your own
random.py,json.py, ortypes.pyreplaces the library version for any script that runs from that folder. Rename the file, and never fix this by fiddling withsys.path. - Running a script from the wrong folder.
sys.path[0]is the folder of the script and not the folder that you typed the command in, so a sibling module that "clearly exists" stays invisible from somewhere else. - Expecting a package to be a single file.
import mypkgneeds a folder with an__init__.pyinside it, whileimport mypkg.modneeds amod.pyfile inside that folder. - Wildcard imports.
from mypkg import *hides where the names came from, overwrites your own variables without telling you, and breaks as soon as the module grows a new export. - Installing into the wrong interpreter.
pip installcan target a different Python than the one that runs your script, whilepython -m pip installalways targets the interpreter that you are holding.
Key takeaways
- An import is a folder search, followed by a one-time execution, followed by a name binding.
sys.pathis the search list, andsys.modulesis the cache of everything that has already been imported.- Module code runs once, at import time, so put side effects behind
if __name__ == '__main__':. find_specanswers the question "could Python find this?" without raising an exception.- Circular imports are a design smell, so import modules rather than names, or move the shared code out into a third module.
Try it yourself: Create a folder called import-lab with two files inside it. Put a shout function and a __main__ guard that prints a message into toolbox.py, and put an import of shout plus a call to it into app.py. Run python app.py and notice that the message from the guard never appears. Then run python toolbox.py and watch it appear. Next, add import sys and a line that prints sys.path[0] to app.py, and run the same command from the parent folder to see the path change.