Compare commits

..

No commits in common. "main" and "4.0.1" have entirely different histories.
main ... 4.0.1

156 changed files with 6344 additions and 5085 deletions

33
.coveragerc Normal file
View file

@ -0,0 +1,33 @@
[run]
parallel = True
branch = True
source =
flake8
tests
omit =
# Don't complain if non-runnable code isn't run
*/__main__.py
[paths]
source =
src/flake8
.tox/*/lib/python*/site-packages/flake8
.tox/pypy/site-packages/flake8
[report]
show_missing = True
skip_covered = True
exclude_lines =
# Have to re-enable the standard pragma
\#\s*pragma: no cover
# Don't complain if tests don't hit defensive assertion code:
^\s*raise AssertionError\b
^\s*raise NotImplementedError\b
^\s*return NotImplemented\b
^\s*raise$
# Don't complain if non-runnable code isn't run:
^if __name__ == ['"]__main__['"]:$
^\s*if False:
^\s*if TYPE_CHECKING:

2
.github/FUNDING.yml vendored
View file

@ -1,2 +0,0 @@
github: asottile
tidelift: pypi/flake8

33
.github/ISSUE_TEMPLATE.md vendored Normal file
View file

@ -0,0 +1,33 @@
Please read this brief portion of documentation before going any further: http://flake8.pycqa.org/en/latest/internal/contributing.html#filing-a-bug
<!--
*************************************************************************
NOTE: flake8 is a linting framework and does not implement any checks
if you are reporting a problem with a particular check, please track down
the plugin which implements that check.
some common ones:
- F###: https://github.com/pycqa/pyflakes
- E###, W###: https://github.com/pycqa/pycodestyle
*************************************************************************
-->
*Please describe how you installed Flake8*
Example:
```
$ pip install --user flake8
$ brew install flake8
# etc.
```
**Note**: Some *nix distributions patch Flake8 arbitrarily to accommodate incompatible software versions. If you're on one of those distributions, your issue may be closed and you will be asked to open an issue with your distribution package maintainers instead.
*Please provide the exact, unmodified output of `flake8 --bug-report`*
*Please describe the problem or feature*
*If this is a bug report, please explain with examples (and example code) what you expected to happen and what actually happened.*

View file

@ -1,86 +0,0 @@
name: bug report
description: something went wrong
body:
- type: markdown
attributes:
value: >
Please read this brief portion of documentation before going any
further:
https://flake8.pycqa.org/en/latest/internal/contributing.html#filing-a-bug
- type: markdown
attributes:
value: >
**NOTE: flake8 is a linting framework and does not implement any
checks**
- type: markdown
attributes:
value: >
_if you are reporting a problem with a particular check, please track
down the plugin which implements that check_
- type: textarea
id: install
attributes:
label: how did you install flake8?
description: 'note: this will be rendered as ```console automatically'
placeholder: |
$ pip install flake8 # or `brew install flake8` etc.
Collecting flake8
...
Successfully installed flake8...
render: console
validations:
required: true
- type: markdown
attributes:
value: >
**Note**: Some *nix distributions patch Flake8 arbitrarily to
accommodate incompatible software versions. If you're on one of those
distributions, your issue may be closed and you will be asked to open
an issue with your distribution package maintainers instead.
- type: textarea
id: bug-report
attributes:
label: unmodified output of `flake8 --bug-report`
description: 'note: this will be rendered as ```json automatically'
placeholder: |
{
"platform": {
"...": "...
}
}
render: json
validations:
required: true
- type: textarea
id: freeform
attributes:
label: describe the problem
description: >
please provide **sample code** and **directions for reproducing
your problem** including the **commands you ran**, their
**unedited output**, and **what you expected to happen**
value: |
#### what I expected to happen
...
#### sample code
```python
print('hello world!')
```
#### commands ran
```console
$ flake8 t.py
...
```
validations:
required: true

View file

@ -1,27 +0,0 @@
name: feature request
description: a new feature!
body:
- type: markdown
attributes:
value: >
Please read this brief portion of documentation before going any
further:
https://flake8.pycqa.org/en/latest/internal/contributing.html#filing-a-bug
- type: markdown
attributes:
value: '**NOTE: flake8 is a linting framework and does not implement any checks**'
- type: markdown
attributes:
value: '**NOTE: if you ask about `pyproject.toml` your issue will be closed as a duplicate of [#234](https://github.com/PyCQA/flake8/issues/234)**'
- type: textarea
id: freeform
attributes:
label: describe the request
description: >
please describe your use case and why the current feature set does
not satisfy your needs
validations:
required: true

View file

@ -1,11 +0,0 @@
blank_issues_enabled: false
contact_links:
- name: problem with E___ or W___ codes
url: https://github.com/PyCQA/pycodestyle/issues
about: flake8 does not implement any checks, perhaps you want pycodestyle?
- name: problem with F___ codes
url: https://github.com/PyCQA/pyflakes/issues
about: flake8 does not implement any checks, perhaps you want pyflakes?
- name: problem with C___ codes
url: https://github.com/PyCQA/mccabe/issues
about: flake8 does not implement any checks, perhaps you want mccabe?

5
.github/SECURITY.md vendored
View file

@ -1,5 +0,0 @@
## security contact information
to report a security vulnerability, please use the
[Tidelift security contact](https://tidelift.com/security).
Tidelift will coordinate the fix and disclosure.

View file

@ -13,41 +13,41 @@ jobs:
include: include:
# linux # linux
- os: ubuntu-latest - os: ubuntu-latest
python: pypy-3.11 python: pypy-3.7
toxenv: py toxenv: py
- os: ubuntu-latest - os: ubuntu-latest
python: '3.10' python: 3.6
toxenv: py toxenv: py
- os: ubuntu-latest - os: ubuntu-latest
python: '3.11' python: 3.7
toxenv: py toxenv: py
- os: ubuntu-latest - os: ubuntu-latest
python: '3.12' python: 3.8
toxenv: py toxenv: py
- os: ubuntu-latest - os: ubuntu-latest
python: '3.13' python: 3.9
toxenv: py toxenv: py
- os: ubuntu-latest - os: ubuntu-latest
python: '3.14' python: '3.10.0-alpha - 3.10.999'
toxenv: py toxenv: py
# windows # windows
- os: windows-latest - os: windows-latest
python: '3.10' python: 3.6
toxenv: py toxenv: py
# misc # misc
- os: ubuntu-latest - os: ubuntu-latest
python: '3.10' python: 3.9
toxenv: docs toxenv: docs
- os: ubuntu-latest - os: ubuntu-latest
python: '3.10' python: 3.9
toxenv: linters toxenv: linters
- os: ubuntu-latest - os: ubuntu-latest
python: '3.10' python: 3.9
toxenv: dogfood toxenv: dogfood
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v2
- uses: actions/setup-python@v5 - uses: actions/setup-python@v2
with: with:
python-version: ${{ matrix.python }} python-version: ${{ matrix.python }}
- run: python -mpip install --upgrade setuptools pip tox virtualenv - run: python -mpip install --upgrade setuptools pip tox virtualenv

13
.hgignore Normal file
View file

@ -0,0 +1,13 @@
^bin/
^dist/
^include/
^lib/
^man/
^\.tox/
/__pycache__/
\.egg$
\.egg-info/
\.Python
\.orig$
\.pyc$
\.swp$

31
.hgtags Normal file
View file

@ -0,0 +1,31 @@
72a440d1fa2ddd69590cd71a51509a653fcdc235 0.1
4115a858709a3b625d3f7ebbc14dc77182052235 0.2
874fdfd29b704f1131420cc9f0d226c929fe255f 0.3
28547d53010248c55e77678087a7ef4195ab7a8a 0.4
ea4d5abfd0c02a96dd10cfe7ec3e7a76a080f38f 0.5
a914ae72b4eae171ca50b0ebda12765a68c7f744 0.6
405ffe2bfa854ba85428ee1627f5e0b01246ee22 0.7
b04dcad949573b1cef99392cf66caa5c55a0f930 0.8
6403df46028ffe602f5834743099c5160938aa6b 0.9
f49f8afba1da337074eb1dbb911ad0ec2a2c6199 1.0
33104f05ce6c430c27b6414d85e37a88468e6aff 1.1
575a782a8fb5d42e87954fd0a9253ffae6268023 1.2
575a782a8fb5d42e87954fd0a9253ffae6268023 1.2
de690f0eb4029802d6dc67ab7e1760a914d3eb0c 1.2
0407b6714ca42766edea6f3b17e183cac8fa596b 1.3
c522d468c5b86329a8b562ca7e392e544a45fffa 1.3.1
ff671fabec71e85d32395c35c40a125432859e49 1.5
30bf3a998b09303da032c03d61041180e6ba3d83 1.6
1184216fb3619680517d3f8386dc138ab2d5ee26 1.6.1
bca1826148f9ea22a89d9533d19a79ba6678293f 1.6.2
61b1bc18f258cf2647f4af29c3dfe48d268eeb0b 1.7.0
374b8e63d93b8743c3dad093bca449e01fdd287f 2.0
9b641817ffe6be1ff7d34711d203e27c8f3733f8 2.1.0
cacf6cc4290692456a9164d27b661acfcbdfcd12 2.2.0
fabf3cf87eafa8826aae91572e20e2e232d310ab 2.2.1
3f35951906d20667b9f4b67baff0299e637ce611 2.2.2
3f35951906d20667b9f4b67baff0299e637ce611 2.2.2
0000000000000000000000000000000000000000 2.2.2
0000000000000000000000000000000000000000 2.2.2
9ba7be5e4374230e00dc1203b5b45ab0c67ecc23 2.2.2
5dc9b776132736d7e11331aafd9d5c36faf6839b 2.2.3

View file

@ -1,44 +1,29 @@
exclude: ^tests/fixtures/
repos: repos:
- repo: https://github.com/asottile/add-trailing-comma
rev: v4.0.0
hooks:
- id: add-trailing-comma
- repo: https://github.com/pre-commit/pre-commit-hooks - repo: https://github.com/pre-commit/pre-commit-hooks
rev: v6.0.0 rev: v4.0.1
hooks: hooks:
- id: check-yaml - id: check-yaml
- id: debug-statements - id: debug-statements
- id: end-of-file-fixer - id: end-of-file-fixer
- id: trailing-whitespace - id: trailing-whitespace
exclude: ^tests/fixtures/ - repo: https://github.com/asottile/reorder_python_imports
- repo: https://github.com/asottile/setup-cfg-fmt rev: v2.6.0
rev: v3.2.0
hooks:
- id: setup-cfg-fmt
- repo: https://github.com/asottile/reorder-python-imports
rev: v3.16.0
hooks: hooks:
- id: reorder-python-imports - id: reorder-python-imports
args: [ args: [--application-directories, '.:src', --py36-plus]
--application-directories, '.:src', - repo: https://github.com/psf/black
--py310-plus, rev: 21.9b0
--add-import, 'from __future__ import annotations', hooks:
] - id: black
args: [--line-length=79]
- repo: https://github.com/asottile/pyupgrade - repo: https://github.com/asottile/pyupgrade
rev: v3.21.2 rev: v2.29.0
hooks: hooks:
- id: pyupgrade - id: pyupgrade
args: [--py310-plus] args: [--py36-plus]
- repo: https://github.com/hhatto/autopep8
rev: v2.3.2
hooks:
- id: autopep8
- repo: https://github.com/PyCQA/flake8
rev: 7.3.0
hooks:
- id: flake8
- repo: https://github.com/pre-commit/mirrors-mypy - repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.19.1 rev: v0.910
hooks: hooks:
- id: mypy - id: mypy
exclude: ^(docs/|example-plugin/) exclude: ^(docs/|example-plugin/)

View file

@ -365,3 +365,10 @@ ext-import-graph=
# Create a graph of internal dependencies in the given file (report RP0402 must # Create a graph of internal dependencies in the given file (report RP0402 must
# not be disabled) # not be disabled)
int-import-graph= int-import-graph=
[EXCEPTIONS]
# Exceptions that will emit a warning when being caught. Defaults to
# "Exception"
overgeneral-exceptions=Exception

View file

@ -1,12 +0,0 @@
version: 2
build:
os: ubuntu-22.04
tools:
python: "3.11"
python:
install:
- path: .
- requirements: docs/source/requirements.txt
sphinx:
configuration: docs/source/conf.py

View file

@ -1,3 +1,3 @@
Please refer to `Contributing to Flake8 Please refer to `Contributing to Flake8
<https://flake8.pycqa.org/en/latest/internal/contributing.html>`_ <http://flake8.pycqa.org/en/latest/internal/contributing.html>`_
on our website. on our website.

9
MANIFEST.in Normal file
View file

@ -0,0 +1,9 @@
include *.rst
include CONTRIBUTORS.txt
include LICENSE
include *.ini
global-exclude *.pyc
recursive-include docs *.rst *.py
recursive-include tests *.py *.ini *.rst *_diff
recursive-include src *.py
prune docs/build/

View file

@ -6,10 +6,6 @@
:target: https://results.pre-commit.ci/latest/github/PyCQA/flake8/main :target: https://results.pre-commit.ci/latest/github/PyCQA/flake8/main
:alt: pre-commit.ci status :alt: pre-commit.ci status
.. image:: https://img.shields.io/discord/825463413634891776.svg
:target: https://discord.gg/qYxpadCgkx
:alt: Discord
======== ========
Flake8 Flake8
======== ========
@ -41,14 +37,14 @@ Quickstart
========== ==========
See our `quickstart documentation See our `quickstart documentation
<https://flake8.pycqa.org/en/latest/index.html#quickstart>`_ for how to install <http://flake8.pycqa.org/en/latest/index.html#quickstart>`_ for how to install
and get started with Flake8. and get started with Flake8.
Frequently Asked Questions Frequently Asked Questions
========================== ==========================
Flake8 maintains an `FAQ <https://flake8.pycqa.org/en/latest/faq.html>`_ in its Flake8 maintains an `FAQ <http://flake8.pycqa.org/en/latest/faq.html>`_ in its
documentation. documentation.
@ -65,7 +61,7 @@ to suggest, the mailing list would be the best place for it.
Links Links
===== =====
* `Flake8 Documentation <https://flake8.pycqa.org/en/latest/>`_ * `Flake8 Documentation <http://flake8.pycqa.org/en/latest/>`_
* `GitHub Project <https://github.com/pycqa/flake8>`_ * `GitHub Project <https://github.com/pycqa/flake8>`_
@ -76,15 +72,14 @@ Links
<https://mail.python.org/mailman/listinfo/code-quality>`_ <https://mail.python.org/mailman/listinfo/code-quality>`_
* `Code of Conduct * `Code of Conduct
<https://flake8.pycqa.org/en/latest/internal/contributing.html#code-of-conduct>`_ <http://flake8.pycqa.org/en/latest/internal/contributing.html#code-of-conduct>`_
* `Getting Started Contributing * `Getting Started Contributing
<https://flake8.pycqa.org/en/latest/internal/contributing.html>`_ <http://flake8.pycqa.org/en/latest/internal/contributing.html>`_
Maintenance Maintenance
=========== ===========
Flake8 was created by Tarek Ziadé and is currently maintained by `anthony sottile Flake8 was created by Tarek Ziadé and is currently maintained by `Ian Cordasco
<https://github.com/sponsors/asottile>`_ and `Ian Cordasco <http://www.coglib.com/~icordasc/>`_
<https://www.coglib.com/~icordasc/>`_

View file

@ -1,98 +0,0 @@
#!/usr/bin/env python3
from __future__ import annotations
import inspect
import os.path
from collections.abc import Callable
from collections.abc import Generator
from typing import Any
from typing import NamedTuple
import pycodestyle
def _too_long(s: str) -> str:
if len(s) >= 80:
return f"{s} # noqa: E501"
else:
return s
class Call(NamedTuple):
name: str
is_generator: bool
params: tuple[str, ...]
def to_src(self) -> str:
params_s = ", ".join(self.params)
if self.is_generator:
return _too_long(f" yield from _{self.name}({params_s})")
else:
lines = (
_too_long(f" ret = _{self.name}({params_s})"),
" if ret is not None:",
" yield ret",
)
return "\n".join(lines)
@classmethod
def from_func(cls, func: Callable[..., Any]) -> Call:
spec = inspect.getfullargspec(func)
params = tuple(spec.args)
return cls(func.__name__, inspect.isgeneratorfunction(func), params)
def lines() -> Generator[str]:
logical = []
physical = []
logical = [
Call.from_func(check) for check in pycodestyle._checks["logical_line"]
]
physical = [
Call.from_func(check) for check in pycodestyle._checks["physical_line"]
]
assert not pycodestyle._checks["tree"]
yield f'"""Generated using ./bin/{os.path.basename(__file__)}."""'
yield "# fmt: off"
yield "from __future__ import annotations"
yield ""
yield "from collections.abc import Generator"
yield "from typing import Any"
yield ""
imports = sorted(call.name for call in logical + physical)
for name in imports:
yield _too_long(f"from pycodestyle import {name} as _{name}")
yield ""
yield ""
yield "def pycodestyle_logical("
logical_params = {param for call in logical for param in call.params}
for param in sorted(logical_params):
yield f" {param}: Any,"
yield ") -> Generator[tuple[int, str]]:"
yield ' """Run pycodestyle logical checks."""'
for call in sorted(logical):
yield call.to_src()
yield ""
yield ""
yield "def pycodestyle_physical("
physical_params = {param for call in physical for param in call.params}
for param in sorted(physical_params):
yield f" {param}: Any,"
yield ") -> Generator[tuple[int, str]]:"
yield ' """Run pycodestyle physical checks."""'
for call in sorted(physical):
yield call.to_src()
def main() -> int:
for line in lines():
print(line)
return 0
if __name__ == "__main__":
raise SystemExit(main())

42
codecov.yml Normal file
View file

@ -0,0 +1,42 @@
codecov:
branch: main
bot: null
coverage:
precision: 2
round: down
range: "60...100"
notify:
irc:
default:
server: chat.freenode.net
channel: '##python-code-quality'
branches: main
threshold: 2
message: null
status:
project:
default:
target: auto
threshold: null
branches: null
patch:
default:
target: auto
branches: null
changes:
default:
branches: null
ignore: null
fixes:
- .tox
comment:
layout: "header, diff, changes, sunburst, uncovered, tree"
branches: null
behavior: default

View file

@ -10,18 +10,18 @@
# #
# All configuration values have a default; values that are commented out # All configuration values have a default; values that are commented out
# serve to show the default. # serve to show the default.
import os
import sys
# If extensions (or modules to document with autodoc) are in another directory, # If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the # add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here. # documentation root, use os.path.abspath to make it absolute, like shown here.
# sys.path.insert(0, os.path.abspath('.')) # sys.path.insert(0, os.path.abspath('.'))
from __future__ import annotations
import flake8
# -- General configuration ------------------------------------------------ # -- General configuration ------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here. # If your documentation needs a minimal Sphinx version, state it here.
needs_sphinx = "2.1" needs_sphinx = "1.3"
# Add any Sphinx extension module names here, as strings. They can be # Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
@ -34,7 +34,7 @@ extensions = [
"sphinx.ext.todo", "sphinx.ext.todo",
"sphinx.ext.coverage", "sphinx.ext.coverage",
"sphinx.ext.viewcode", "sphinx.ext.viewcode",
"sphinx_prompt", "sphinx-prompt",
] ]
# Add any paths that contain templates here, relative to this directory. # Add any paths that contain templates here, relative to this directory.
@ -56,6 +56,8 @@ project = "flake8"
copyright = "2016, Ian Stapleton Cordasco" copyright = "2016, Ian Stapleton Cordasco"
author = "Ian Stapleton Cordasco" author = "Ian Stapleton Cordasco"
import flake8
# The version info for the project you're documenting, acts as replacement for # The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the # |version| and |release|, also used in various other places throughout the
# built documents. # built documents.
@ -74,7 +76,7 @@ rst_epilog = """
# #
# This is also used if you do content translation via gettext catalogs. # This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases. # Usually you set "language" from the command line for these cases.
language = "en" language = None
# There are two options for replacing |today|: either, you set today to some # There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used: # non-false value, then it is used:
@ -216,13 +218,13 @@ htmlhelp_basename = "flake8doc"
latex_elements = { latex_elements = {
# The paper size ('letterpaper' or 'a4paper'). # The paper size ('letterpaper' or 'a4paper').
# 'papersize': 'letterpaper', #'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt'). # The font size ('10pt', '11pt' or '12pt').
# 'pointsize': '10pt', #'pointsize': '10pt',
# Additional stuff for the LaTeX preamble. # Additional stuff for the LaTeX preamble.
# 'preamble': '', #'preamble': '',
# Latex figure (float) alignment # Latex figure (float) alignment
# 'figure_align': 'htbp', #'figure_align': 'htbp',
} }
# Grouping the document tree into LaTeX files. List of tuples # Grouping the document tree into LaTeX files. List of tuples
@ -261,6 +263,12 @@ latex_documents = [
# -- Options for manual page output --------------------------------------- # -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
("manpage", "flake8", "Flake8 Command Line Documentation", [author], 1)
]
# If true, show URL addresses after external links. # If true, show URL addresses after external links.
# man_show_urls = False # man_show_urls = False
@ -296,15 +304,9 @@ texinfo_documents = [
# Example configuration for intersphinx: refer to the Python standard library. # Example configuration for intersphinx: refer to the Python standard library.
intersphinx_mapping = { intersphinx_mapping = {"python": ("https://docs.python.org/3/", None)}
"python": ("https://docs.python.org/3/", None),
"packaging": ("https://packaging.python.org/en/latest/", None),
"setuptools": ("https://setuptools.pypa.io/en/latest/", None),
}
extlinks = { extlinks = {
"issue": ("https://github.com/pycqa/flake8/issues/%s", "#%s"), "issue": ("https://github.com/pycqa/flake8/issues/%s", "#"),
"pull": ("https://github.com/pycqa/flake8/pull/%s", "#%s"), "pull": ("https://github.com/pycqa/flake8/pull/%s", "#"),
} }
autodoc_typehints = "description"

View file

@ -60,11 +60,11 @@ If you only want to see the instances of a specific warning or error, you can
flake8 --select E123,W503 path/to/code/ flake8 --select E123,W503 path/to/code/
Alternatively, if you want to add a specific warning or error to *ignore*: Alternatively, if you want to *ignore* only one specific warning or error:
.. code:: .. code::
flake8 --extend-ignore E203,W234 path/to/code/ flake8 --ignore E24,W504 path/to/code/
Please read our user guide for more information about how to use and configure Please read our user guide for more information about how to use and configure
|Flake8|. |Flake8|.
@ -89,6 +89,7 @@ and how to specify them on the command-line or in configuration files.
:maxdepth: 2 :maxdepth: 2
user/index user/index
Flake8 man page <manpage>
Plugin Developer Guide Plugin Developer Guide
====================== ======================

View file

@ -57,6 +57,8 @@ Utility Functions
.. autofunction:: flake8.processor.is_multiline_string .. autofunction:: flake8.processor.is_multiline_string
.. autofunction:: flake8.processor.log_token
.. autofunction:: flake8.processor.mutate_string .. autofunction:: flake8.processor.mutate_string
.. autofunction:: flake8.processor.token_is_newline .. autofunction:: flake8.processor.token_is_newline

View file

@ -197,7 +197,7 @@ delivered.
.. links .. links
.. _Python Code Quality Authority's Code of Conduct: .. _Python Code Quality Authority's Code of Conduct:
https://meta.pycqa.org/code-of-conduct.html http://meta.pycqa.org/en/latest/code-of-conduct.html
.. _tox: .. _tox:
https://tox.readthedocs.io/ https://tox.readthedocs.io/

View file

@ -41,7 +41,7 @@ three new parameters:
The last two are not specifically for configuration file handling, but they The last two are not specifically for configuration file handling, but they
do improve that dramatically. We found that there were options that, when do improve that dramatically. We found that there were options that, when
specified in a configuration file, often necessitated being split across specified in a configuration file, often necessitated being spit
multiple lines and those options were almost always comma-separated. For multiple lines and those options were almost always comma-separated. For
example, let's consider a user's list of ignored error codes for a project: example, let's consider a user's list of ignored error codes for a project:
@ -157,22 +157,42 @@ problems with pep8's 1.6 series. As such, |Flake8| has separated out
discovery, management, and merging into a module to make reasoning about each discovery, management, and merging into a module to make reasoning about each
of these pieces easier and more explicit (as well as easier to test). of these pieces easier and more explicit (as well as easier to test).
Configuration file discovery and raw ini reading is managed by Configuration file discovery is managed by the
:func:`~flake8.options.config.load_config`. This produces a loaded :class:`~flake8.options.config.ConfigFileFinder` object. This object needs to
:class:`~configparser.RawConfigParser` and a config directory (which will be know information about the program's name, any extra arguments passed to it,
used later to normalize paths). and any configuration files that should be appended to the list of discovered
files. It provides methods for finding the files and similar methods for
parsing those fles. For example, it provides
:meth:`~flake8.options.config.ConfigFileFinder.local_config_files` to find
known local config files (and append the extra configuration files) and it
also provides :meth:`~flake8.options.config.ConfigFileFinder.local_configs`
to parse those configuration files.
Next, :func:`~flake8.options.config.parse_config` parses options using the .. note:: ``local_config_files`` also filters out non-existent files.
types in the ``OptionManager``.
Most of this is done in :func:`~flake8.options.aggregator.aggregate_options`. Configuration file merging and managemnt is controlled by the
:class:`~flake8.options.config.ConfigParser`. This requires the instance
of :class:`~flake8.options.manager.OptionManager` that the program is using,
the list of appended config files, and the list of extra arguments. This
object is currently the sole user of the
:class:`~flake8.options.config.ConfigFileFinder` object. It appropriately
initializes the object and uses it in each of
- :meth:`~flake8.options.config.ConfigParser.parse_cli_config`
- :meth:`~flake8.options.config.ConfigParser.parse_local_config`
Finally, :meth:`~flake8.options.config.ConfigParser.parse` returns the
appropriate configuration dictionary for this execution of |Flake8|. The
main usage of the ``ConfigParser`` is in
:func:`~flake8.options.aggregator.aggregate_options`.
Aggregating Configuration File and Command Line Arguments Aggregating Configuration File and Command Line Arguments
--------------------------------------------------------- ---------------------------------------------------------
:func:`~flake8.options.aggregator.aggregate_options` accepts an instance of :func:`~flake8.options.aggregator.aggregate_options` accepts an instance of
:class:`~flake8.options.manager.OptionManager` and does the work to parse the :class:`~flake8.options.manager.OptionManager` and does the work to parse the
command-line arguments. command-line arguments passed by the user necessary for creating an instance
of :class:`~flake8.options.config.ConfigParser`.
After parsing the configuration file, we determine the default ignore list. We After parsing the configuration file, we determine the default ignore list. We
use the defaults from the OptionManager and update those with the parsed use the defaults from the OptionManager and update those with the parsed
@ -196,6 +216,10 @@ API Documentation
:members: :members:
:special-members: :special-members:
.. autofunction:: flake8.options.config.load_config .. autoclass:: flake8.options.config.ConfigFileFinder
:members:
:special-members:
.. autofunction:: flake8.options.config.parse_config .. autoclass:: flake8.options.config.ConfigParser
:members:
:special-members:

View file

@ -11,6 +11,44 @@ new checks. It now supports:
- alternative report formatters - alternative report formatters
To facilitate this, |Flake8| needed a more mature way of managing plugins.
Thus, we developed the |PluginManager| which accepts a namespace and will load
the plugins for that namespace. A |PluginManager| creates and manages many
|Plugin| instances.
A |Plugin| lazily loads the underlying entry-point provided by setuptools.
The entry-point will be loaded either by calling
:meth:`~flake8.plugins.manager.Plugin.load_plugin` or accessing the ``plugin``
attribute. We also use this abstraction to retrieve options that the plugin
wishes to register and parse.
The only public method the |PluginManager| provides is
:meth:`~flake8.plugins.manager.PluginManager.map`. This will accept a function
(or other callable) and call it with each plugin as the first parameter.
We build atop the |PluginManager| with the |PTM|. It is expected that users of
the |PTM| will subclass it and specify the ``namespace``, e.g.,
.. code-block:: python
class ExamplePluginType(flake8.plugin.manager.PluginTypeManager):
namespace = 'example-plugins'
This provides a few extra methods via the |PluginManager|'s ``map`` method.
Finally, we create two classes of plugins:
- :class:`~flake8.plugins.manager.Checkers`
- :class:`~flake8.plugins.manager.ReportFormatters`
These are used to interact with each of the types of plugins individually.
.. note::
Our inspiration for our plugin handling comes from the author's extensive
experience with ``stevedore``.
Default Plugins Default Plugins
--------------- ---------------
@ -18,26 +56,40 @@ Finally, |Flake8| has always provided its own plugin shim for Pyflakes. As
part of that we carry our own shim in-tree and now store that in part of that we carry our own shim in-tree and now store that in
:mod:`flake8.plugins.pyflakes`. :mod:`flake8.plugins.pyflakes`.
|Flake8| also registers plugins for pycodestyle. Each check in pycodestyle |Flake8| also registers plugins for pep8. Each check in pep8 requires
requires different parameters and it cannot easily be shimmed together like different parameters and it cannot easily be shimmed together like Pyflakes
Pyflakes was. As such, plugins have a concept of a "group". If you look at our was. As such, plugins have a concept of a "group". If you look at our
:file:`setup.py` you will see that we register pycodestyle checks roughly like :file:`setup.py` you will see that we register pep8 checks roughly like so:
so:
.. code:: .. code::
pycodestyle.<check-name> = pycodestyle:<check-name> pep8.<check-name> = pep8:<check-name>
We do this to identify that ``<check-name>>`` is part of a group. This also We do this to identify that ``<check-name>>`` is part of a group. This also
enables us to special-case how we handle reporting those checks. Instead of enables us to special-case how we handle reporting those checks. Instead of
reporting each check in the ``--version`` output, we only report reporting each check in the ``--version`` output, we report ``pep8`` and check
``pycodestyle`` once. ``pep8`` the module for a ``__version__`` attribute. We only report it once
to avoid confusing users.
API Documentation API Documentation
----------------- -----------------
.. autofunction:: flake8.plugins.finder.parse_plugin_options .. autoclass:: flake8.plugins.manager.PluginManager
:members:
:special-members: __init__
.. autofunction:: flake8.plugins.finder.find_plugins .. autoclass:: flake8.plugins.manager.Plugin
:members:
:special-members: __init__
.. autofunction:: flake8.plugins.finder.load_plugins .. autoclass:: flake8.plugins.manager.PluginTypeManager
:members:
.. autoclass:: flake8.plugins.manager.Checkers
:members:
.. autoclass:: flake8.plugins.manager.ReportFormatters
.. |PluginManager| replace:: :class:`~flake8.plugins.manager.PluginManager`
.. |Plugin| replace:: :class:`~flake8.plugins.manager.Plugin`
.. |PTM| replace:: :class:`~flake8.plugins.manager.PluginTypeManager`

View file

@ -28,9 +28,9 @@ Historically, |Flake8| has generated major releases for:
- Unvendoring dependencies (2.0) - Unvendoring dependencies (2.0)
- Large scale refactoring (2.0, 3.0, 5.0, 6.0) - Large scale refactoring (2.0, 3.0)
- Subtly breaking CLI changes (3.0, 4.0, 5.0, 6.0, 7.0) - Subtly breaking CLI changes (3.0, 4.0)
- Breaking changes to its plugin interface (3.0) - Breaking changes to its plugin interface (3.0)
@ -81,9 +81,9 @@ for users.
Before releasing, the following tox test environments must pass: Before releasing, the following tox test environments must pass:
- Python 3.9 (a.k.a., ``tox -e py39``) - Python 3.6 (a.k.a., ``tox -e py36``)
- Python 3.13 (a.k.a., ``tox -e py313``) - Python 3.7 (a.k.a., ``tox -e py37``)
- PyPy 3 (a.k.a., ``tox -e pypy3``) - PyPy 3 (a.k.a., ``tox -e pypy3``)

View file

@ -54,15 +54,63 @@ normalized path.
This function retrieves and caches the value provided on ``sys.stdin``. This This function retrieves and caches the value provided on ``sys.stdin``. This
allows plugins to use this to retrieve ``stdin`` if necessary. allows plugins to use this to retrieve ``stdin`` if necessary.
.. autofunction:: flake8.utils.is_windows
This provides a convenient and explicitly named function that checks if we are
currently running on a Windows (or ``nt``) operating system.
.. autofunction:: flake8.utils.is_using_stdin .. autofunction:: flake8.utils.is_using_stdin
Another helpful function that is named only to be explicit given it is a very Another helpful function that is named only to be explicit given it is a very
trivial check, this checks if the user specified ``-`` in their arguments to trivial check, this checks if the user specified ``-`` in their arguments to
|Flake8| to indicate we should read from stdin. |Flake8| to indicate we should read from stdin.
.. autofunction:: flake8.utils.filenames_from
When provided an argument to |Flake8|, we need to be able to traverse
directories in a convenient manner. For example, if someone runs
.. code::
$ flake8 flake8/
Then they want us to check all of the files in the directory ``flake8/``. This
function will handle that while also handling the case where they specify a
file like:
.. code::
$ flake8 flake8/__init__.py
.. autofunction:: flake8.utils.fnmatch .. autofunction:: flake8.utils.fnmatch
The standard library's :func:`fnmatch.fnmatch` is excellent at deciding if a The standard library's :func:`fnmatch.fnmatch` is excellent at deciding if a
filename matches a single pattern. In our use case, however, we typically have filename matches a single pattern. In our use case, however, we typically have
a list of patterns and want to know if the filename matches any of them. This a list of patterns and want to know if the filename matches any of them. This
function abstracts that logic away with a little extra logic. function abstracts that logic away with a little extra logic.
.. autofunction:: flake8.utils.parameters_for
|Flake8| analyzes the parameters to plugins to determine what input they are
expecting. Plugins may expect one of the following:
- ``physical_line`` to receive the line as it appears in the file
- ``logical_line`` to receive the logical line (not as it appears in the file)
- ``tree`` to receive the abstract syntax tree (AST) for the file
We also analyze the rest of the parameters to provide more detail to the
plugin. This function will return the parameters in a consistent way across
versions of Python and will handle both classes and functions that are used as
plugins. Further, if the plugin is a class, it will strip the ``self``
argument so we can check the parameters of the plugin consistently.
.. autofunction:: flake8.utils.parse_unified_diff
To handle usage of :option:`flake8 --diff`, |Flake8| needs to be able
to parse the name of the files in the diff as well as the ranges indicated the
sections that have been changed. This function either accepts the diff as an
argument or reads the diff from standard-in. It then returns a dictionary with
filenames as the keys and sets of line numbers as the value.

View file

@ -34,21 +34,23 @@ accepts as well as what it returns.
.. code-block:: python .. code-block:: python
# src/flake8/main/git.py # src/flake8/main/git.py
def hook(lazy: bool = False, strict: bool = False) -> int: def hook(lazy=False, strict=False):
"""Execute Flake8 on the files in git's index. """Execute Flake8 on the files in git's index.
Determine which files are about to be committed and run Flake8 over them Determine which files are about to be committed and run Flake8 over them
to check for violations. to check for violations.
:param lazy: :param bool lazy:
Find files not added to the index prior to committing. This is useful Find files not added to the index prior to committing. This is useful
if you frequently use ``git commit -a`` for example. This defaults to if you frequently use ``git commit -a`` for example. This defaults to
False since it will otherwise include files not in the index. False since it will otherwise include files not in the index.
:param strict: :param bool strict:
If True, return the total number of errors/violations found by Flake8. If True, return the total number of errors/violations found by Flake8.
This will cause the hook to fail. This will cause the hook to fail.
:returns: :returns:
Total number of errors found during the run. Total number of errors found during the run.
:rtype:
int
""" """
# NOTE(sigmavirus24): Delay import of application until we need it. # NOTE(sigmavirus24): Delay import of application until we need it.
from flake8.main import application from flake8.main import application
@ -64,9 +66,39 @@ accepts as well as what it returns.
return app.result_count return app.result_count
return 0 return 0
Note that we begin the description of the parameter on a new-line and Note that because the parameters ``hook`` and ``strict`` are simply boolean
parameters, we inline the type declaration for those parameters, e.g.,
.. code-block:: restructuredtext
:param bool lazy:
Also note that we begin the description of the parameter on a new-line and
indented 4 spaces. indented 4 spaces.
On the other hand, we also separate the parameter type declaration in some
places where the name is a little longer, e.g.,
.. code-block:: python
# src/flake8/formatting/base.py
def format(self, error):
"""Format an error reported by Flake8.
This method **must** be implemented by subclasses.
:param error:
This will be an instance of :class:`~flake8.style_guide.Error`.
:type error:
flake8.style_guide.Error
:returns:
The formatted error string.
:rtype:
str
"""
Here we've separated ``:param error:`` and ``:type error:``.
Following the above examples and guidelines should help you write doc-strings Following the above examples and guidelines should help you write doc-strings
that are stylistically correct for |Flake8|. that are stylistically correct for |Flake8|.

147
docs/source/manpage.rst Normal file
View file

@ -0,0 +1,147 @@
========
flake8
========
SYNOPSIS
========
.. code::
flake8 [options] [<path> <path> ...]
flake8 --help
DESCRIPTION
===========
``flake8`` is a command-line utility for enforcing style consistency across
Python projects. By default it includes lint checks provided by the PyFlakes
project, PEP-0008 inspired style checks provided by the PyCodeStyle project,
and McCabe complexity checking provided by the McCabe project. It will also
run third-party extensions if they are found and installed.
OPTIONS
=======
It is important to note that third-party extensions may add options which are
not represented here. To see all options available in your installation, run::
flake8 --help
All options available as of Flake8 3.1.0::
--version show program's version number and exit
-h, --help show this help message and exit
-v, --verbose Print more information about what is happening in
flake8. This option is repeatable and will increase
verbosity each time it is repeated.
-q, --quiet Report only file names, or nothing. This option is
repeatable.
--count Print total number of errors and warnings to standard
error and set the exit code to 1 if total is not
empty.
--diff Report changes only within line number ranges in the
unified diff provided on standard in by the user.
--exclude=patterns Comma-separated list of files or directories to
exclude. (Default:
.svn,CVS,.bzr,.hg,.git,__pycache__,.tox,.eggs,*.egg)
--filename=patterns Only check for filenames matching the patterns in this
comma-separated list. (Default: *.py)
--stdin-display-name=STDIN_DISPLAY_NAME
The name used when reporting errors from code passed
via stdin. This is useful for editors piping the file
contents to flake8. (Default: stdin)
--format=format Format errors according to the chosen formatter.
--hang-closing Hang closing bracket instead of matching indentation
of opening bracket's line.
--ignore=errors Comma-separated list of errors and warnings to ignore
(or skip). For example, ``--ignore=E4,E51,W234``.
(Default: E121,E123,E126,E226,E24,E704,W503,W504)
--max-line-length=n Maximum allowed line length for the entirety of this
run. (Default: 79)
--select=errors Comma-separated list of errors and warnings to enable.
For example, ``--select=E4,E51,W234``. (Default:
E,F,W,C90)
--disable-noqa Disable the effect of "# noqa". This will report
errors on lines with "# noqa" at the end.
--show-source Show the source generate each error or warning.
--statistics Count errors and warnings.
--enable-extensions=ENABLE_EXTENSIONS
Enable plugins and extensions that are otherwise
disabled by default
--exit-zero Exit with status code "0" even if there are errors.
-j JOBS, --jobs=JOBS Number of subprocesses to use to run checks in
parallel. This is ignored on Windows. The default,
"auto", will auto-detect the number of processors
available to use. (Default: auto)
--output-file=OUTPUT_FILE
Redirect report to a file.
--tee Write to stdout and output-file.
--append-config=APPEND_CONFIG
Provide extra config files to parse in addition to the
files found by Flake8 by default. These files are the
last ones read and so they take the highest precedence
when multiple files provide the same option.
--config=CONFIG Path to the config file that will be the authoritative
config source. This will cause Flake8 to ignore all
other configuration files.
--isolated Ignore all configuration files.
--benchmark Print benchmark information about this run of Flake8
--bug-report Print information necessary when preparing a bug
report
--builtins=BUILTINS define more built-ins, comma separated
--doctests check syntax of the doctests
--include-in-doctest=INCLUDE_IN_DOCTEST
Run doctests only on these files
--exclude-from-doctest=EXCLUDE_FROM_DOCTEST
Skip these files when running doctests
--max-complexity=MAX_COMPLEXITY
McCabe complexity threshold
EXAMPLES
========
Simply running flake8 against the current directory::
flake8
flake8 .
Running flake8 against a specific path::
flake8 path/to/file.py
Ignoring violations from flake8::
flake8 --ignore E101
flake8 --ignore E1,E202
Only report certain violations::
flake8 --select E101
flake8 --select E2,E742
Analyzing only a diff::
git diff -U0 | flake8 --diff -
Generate information for a bug report::
flake8 --bug-report
SEE ALSO
========
Flake8 documentation: http://flake8.pycqa.org
Flake8 Options and Examples: http://flake8.pycqa.org/en/latest/user/options.html
PyCodeStyle documentation: http://pycodestyle.pycqa.org
PyFlakes: https://github.com/pycqa/pyflakes
McCabe: https://github.com/pycqa/mccabe
BUGS
====
Please report all bugs to https://github.com/pycqa/flake8

View file

@ -0,0 +1,187 @@
====================================
Writing Plugins For Flake8 2 and 3
====================================
Plugins have existed for |Flake8| 2.x for a few years. There are a number of
these on PyPI already. While it did not seem reasonable for |Flake8| to attempt
to provide a backwards compatible shim for them, we did decide to try to
document the easiest way to write a plugin that's compatible across both
versions.
.. note::
If your plugin does not register options, it *should* Just Work.
The **only two** breaking changes in |Flake8| 3.0 is the fact that we no
longer check the option parser for a list of strings to parse from a config
file and we no longer patch pep8 or pycodestyle's ``stdin_get_value``
functions. On |Flake8| 2.x, to have an option parsed from the configuration
files that |Flake8| finds and parses you would have to do something like:
.. code-block:: python
parser.add_option('-X', '--example-flag', type='string',
help='...')
parser.config_options.append('example-flag')
For |Flake8| 3.0, we have added *three* arguments to the
:meth:`~flake8.options.manager.OptionManager.add_option` method you will call
on the parser you receive:
- ``parse_from_config`` which expects ``True`` or ``False``
When ``True``, |Flake8| will parse the option from the config files |Flake8|
finds.
- ``comma_separated_list`` which expects ``True`` or ``False``
When ``True``, |Flake8| will split the string intelligently and handle
extra whitespace. The parsed value will be a list.
- ``normalize_paths`` which expects ``True`` or ``False``
When ``True``, |Flake8| will:
* remove trailing path separators (i.e., ``os.path.sep``)
* return the absolute path for values that have the separator in them
All three of these options can be combined or used separately.
Parsing Options from Configuration Files
========================================
The example from |Flake8| 2.x now looks like:
.. code-block:: python
parser.add_option('-X', '--example-flag', type='string',
parse_from_config=True,
help='...')
Parsing Comma-Separated Lists
=============================
Now let's imagine that the option we want to add is expecting a comma-separatd
list of values from the user (e.g., ``--select E123,W503,F405``). |Flake8| 2.x
often forced users to parse these lists themselves since pep8 special-cased
certain flags and left others on their own. |Flake8| 3.0 adds
``comma_separated_list`` so that the parsed option is already a list for
plugin authors. When combined with ``parse_from_config`` this means that users
can also do something like:
.. code-block:: ini
example-flag =
first,
second,
third,
fourth,
fifth
And |Flake8| will just return the list:
.. code-block:: python
["first", "second", "third", "fourth", "fifth"]
Normalizing Values that Are Paths
=================================
Finally, let's imagine that our new option wants a path or list of paths. To
ensure that these paths are semi-normalized (the way |Flake8| 2.x used to
work) we need only pass ``normalize_paths=True``. If you have specified
``comma_separated_list=True`` then this will parse the value as a list of
paths that have been normalized. Otherwise, this will parse the value
as a single path.
Option Handling on Flake8 2 and 3
=================================
To ease the transition, the |Flake8| maintainers have released
`flake8-polyfill`_. |polyfill| provides a convenience function to help users
transition between Flake8 2 and 3 without issue. For example, if your plugin
has to work on Flake8 2.x and 3.x but you want to take advantage of some of
the new options to ``add_option``, you can do
.. code-block:: python
from flake8_polyfill import options
class MyPlugin(object):
@classmethod
def add_options(cls, parser):
options.register(
parser,
'--application-names', default='', type='string',
help='Names of the applications to be checked.',
parse_from_config=True,
comma_separated_list=True,
)
options.register(
parser,
'--style-name', default='', type='string',
help='The name of the style convention you want to use',
parse_from_config=True,
)
options.register(
parser,
'--application-paths', default='', type='string',
help='Locations of the application code',
parse_from_config=True,
comma_separated_list=True,
normalize_paths=True,
)
@classmethod
def parse_options(cls, parsed_options):
cls.application_names = parsed_options.application_names
cls.style_name = parsed_options.style_name
cls.application_paths = parsed_options.application_paths
|polyfill| will handle these extra options using *callbacks* to the option
parser. The project has direct replications of the functions that |Flake8|
uses to provide the same functionality. This means that the values you receive
should be identically parsed whether you're using Flake8 2.x or 3.x.
.. autofunction:: flake8_polyfill.options.register
Standard In Handling on Flake8 2.5, 2.6, and 3
==============================================
After releasing |Flake8| 2.6, handling standard-in became a bit trickier for
some plugins. |Flake8| 2.5 and earlier had started monkey-patching pep8's
``stdin_get_value`` function. 2.6 switched to pycodestyle and only
monkey-patched that. 3.0 has its own internal implementation and uses that but
does not directly provide anything for plugins using pep8 and pycodestyle's
``stdin_get_value`` function. |polyfill| provides this functionality for
plugin developers via its :mod:`flake8_polyfill.stdin` module.
If a plugin needs to read the content from stdin, it can do the following:
.. code-block:: python
from flake8_polyfill import stdin
stdin.monkey_patch('pep8') # To monkey-patch only pep8
stdin.monkey_patch('pycodestyle') # To monkey-patch only pycodestyle
stdin.monkey_patch('all') # To monkey-patch both pep8 and pycodestyle
Further, when using ``all``, |polyfill| does not require both packages to be
installed but will attempt to monkey-patch both and will silently ignore the
fact that pep8 or pycodestyle is not installed.
.. autofunction:: flake8_polyfill.stdin.monkey_patch
.. links
.. _flake8-polyfill: https://pypi.org/project/flake8-polyfill/
.. |polyfill| replace:: ``flake8-polyfill``

View file

@ -30,8 +30,7 @@ To get started writing a |Flake8| :term:`plugin` you first need:
Once you've gathered these things, you can get started. Once you've gathered these things, you can get started.
All plugins for |Flake8| must be registered via All plugins for |Flake8| must be registered via `entry points`_. In this
:external+packaging:doc:`entry points<specifications/entry-points>`. In this
section we cover: section we cover:
- How to register your plugin so |Flake8| can find it - How to register your plugin so |Flake8| can find it
@ -55,8 +54,6 @@ Here's a tutorial which goes over building an ast checking plugin from scratch:
<iframe src="https://www.youtube.com/embed/ot5Z4KQPBL8" frameborder="0" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe> <iframe src="https://www.youtube.com/embed/ot5Z4KQPBL8" frameborder="0" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe>
</div> </div>
Detailed Plugin Development Documentation
=========================================
.. toctree:: .. toctree::
:caption: Plugin Developer Documentation :caption: Plugin Developer Documentation
@ -65,3 +62,8 @@ Detailed Plugin Development Documentation
registering-plugins registering-plugins
plugin-parameters plugin-parameters
formatters formatters
cross-compatibility
.. _entry points:
https://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points

View file

@ -22,8 +22,8 @@ Indicating Desired Data
======================= =======================
|Flake8| inspects the plugin's signature to determine what parameters it |Flake8| inspects the plugin's signature to determine what parameters it
expects using :func:`flake8.plugins.finder._parameters_for`. expects using :func:`flake8.utils.parameters_for`.
:attr:`flake8.plugins.finder.LoadedPlugin.parameters` caches the values so that :attr:`flake8.plugins.manager.Plugin.parameters` caches the values so that
each plugin makes that fairly expensive call once per plugin. When processing each plugin makes that fairly expensive call once per plugin. When processing
a file, a plugin can ask for any of the following: a file, a plugin can ask for any of the following:

View file

@ -12,17 +12,16 @@ To register any kind of plugin with |Flake8|, you need:
#. A name for your plugin that will (ideally) be unique. #. A name for your plugin that will (ideally) be unique.
|Flake8| relies on functionality provided by build tools called #. A somewhat recent version of setuptools (newer than 0.7.0 but preferably as
:external+packaging:doc:`entry points<specifications/entry-points>`. These recent as you can attain).
allow any package to register a plugin with |Flake8| via that package's
metadata. |Flake8| relies on functionality provided by setuptools called
`Entry Points`_. These allow any package to register a plugin with |Flake8|
via that package's ``setup.py`` file.
Let's presume that we already have our plugin written and it's in a module Let's presume that we already have our plugin written and it's in a module
called ``flake8_example``. We will also assume ``setuptools`` is used as a called ``flake8_example``. We might have a ``setup.py`` that looks something
:external+packaging:term:`Build Backend`, but be aware that most backends like:
support entry points.
We might have a ``setup.py`` that looks something like:
.. code-block:: python .. code-block:: python
@ -113,17 +112,11 @@ look like::
X101 = flake8_example:ExamplePlugin X101 = flake8_example:ExamplePlugin
In the above case, the entry-point name and the error code produced by your
plugin are the same.
If your plugin reports several error codes that all start with ``X10``, then If your plugin reports several error codes that all start with ``X10``, then
it would look like:: it would look like::
X10 = flake8_example:ExamplePlugin X10 = flake8_example:ExamplePlugin
In this case as well as the following case, your entry-point name acts as
a prefix to the error codes produced by your plugin.
If all of your plugin's error codes start with ``X1`` then it would look If all of your plugin's error codes start with ``X1`` then it would look
like:: like::
@ -137,21 +130,9 @@ in the users environment. Selecting an entry point that is already used can
cause plugins to be deactivated without warning! cause plugins to be deactivated without warning!
**Please Note:** Your entry point does not need to be exactly 4 characters **Please Note:** Your entry point does not need to be exactly 4 characters
as of |Flake8| 3.0. Single letter entry point prefixes (such as the as of |Flake8| 3.0. *Consider using an entry point with 3 letters followed
'X' in the examples above) have caused issues in the past. As such, by 3 numbers (i.e.* ``ABC123`` *).*
please consider using a 2 or 3 character entry point prefix,
i.e., ``ABC`` is better than ``A`` but ``ABCD`` is invalid.
*A 3 letters entry point prefix followed by 3 numbers (i.e.* ``ABC123`` *)
is currently the longest allowed entry point name.*
.. _off-by-default:
If your plugin is intended to be opt-in, it can set the attribute .. _Entry Points:
``off_by_default = True``. Users of your plugin will then need to utilize https://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points
:ref:`enable-extensions<option-enable-extensions>` with your plugin's entry
point.
.. seealso::
The :external+setuptools:doc:`setuptools user guide <userguide/entry_point>`
about entry points.

View file

@ -1,7 +1,7 @@
3.0.0 -- 2016-07-25 3.0.0 -- 2016-07-25
------------------- -------------------
- Rewrite our documentation from scratch! (https://flake8.pycqa.org) - Rewrite our documentation from scratch! (http://flake8.pycqa.org)
- Drop explicit support for Pythons 2.6, 3.2, and 3.3. - Drop explicit support for Pythons 2.6, 3.2, and 3.3.

View file

@ -1,76 +0,0 @@
5.0.0 -- 2022-07-30
-------------------
You can view the `5.0.0 milestone`_ on GitHub for more details.
Backwards Incompatible Changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Remove ``indent_size_str`` (See also :pull:`1411`).
- Remove some dead code (See also :pull:`1453`, :pull:`1540`, :pull:`1541`).
- Missing explicitly-specified configuration is now an error (See also
:issue:`1497`, :pull:`1498`).
- Always read configuration files as UTF-8 (See also :issue:`1532`,
:pull:`1533`).
- Remove manpage from docs -- use ``help2man`` or related tools instead (See
also :pull:`1557`).
- Forbid invalid plugin codes (See also :issue:`325`, :pull:`1579`).
Deprecations
~~~~~~~~~~~~
- Deprecate ``--diff`` option (See also :issue:`1389`, :pull:`1441`).
New Dependency Information
~~~~~~~~~~~~~~~~~~~~~~~~~~
- pycodestyle has been updated to >= 2.9.0, < 2.10.0 (See also :pull:`1626`).
- Pyflakes has been updated to >= 2.5.0, < 2.6.0 (See also :pull:`1625`).
- mccabe has been updated to >= 0.7.0, < 0.8.0 (See also :pull:`1542`).
Features
~~~~~~~~
- Add colors to output, configurable via ``--color`` (See also :issue:`1379`,
:pull:`1440`).
- Add ``.nox`` to the default exclude list (See also :issue:`1442`,
:pull:`1443`).
- Don't consider a config file which does not contain flake8 settings (See
also :issue:`199`, :pull:`1472`).
- Duplicate ``local-plugins`` names are now allowed (See also :issue:`362`,
:pull:`1504`).
- Consider ``.`` to be a path in config files (See also :issue:`1494`,
:pull:`1508`)
- Add ``--require-plugins`` option taking distribution names (See also
:issue:`283`, :pull:`1535`).
- Improve performance by removing debug logs (See also :pull:`1537`,
:pull:`1544`).
- Include failing file path in plugin execution error (See also :issue:`265`,
:pull:`1543`).
- Improve performance by pre-generating a ``pycodestyle`` plugin (See also
:pull:`1545`).
- Properly differentiate between explicitly ignored / selected and default
ignored / selected options (See also :issue:`284`, :pull:`1576`,
:pull:`1609`).
Bugs Fixed
~~~~~~~~~~
- Fix physical line plugins not receiving all lines in the case of
triple-quoted strings (See also :issue:`1534`, :pull:`1536`).
- Fix duplicate error logging in the case of plugin issues (See also
:pull:`1538`).
- Fix inconsistent ordering of ``--ignore`` in ``--help`` (See also
:issue:`1550`, :pull:`1552`).
- Fix memory leak of style guides by avoiding ``@lru_cache`` of a method (See
also :pull:`1573`).
- Fix ignoring of configuration files exactly in the home directory (See also
:issue:`1617`, :pull:`1618`).
.. all links
.. _5.0.0 milestone:
https://github.com/PyCQA/flake8/milestone/42

View file

@ -1,15 +0,0 @@
5.0.1 -- 2022-07-31
-------------------
You can view the `5.0.1 milestone`_ on GitHub for more details.
Bugs Fixed
~~~~~~~~~~
- Fix duplicate plugin discovery on misconfigured pythons (See also
:issue:`1627`, :pull:`1631`).
.. all links
.. _5.0.1 milestone:
https://github.com/PyCQA/flake8/milestone/43

View file

@ -1,16 +0,0 @@
5.0.2 -- 2022-08-01
-------------------
You can view the `5.0.2 milestone`_ on GitHub for more details.
Bugs Fixed
~~~~~~~~~~
- Fix execution on python == 3.8.0 (See also :issue:`1637`, :pull:`1641`).
- Fix config discovery when home does not exist (See also :issue:`1640`,
:pull:`1642`).
.. all links
.. _5.0.2 milestone:
https://github.com/PyCQA/flake8/milestone/44

View file

@ -1,15 +0,0 @@
5.0.3 -- 2022-08-01
-------------------
You can view the `5.0.3 milestone`_ on GitHub for more details.
Bugs Fixed
~~~~~~~~~~
- Work around partial reads of configuration files with syntax errors (See
also :issue:`1647`, :pull:`1648`).
.. all links
.. _5.0.3 milestone:
https://github.com/PyCQA/flake8/milestone/45

View file

@ -1,15 +0,0 @@
5.0.4 -- 2022-08-03
-------------------
You can view the `5.0.4 milestone`_ on GitHub for more details.
Bugs Fixed
~~~~~~~~~~
- Set a lower bound on ``importlib-metadata`` to prevent ``RecursionError``
(See also :issue:`1650`, :pull:`1653`).
.. all links
.. _5.0.4 milestone:
https://github.com/PyCQA/flake8/milestone/46

View file

@ -1,35 +0,0 @@
6.0.0 -- 2022-11-23
-------------------
You can view the `6.0.0 milestone`_ on GitHub for more details.
Backwards Incompatible Changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Remove ``--diff`` option (See also :issue:`1389`, :pull:`1720`).
- Produce an error when invalid codes are specified in configuration (See also
:issue:`1689`, :pull:`1713`).
- Produce an error if the file specified in ``--extend-config`` does not exist
(See also :issue:`1729`, :pull:`1732`).
- Remove ``optparse`` compatibility support (See also :pull:`1739`).
New Dependency Information
~~~~~~~~~~~~~~~~~~~~~~~~~~
- pycodestyle has been updated to >= 2.10.0, < 2.11.0 (See also :pull:`1746`).
- Pyflakes has been updated to >= 3.0.0, < 3.1.0 (See also :pull:`1748`).
Features
~~~~~~~~
- Require python >= 3.8.1 (See also :pull:`1633`, :pull:`1741`).
- List available formatters in for ``--format`` option in ``--help`` (See also
:issue:`223`, :pull:`1624`).
- Improve multiprocessing performance (See also :pull:`1723`).
- Enable multiprocessing on non-fork platforms (See also :pull:`1723`).
- Ensure results are sorted when discovered from files (See also :issue:`1670`,
:pull:`1726`).
.. all links
.. _6.0.0 milestone:
https://github.com/PyCQA/flake8/milestone/47

View file

@ -1,22 +0,0 @@
6.1.0 -- 2023-07-29
-------------------
You can view the `6.1.0 milestone`_ on GitHub for more details.
New Dependency Information
~~~~~~~~~~~~~~~~~~~~~~~~~~
- Pyflakes has been updated to >= 3.1.0, < 3.2.0 (See also :pull:`1847`).
- pycodestyle has been updated to >= 2.11.0, < 2.12.0 (See also :pull:`1848`).
Features
~~~~~~~~
- Deprecate ``--include-in-doctest``, ``--exclude-from-doctest`` (See also
:issue:`1747`, :pull:`1768`).
- Add support for python 3.12 (See also :pull:`1832`, :pull:`1849`,
:pull:`1850`).
.. all links
.. _6.1.0 milestone:
https://github.com/PyCQA/flake8/milestone/48

View file

@ -1,19 +0,0 @@
7.0.0 -- 2024-01-04
-------------------
You can view the `7.0.0 milestone`_ on GitHub for more details.
Backwards Incompatible Changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Remove ``--include-in-doctest`` and ``--exclude-from-doctest`` options.
(See also :issue:`1747`, :pull:`1854`)
New Dependency Information
~~~~~~~~~~~~~~~~~~~~~~~~~~
- Pyflakes has been updated to >= 3.2.0, < 3.3.0 (See also :pull:`1906`).
.. all links
.. _7.0.0 milestone:
https://github.com/PyCQA/flake8/milestone/49

View file

@ -1,13 +0,0 @@
7.1.0 -- 2024-06-15
-------------------
You can view the `7.1.0 milestone`_ on GitHub for more details.
New Dependency Information
~~~~~~~~~~~~~~~~~~~~~~~~~~
- pycodestyle has been updated to >= 2.12.0, < 2.13.0 (See also :pull:`1939`).
.. all links
.. _7.1.0 milestone:
https://github.com/PyCQA/flake8/milestone/50

View file

@ -1,15 +0,0 @@
7.1.1 -- 2024-08-04
-------------------
You can view the `7.1.1 milestone`_ on GitHub for more details.
Bugs Fixed
~~~~~~~~~~
- Properly preserve escaped `{` and `}` in fstrings in logical lines in 3.12+.
(See also :issue:`1948`, :pull:`1949`).
.. all links
.. _7.1.1 milestone:
https://github.com/PyCQA/flake8/milestone/51

View file

@ -1,15 +0,0 @@
7.1.2 -- 2025-02-16
-------------------
You can view the `7.1.2 milestone`_ on GitHub for more details.
Bugs Fixed
~~~~~~~~~~
- Avoid starting unnecessary processes when "# files" < "jobs".
(See also :pull:`1966`).
.. all links
.. _7.1.2 milestone:
https://github.com/PyCQA/flake8/milestone/52

View file

@ -1,19 +0,0 @@
7.2.0 -- 2025-03-29
-------------------
You can view the `7.2.0 milestone`_ on GitHub for more details.
New Dependency Information
~~~~~~~~~~~~~~~~~~~~~~~~~~
- pycodestyle has been updated to >= 2.13.0, < 2.14.0 (See also :pull:`1974`).
- pyflakes has been updated to >= 3.3.0, < 3.4.0 (See also :pull:`1974`).
Features
~~~~~~~~
- Require python >= 3.9 (See also :pull:`1973`).
.. all links
.. _7.2.0 milestone:
https://github.com/PyCQA/flake8/milestone/53

View file

@ -1,15 +0,0 @@
7.3.0 -- 2025-06-20
-------------------
You can view the `7.3.0 milestone`_ on GitHub for more details.
New Dependency Information
~~~~~~~~~~~~~~~~~~~~~~~~~~
- Added support for python 3.14 (See also :pull:`1983`).
- pycodestyle has been updated to >= 2.14.0, < 2.15.0 (See also :pull:`1985`).
- Pyflakes has been updated to >= 3.4.0, < 3.5.0 (See also :pull:`1985`).
.. all links
.. _7.3.0 milestone:
https://github.com/PyCQA/flake8/milestone/54

View file

@ -5,34 +5,6 @@
All of the release notes that have been recorded for Flake8 are organized here All of the release notes that have been recorded for Flake8 are organized here
with the newest releases first. with the newest releases first.
7.x Release Series
==================
.. toctree::
7.3.0
7.2.0
7.1.2
7.1.1
7.1.0
7.0.0
6.x Release Series
==================
.. toctree::
6.1.0
6.0.0
5.x Release Series
==================
.. toctree::
5.0.4
5.0.3
5.0.2
5.0.1
5.0.0
4.x Release Series 4.x Release Series
================== ==================

View file

@ -1,4 +1,4 @@
sphinx>=2.1.0,!=3.1.0 sphinx>=1.3.0,!=3.1.0
sphinx-rtd-theme>=1.2.2 sphinx_rtd_theme
sphinx-prompt>=1.8.0 sphinx-prompt
docutils!=0.18 flake8-polyfill

View file

@ -24,12 +24,36 @@ Remember that you want to specify certain options without writing
Configuration Locations Configuration Locations
======================= =======================
|Flake8| supports storing its configuration in your project in one of |Flake8| supports storing its configuration in the following places:
``setup.cfg``, ``tox.ini``, or ``.flake8``.
- Your top-level user directory
- In your project in one of ``setup.cfg``, ``tox.ini``, or ``.flake8``.
Values set at the command line have highest priority, then those in the Values set at the command line have highest priority, then those in the
project configuration file, and finally there are the defaults. However, project configuration file, then those in your user directory, and finally
there are additional command line options which can alter this. there are the defaults. However, there are additional command line options
which can alter this.
"User" Configuration
--------------------
|Flake8| allows a user to use "global" configuration file to store preferences.
The user configuration file is expected to be stored somewhere in the user's
"home" directory.
- On Windows the "home" directory will be something like
``C:\\Users\sigmavirus24``, a.k.a, ``~\``.
- On Linux and other Unix like systems (including OS X) we will look in
``~/``.
Note that |Flake8| looks for ``~\.flake8`` on Windows and ``~/.config/flake8``
on Linux and other Unix systems.
User configuration files use the same syntax as Project Configuration files.
Keep reading to see that syntax.
Project Configuration Project Configuration
@ -90,7 +114,7 @@ Let's actually look at |Flake8|'s own configuration section:
.. code-block:: ini .. code-block:: ini
[flake8] [flake8]
extend-ignore = E203 ignore = D203
exclude = .git,__pycache__,docs/source/conf.py,old,build,dist exclude = .git,__pycache__,docs/source/conf.py,old,build,dist
max-complexity = 10 max-complexity = 10
@ -98,7 +122,7 @@ This is equivalent to:
.. prompt:: bash .. prompt:: bash
flake8 --extend-ignore E203 \ flake8 --ignore D203 \
--exclude .git,__pycache__,docs/source/conf.py,old,build,dist \ --exclude .git,__pycache__,docs/source/conf.py,old,build,dist \
--max-complexity 10 --max-complexity 10
@ -107,7 +131,7 @@ In our case, if we wanted to, we could also do
.. code-block:: ini .. code-block:: ini
[flake8] [flake8]
extend-ignore = E203 ignore = D203
exclude = exclude =
.git, .git,
__pycache__, __pycache__,
@ -122,7 +146,7 @@ This allows us to add comments for why we're excluding items, e.g.
.. code-block:: ini .. code-block:: ini
[flake8] [flake8]
extend-ignore = E203 ignore = D203
exclude = exclude =
# No need to traverse our git directory # No need to traverse our git directory
.git, .git,
@ -180,6 +204,7 @@ look at a portion of a project's Flake8 configuration in their ``tox.ini``:
[flake8] [flake8]
# it's not a bug that we aren't using all of hacking, ignore: # it's not a bug that we aren't using all of hacking, ignore:
# F812: list comprehension redefines ...
# H101: Use TODO(NAME) # H101: Use TODO(NAME)
# H202: assertRaises Exception too broad # H202: assertRaises Exception too broad
# H233: Python 3.x incompatible use of print operator # H233: Python 3.x incompatible use of print operator
@ -190,7 +215,7 @@ look at a portion of a project's Flake8 configuration in their ``tox.ini``:
# H404: multi line docstring should start without a leading new line # H404: multi line docstring should start without a leading new line
# H405: multi line docstring summary not separated with an empty line # H405: multi line docstring summary not separated with an empty line
# H501: Do not use self.__dict__ for string formatting # H501: Do not use self.__dict__ for string formatting
extend-ignore = H101,H202,H233,H301,H306,H401,H403,H404,H405,H501 ignore = F812,H101,H202,H233,H301,H306,H401,H403,H404,H405,H501
They use the comments to describe the check but they could also write this as: They use the comments to describe the check but they could also write this as:
@ -198,7 +223,9 @@ They use the comments to describe the check but they could also write this as:
[flake8] [flake8]
# it's not a bug that we aren't using all of hacking # it's not a bug that we aren't using all of hacking
extend-ignore = ignore =
# F812: list comprehension redefines ...
F812,
# H101: Use TODO(NAME) # H101: Use TODO(NAME)
H101, H101,
# H202: assertRaises Exception too broad # H202: assertRaises Exception too broad

View file

@ -59,8 +59,6 @@ generates its own :term:`error code`\ s for ``pyflakes``:
+------+---------------------------------------------------------------------+ +------+---------------------------------------------------------------------+
| F541 | f-string without any placeholders | | F541 | f-string without any placeholders |
+------+---------------------------------------------------------------------+ +------+---------------------------------------------------------------------+
| F542 | t-string without any placeholders |
+------+---------------------------------------------------------------------+
+------+---------------------------------------------------------------------+ +------+---------------------------------------------------------------------+
| F601 | dictionary key ``name`` repeated with different values | | F601 | dictionary key ``name`` repeated with different values |
+------+---------------------------------------------------------------------+ +------+---------------------------------------------------------------------+
@ -83,8 +81,12 @@ generates its own :term:`error code`\ s for ``pyflakes``:
+------+---------------------------------------------------------------------+ +------+---------------------------------------------------------------------+
| F702 | a ``continue`` statement outside of a ``while`` or ``for`` loop | | F702 | a ``continue`` statement outside of a ``while`` or ``for`` loop |
+------+---------------------------------------------------------------------+ +------+---------------------------------------------------------------------+
| F703 | a ``continue`` statement in a ``finally`` block in a loop |
+------+---------------------------------------------------------------------+
| F704 | a ``yield`` or ``yield from`` statement outside of a function | | F704 | a ``yield`` or ``yield from`` statement outside of a function |
+------+---------------------------------------------------------------------+ +------+---------------------------------------------------------------------+
| F705 | a ``return`` statement with arguments inside a generator |
+------+---------------------------------------------------------------------+
| F706 | a ``return`` statement outside of a function/method | | F706 | a ``return`` statement outside of a function/method |
+------+---------------------------------------------------------------------+ +------+---------------------------------------------------------------------+
| F707 | an ``except:`` block as not the last exception handler | | F707 | an ``except:`` block as not the last exception handler |
@ -104,9 +106,6 @@ generates its own :term:`error code`\ s for ``pyflakes``:
+------+---------------------------------------------------------------------+ +------+---------------------------------------------------------------------+
| F823 | local variable ``name`` ... referenced before assignment | | F823 | local variable ``name`` ... referenced before assignment |
+------+---------------------------------------------------------------------+ +------+---------------------------------------------------------------------+
| F824 | ``global name`` / ``nonlocal name`` is unused: name is never |
| | assigned in scope |
+------+---------------------------------------------------------------------+
| F831 | duplicate argument ``name`` in function definition | | F831 | duplicate argument ``name`` in function definition |
+------+---------------------------------------------------------------------+ +------+---------------------------------------------------------------------+
| F841 | local variable ``name`` is assigned to but never used | | F841 | local variable ``name`` is assigned to but never used |

View file

@ -14,25 +14,25 @@ like so:
Where you simply allow the shell running in your terminal to locate |Flake8|. Where you simply allow the shell running in your terminal to locate |Flake8|.
In some cases, though, you may have installed |Flake8| for multiple versions In some cases, though, you may have installed |Flake8| for multiple versions
of Python (e.g., Python 3.13 and Python 3.14) and you need to call a specific of Python (e.g., Python 3.8 and Python 3.9) and you need to call a specific
version. In that case, you will have much better results using: version. In that case, you will have much better results using:
.. prompt:: bash .. prompt:: bash
python3.13 -m flake8 python3.8 -m flake8
Or Or
.. prompt:: bash .. prompt:: bash
python3.14 -m flake8 python3.9 -m flake8
Since that will tell the correct version of Python to run |Flake8|. Since that will tell the correct version of Python to run |Flake8|.
.. note:: .. note::
Installing |Flake8| once will not install it on both Python 3.13 and Installing |Flake8| once will not install it on both Python 3.8 and
Python 3.14. It will only install it for the version of Python that Python 3.9. It will only install it for the version of Python that
is running pip. is running pip.
It is also possible to specify command-line options directly to |Flake8|: It is also possible to specify command-line options directly to |Flake8|:
@ -51,7 +51,7 @@ Or
This is the last time we will show both versions of an invocation. This is the last time we will show both versions of an invocation.
From now on, we'll simply use ``flake8`` and assume that the user From now on, we'll simply use ``flake8`` and assume that the user
knows they can instead use ``python<version> -m flake8``. knows they can instead use ``python<version> -m flake8`` instead.
It's also possible to narrow what |Flake8| will try to check by specifying It's also possible to narrow what |Flake8| will try to check by specifying
exactly the paths and directories you want it to check. Let's assume that exactly the paths and directories you want it to check. Let's assume that
@ -86,5 +86,68 @@ And you should see something like:
Options: Options:
--version show program's version number and exit --version show program's version number and exit
-h, --help show this help message and exit -h, --help show this help message and exit
-v, --verbose Print more information about what is happening in
flake8. This option is repeatable and will increase
verbosity each time it is repeated.
-q, --quiet Report only file names, or nothing. This option is
repeatable.
--count Print total number of errors and warnings to standard
error and set the exit code to 1 if total is not
empty.
--diff Report changes only within line number ranges in the
unified diff provided on standard in by the user.
--exclude=patterns Comma-separated list of files or directories to
exclude.(Default:
.svn,CVS,.bzr,.hg,.git,__pycache__,.tox,.eggs,*.egg)
--filename=patterns Only check for filenames matching the patterns in this
comma-separated list. (Default: *.py)
--format=format Format errors according to the chosen formatter.
--hang-closing Hang closing bracket instead of matching indentation
of opening bracket's line.
--ignore=errors Comma-separated list of errors and warnings to ignore
(or skip). For example, ``--ignore=E4,E51,W234``.
(Default: E121,E123,E126,E226,E24,E704)
--extend-ignore=errors
Comma-separated list of errors and warnings to add to
the list of ignored ones. For example, ``--extend-
ignore=E4,E51,W234``.
--max-line-length=n Maximum allowed line length for the entirety of this
run. (Default: 79)
--select=errors Comma-separated list of errors and warnings to enable.
For example, ``--select=E4,E51,W234``. (Default: )
--extend-select errors
Comma-separated list of errors and warnings to add to
the list of selected ones. For example, ``--extend-
select=E4,E51,W234``.
--disable-noqa Disable the effect of "# noqa". This will report
errors on lines with "# noqa" at the end.
--show-source Show the source generate each error or warning.
--statistics Count errors and warnings.
--enabled-extensions=ENABLED_EXTENSIONS
Enable plugins and extensions that are otherwise
disabled by default
--exit-zero Exit with status code "0" even if there are errors.
-j JOBS, --jobs=JOBS Number of subprocesses to use to run checks in
parallel. This is ignored on Windows. The default,
"auto", will auto-detect the number of processors
available to use. (Default: auto)
--output-file=OUTPUT_FILE
Redirect report to a file.
--tee Write to stdout and output-file.
--append-config=APPEND_CONFIG
Provide extra config files to parse in addition to the
files found by Flake8 by default. These files are the
last ones read and so they take the highest precedence
when multiple files provide the same option.
--config=CONFIG Path to the config file that will be the authoritative
config source. This will cause Flake8 to ignore all
other configuration files.
--isolated Ignore all configuration files.
--builtins=BUILTINS define more built-ins, comma separated
--doctests check syntax of the doctests
--include-in-doctest=INCLUDE_IN_DOCTEST
Run doctests only on these files
--exclude-from-doctest=EXCLUDE_FROM_DOCTEST
Skip these files when running doctests
... Installed plugins: pyflakes: 1.0.0, pep8: 1.7.0

View file

@ -40,13 +40,11 @@ Index of Options
- :option:`flake8 --quiet` - :option:`flake8 --quiet`
- :option:`flake8 --color`
- :option:`flake8 --count` - :option:`flake8 --count`
- :option:`flake8 --exclude` - :option:`flake8 --diff`
- :option:`flake8 --extend-exclude` - :option:`flake8 --exclude`
- :option:`flake8 --filename` - :option:`flake8 --filename`
@ -78,8 +76,6 @@ Index of Options
- :option:`flake8 --statistics` - :option:`flake8 --statistics`
- :option:`flake8 --require-plugins`
- :option:`flake8 --enable-extensions` - :option:`flake8 --enable-extensions`
- :option:`flake8 --exit-zero` - :option:`flake8 --exit-zero`
@ -100,6 +96,10 @@ Index of Options
- :option:`flake8 --doctests` - :option:`flake8 --doctests`
- :option:`flake8 --include-in-doctest`
- :option:`flake8 --exclude-from-doctest`
- :option:`flake8 --benchmark` - :option:`flake8 --benchmark`
- :option:`flake8 --bug-report` - :option:`flake8 --bug-report`
@ -181,29 +181,6 @@ Options and their Descriptions
quiet = 1 quiet = 1
.. option:: --color
:ref:`Go back to index <top>`
Whether to use color in output. Defaults to ``auto``.
Possible options are ``auto``, ``always``, and ``never``.
This **can not** be specified in config files.
When color is enabled, the following substitutions are enabled:
- ``%(bold)s``
- ``%(black)s``
- ``%(red)s``
- ``%(green)s``
- ``%(yellow)s``
- ``%(blue)s``
- ``%(magenta)s``
- ``%(cyan)s``
- ``%(white)s``
- ``%(reset)s``
.. option:: --count .. option:: --count
@ -226,13 +203,29 @@ Options and their Descriptions
count = True count = True
.. option:: --diff
:ref:`Go back to index <top>`
Use the unified diff provided on standard in to only check the modified
files and report errors included in the diff.
Command-line example:
.. prompt:: bash
git diff -u | flake8 --diff
This **can not** be specified in config files.
.. option:: --exclude=<patterns> .. option:: --exclude=<patterns>
:ref:`Go back to index <top>` :ref:`Go back to index <top>`
Provide a comma-separated list of glob patterns to exclude from checks. Provide a comma-separated list of glob patterns to exclude from checks.
This defaults to: ``.svn,CVS,.bzr,.hg,.git,__pycache__,.tox,.nox,.eggs,*.egg`` This defaults to: ``.svn,CVS,.bzr,.hg,.git,__pycache__,.tox,.eggs,*.egg``
Example patterns: Example patterns:
@ -537,9 +530,9 @@ Options and their Descriptions
# https://some-super-long-domain-name.com/with/some/very/long/path # https://some-super-long-domain-name.com/with/some/very/long/path
url = '''\ url = (
https://... 'http://...'
''' )
This defaults to: ``79`` This defaults to: ``79``
@ -605,14 +598,13 @@ Options and their Descriptions
:ref:`Go back to index <top>` :ref:`Go back to index <top>`
**You usually do not need to specify this option as the default includes
all installed plugin codes.**
Specify the list of error codes you wish |Flake8| to report. Similarly to Specify the list of error codes you wish |Flake8| to report. Similarly to
:option:`--ignore`. You can specify a portion of an error code to get all :option:`--ignore`. You can specify a portion of an error code to get all
that start with that string. For example, you can use ``E``, ``E4``, that start with that string. For example, you can use ``E``, ``E4``,
``E43``, and ``E431``. ``E43``, and ``E431``.
This defaults to: ``E,F,W,C90``
Command-line example: Command-line example:
.. prompt:: bash .. prompt:: bash
@ -648,9 +640,6 @@ Options and their Descriptions
.. versionadded:: 4.0.0 .. versionadded:: 4.0.0
**You usually do not need to specify this option as the default includes
all installed plugin codes.**
Specify a list of codes to add to the list of selected ones. Similar Specify a list of codes to add to the list of selected ones. Similar
considerations as in :option:`--select` apply here with regard to the considerations as in :option:`--select` apply here with regard to the
value. value.
@ -747,43 +736,15 @@ Options and their Descriptions
statistics = True statistics = True
.. option:: --require-plugins=<names>
:ref:`Go back to index <top>`
Require specific plugins to be installed before running.
This option takes a list of distribution names (usually the name you would
use when running ``pip install``).
Command-line example:
.. prompt:: bash
flake8 --require-plugins=flake8-2020,flake8-typing-extensions dir/
This **can** be specified in config files.
Example config file usage:
.. code-block:: ini
require-plugins =
flake8-2020
flake8-typing-extensions
.. _option-enable-extensions:
.. option:: --enable-extensions=<errors> .. option:: --enable-extensions=<errors>
:ref:`Go back to index <top>` :ref:`Go back to index <top>`
Enable :ref:`off-by-default<off-by-default>` extensions. Enable off-by-default extensions.
Plugins to |Flake8| have the option of registering themselves as Plugins to |Flake8| have the option of registering themselves as
off-by-default. These plugins will not be loaded unless enabled by this off-by-default. These plugins effectively add themselves to the
option. default ignore list.
Command-line example: Command-line example:
@ -886,6 +847,7 @@ Options and their Descriptions
.. code-block:: ini .. code-block:: ini
output-file = output.txt
tee = True tee = True
@ -995,6 +957,62 @@ Options and their Descriptions
doctests = True doctests = True
.. option:: --include-in-doctest=<paths>
:ref:`Go back to index <top>`
Specify which files are checked by PyFlakes for doctest syntax.
This is registered by the default PyFlakes plugin.
Command-line example:
.. prompt:: bash
flake8 --include-in-doctest=dir/subdir/file.py,dir/other/file.py dir/
This **can** be specified in config files.
Example config file usage:
.. code-block:: ini
include-in-doctest =
dir/subdir/file.py,
dir/other/file.py
include_in_doctest =
dir/subdir/file.py,
dir/other/file.py
.. option:: --exclude-from-doctest=<paths>
:ref:`Go back to index <top>`
Specify which files are not to be checked by PyFlakes for doctest syntax.
This is registered by the default PyFlakes plugin.
Command-line example:
.. prompt:: bash
flake8 --exclude-from-doctest=dir/subdir/file.py,dir/other/file.py dir/
This **can** be specified in config files.
Example config file usage:
.. code-block:: ini
exclude-from-doctest =
dir/subdir/file.py,
dir/other/file.py
exclude_from_doctest =
dir/subdir/file.py,
dir/other/file.py
.. option:: --benchmark .. option:: --benchmark
:ref:`Go back to index <top>` :ref:`Go back to index <top>`

View file

@ -70,7 +70,7 @@ namely
.. warning:: .. warning::
These are not *perfectly* backwards compatible. Not all arguments are These are not *perfectly* backwards compatible. Not all arguments are
respected, and some of the types necessary for something to work have respsected, and some of the types necessary for something to work have
changed. changed.
Most people, we observed, were using Most people, we observed, were using

View file

@ -1,5 +1,3 @@
from __future__ import annotations
import setuptools import setuptools
setuptools.setup( setuptools.setup(
@ -23,6 +21,8 @@ setuptools.setup(
"License :: OSI Approved :: MIT License", "License :: OSI Approved :: MIT License",
"Programming Language :: Python", "Programming Language :: Python",
"Programming Language :: Python :: 3", "Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Topic :: Software Development :: Libraries :: Python Modules", "Topic :: Software Development :: Libraries :: Python Modules",
"Topic :: Software Development :: Quality Assurance", "Topic :: Software Development :: Quality Assurance",
], ],

View file

@ -1,6 +1,4 @@
"""Module for an example Flake8 plugin.""" """Module for an example Flake8 plugin."""
from __future__ import annotations
from .off_by_default import ExampleTwo from .off_by_default import ExampleTwo
from .on_by_default import ExampleOne from .on_by_default import ExampleOne

View file

@ -1,10 +1,12 @@
"""Our first example plugin.""" """Our first example plugin."""
from __future__ import annotations
class ExampleTwo: class ExampleTwo:
"""Second Example Plugin.""" """Second Example Plugin."""
name = "off-by-default-example-plugin"
version = "1.0.0"
off_by_default = True off_by_default = True
def __init__(self, tree): def __init__(self, tree):

View file

@ -1,10 +1,12 @@
"""Our first example plugin.""" """Our first example plugin."""
from __future__ import annotations
class ExampleOne: class ExampleOne:
"""First Example Plugin.""" """First Example Plugin."""
name = "on-by-default-example-plugin"
version = "1.0.0"
def __init__(self, tree): def __init__(self, tree):
self.tree = tree self.tree = tree

View file

@ -1,4 +1,6 @@
[pytest] [pytest]
norecursedirs = .git .* *.egg* docs dist build norecursedirs = .git .* *.egg* old docs dist build
addopts = -rw addopts = -rw
filterwarnings = error filterwarnings =
error
ignore:SelectableGroups:DeprecationWarning

View file

@ -10,15 +10,21 @@ author_email = tarek@ziade.org
maintainer = Ian Stapleton Cordasco maintainer = Ian Stapleton Cordasco
maintainer_email = graffatcolmingov@gmail.com maintainer_email = graffatcolmingov@gmail.com
license = MIT license = MIT
license_files = LICENSE license_file = LICENSE
classifiers = classifiers =
Development Status :: 5 - Production/Stable Development Status :: 5 - Production/Stable
Environment :: Console Environment :: Console
Framework :: Flake8 Framework :: Flake8
Intended Audience :: Developers Intended Audience :: Developers
License :: OSI Approved :: MIT License
Programming Language :: Python Programming Language :: Python
Programming Language :: Python :: 3 Programming Language :: Python :: 3
Programming Language :: Python :: 3 :: Only Programming Language :: Python :: 3 :: Only
Programming Language :: Python :: 3.6
Programming Language :: Python :: 3.7
Programming Language :: Python :: 3.8
Programming Language :: Python :: 3.9
Programming Language :: Python :: 3.10
Programming Language :: Python :: Implementation :: CPython Programming Language :: Python :: Implementation :: CPython
Programming Language :: Python :: Implementation :: PyPy Programming Language :: Python :: Implementation :: PyPy
Topic :: Software Development :: Libraries :: Python Modules Topic :: Software Development :: Libraries :: Python Modules
@ -26,13 +32,18 @@ classifiers =
[options] [options]
packages = find: packages = find:
install_requires =
mccabe>=0.7.0,<0.8.0
pycodestyle>=2.14.0,<2.15.0
pyflakes>=3.4.0,<3.5.0
python_requires = >=3.10
package_dir = package_dir =
=src =src
# We document the reasoning for using ranges here:
# http://flake8.pycqa.org/en/latest/faq.html#why-does-flake8-use-ranges-for-its-dependencies
# And in which releases we will update those ranges here:
# http://flake8.pycqa.org/en/latest/internal/releases.html#releasing-flake8
install_requires =
mccabe>=0.6.0,<0.7.0
pycodestyle>=2.8.0,<2.9.0
pyflakes>=2.4.0,<2.5.0
importlib-metadata<4.3;python_version<"3.8"
python_requires = >=3.6
[options.packages.find] [options.packages.find]
where = src where = src
@ -42,8 +53,42 @@ console_scripts =
flake8 = flake8.main.cli:main flake8 = flake8.main.cli:main
flake8.extension = flake8.extension =
F = flake8.plugins.pyflakes:FlakesChecker F = flake8.plugins.pyflakes:FlakesChecker
E = flake8.plugins.pycodestyle:pycodestyle_logical pycodestyle.ambiguous_identifier = pycodestyle:ambiguous_identifier
W = flake8.plugins.pycodestyle:pycodestyle_physical pycodestyle.bare_except = pycodestyle:bare_except
pycodestyle.blank_lines = pycodestyle:blank_lines
pycodestyle.break_after_binary_operator = pycodestyle:break_after_binary_operator
pycodestyle.break_before_binary_operator = pycodestyle:break_before_binary_operator
pycodestyle.comparison_negative = pycodestyle:comparison_negative
pycodestyle.comparison_to_singleton = pycodestyle:comparison_to_singleton
pycodestyle.comparison_type = pycodestyle:comparison_type
pycodestyle.compound_statements = pycodestyle:compound_statements
pycodestyle.continued_indentation = pycodestyle:continued_indentation
pycodestyle.explicit_line_join = pycodestyle:explicit_line_join
pycodestyle.extraneous_whitespace = pycodestyle:extraneous_whitespace
pycodestyle.imports_on_separate_lines = pycodestyle:imports_on_separate_lines
pycodestyle.indentation = pycodestyle:indentation
pycodestyle.maximum_doc_length = pycodestyle:maximum_doc_length
pycodestyle.maximum_line_length = pycodestyle:maximum_line_length
pycodestyle.missing_whitespace = pycodestyle:missing_whitespace
pycodestyle.missing_whitespace_after_import_keyword = pycodestyle:missing_whitespace_after_import_keyword
pycodestyle.missing_whitespace_around_operator = pycodestyle:missing_whitespace_around_operator
pycodestyle.module_imports_on_top_of_file = pycodestyle:module_imports_on_top_of_file
pycodestyle.python_3000_async_await_keywords = pycodestyle:python_3000_async_await_keywords
pycodestyle.python_3000_backticks = pycodestyle:python_3000_backticks
pycodestyle.python_3000_has_key = pycodestyle:python_3000_has_key
pycodestyle.python_3000_invalid_escape_sequence = pycodestyle:python_3000_invalid_escape_sequence
pycodestyle.python_3000_not_equal = pycodestyle:python_3000_not_equal
pycodestyle.python_3000_raise_comma = pycodestyle:python_3000_raise_comma
pycodestyle.tabs_obsolete = pycodestyle:tabs_obsolete
pycodestyle.tabs_or_spaces = pycodestyle:tabs_or_spaces
pycodestyle.trailing_blank_lines = pycodestyle:trailing_blank_lines
pycodestyle.trailing_whitespace = pycodestyle:trailing_whitespace
pycodestyle.whitespace_around_comma = pycodestyle:whitespace_around_comma
pycodestyle.whitespace_around_keywords = pycodestyle:whitespace_around_keywords
pycodestyle.whitespace_around_named_parameter_equals = pycodestyle:whitespace_around_named_parameter_equals
pycodestyle.whitespace_around_operator = pycodestyle:whitespace_around_operator
pycodestyle.whitespace_before_comment = pycodestyle:whitespace_before_comment
pycodestyle.whitespace_before_parameters = pycodestyle:whitespace_before_parameters
flake8.report = flake8.report =
default = flake8.formatting.default:Default default = flake8.formatting.default:Default
pylint = flake8.formatting.default:Pylint pylint = flake8.formatting.default:Pylint
@ -53,22 +98,33 @@ flake8.report =
[bdist_wheel] [bdist_wheel]
universal = 1 universal = 1
[coverage:run]
source =
flake8
tests
plugins = covdefaults
[coverage:report]
fail_under = 97
[mypy] [mypy]
check_untyped_defs = true check_untyped_defs = true
disallow_any_generics = true disallow_any_generics = true
disallow_incomplete_defs = true disallow_incomplete_defs = true
disallow_untyped_defs = true # TODO: disallow_untyped_defs = true
no_implicit_optional = true no_implicit_optional = true
warn_unused_ignores = true warn_unused_ignores = true
# TODO: until we opt in all the modules
[mypy-flake8.defaults]
disallow_untyped_defs = true
[mypy-flake8.exceptions]
disallow_untyped_defs = true
[mypy-flake8.formatting.*]
disallow_untyped_defs = true
[mypy-flake8.options.manager]
disallow_untyped_defs = true
[mypy-flake8.main.cli]
disallow_untyped_defs = true
[mypy-flake8.processor]
disallow_untyped_defs = true
[mypy-flake8.statistics]
disallow_untyped_defs = true
[mypy-flake8.style_guide]
disallow_untyped_defs = true
[mypy-flake8.utils]
disallow_untyped_defs = true
[mypy-tests.*] [mypy-tests.*]
disallow_untyped_defs = false disallow_untyped_defs = false

View file

@ -1,6 +1,4 @@
"""Packaging logic for Flake8.""" """Packaging logic for Flake8."""
from __future__ import annotations
import os import os
import sys import sys

View file

@ -9,22 +9,29 @@ This module
.. autofunction:: flake8.configure_logging .. autofunction:: flake8.configure_logging
""" """
from __future__ import annotations
import logging import logging
import sys import sys
from typing import Type
LOG = logging.getLogger(__name__) LOG = logging.getLogger(__name__)
LOG.addHandler(logging.NullHandler()) LOG.addHandler(logging.NullHandler())
__version__ = "7.3.0" __version__ = "4.0.1"
__version_info__ = tuple(int(i) for i in __version__.split(".") if i.isdigit()) __version_info__ = tuple(int(i) for i in __version__.split(".") if i.isdigit())
# There is nothing lower than logging.DEBUG (10) in the logging library,
# but we want an extra level to avoid being too verbose when using -vv.
_EXTRA_VERBOSE = 5
logging.addLevelName(_EXTRA_VERBOSE, "VERBOSE")
_VERBOSITY_TO_LOG_LEVEL = { _VERBOSITY_TO_LOG_LEVEL = {
# output more than warnings but not debugging info # output more than warnings but not debugging info
1: logging.INFO, # INFO is a numerical level of 20 1: logging.INFO, # INFO is a numerical level of 20
# output debugging information # output debugging information
2: logging.DEBUG, # DEBUG is a numerical level of 10 2: logging.DEBUG, # DEBUG is a numerical level of 10
# output extra verbose debugging information
3: _EXTRA_VERBOSE,
} }
LOG_FORMAT = ( LOG_FORMAT = (
@ -33,16 +40,12 @@ LOG_FORMAT = (
) )
def configure_logging( def configure_logging(verbosity, filename=None, logformat=LOG_FORMAT):
verbosity: int,
filename: str | None = None,
logformat: str = LOG_FORMAT,
) -> None:
"""Configure logging for flake8. """Configure logging for flake8.
:param verbosity: :param int verbosity:
How verbose to be in logging information. How verbose to be in logging information.
:param filename: :param str filename:
Name of the file to append log information to. Name of the file to append log information to.
If ``None`` this will log to ``sys.stderr``. If ``None`` this will log to ``sys.stderr``.
If the name is "stdout" or "stderr" this will log to the appropriate If the name is "stdout" or "stderr" this will log to the appropriate
@ -50,13 +53,14 @@ def configure_logging(
""" """
if verbosity <= 0: if verbosity <= 0:
return return
if verbosity > 3:
verbosity = 3
verbosity = min(verbosity, max(_VERBOSITY_TO_LOG_LEVEL))
log_level = _VERBOSITY_TO_LOG_LEVEL[verbosity] log_level = _VERBOSITY_TO_LOG_LEVEL[verbosity]
if not filename or filename in ("stderr", "stdout"): if not filename or filename in ("stderr", "stdout"):
fileobj = getattr(sys, filename or "stderr") fileobj = getattr(sys, filename or "stderr")
handler_cls: type[logging.Handler] = logging.StreamHandler handler_cls: Type[logging.Handler] = logging.StreamHandler
else: else:
fileobj = filename fileobj = filename
handler_cls = logging.FileHandler handler_cls = logging.FileHandler
@ -66,5 +70,5 @@ def configure_logging(
LOG.addHandler(handler) LOG.addHandler(handler)
LOG.setLevel(log_level) LOG.setLevel(log_level)
LOG.debug( LOG.debug(
"Added a %s logging handler to logger root at %s", filename, __name__, "Added a %s logging handler to logger root at %s", filename, __name__
) )

View file

@ -1,7 +1,4 @@
"""Module allowing for ``python -m flake8 ...``.""" """Module allowing for ``python -m flake8 ...``."""
from __future__ import annotations from flake8.main import cli
from flake8.main.cli import main cli.main()
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -1,18 +1,9 @@
from __future__ import annotations """Expose backports in a single place."""
import sys import sys
import tokenize
if sys.version_info >= (3, 12): # pragma: >=3.12 cover if sys.version_info >= (3, 8): # pragma: no cover (PY38+)
FSTRING_START = tokenize.FSTRING_START import importlib.metadata as importlib_metadata
FSTRING_MIDDLE = tokenize.FSTRING_MIDDLE else: # pragma: no cover (<PY38)
FSTRING_END = tokenize.FSTRING_END import importlib_metadata
else: # pragma: <3.12 cover
FSTRING_START = FSTRING_MIDDLE = FSTRING_END = -1
if sys.version_info >= (3, 14): # pragma: >=3.14 cover __all__ = ("importlib_metadata",)
TSTRING_START = tokenize.TSTRING_START
TSTRING_MIDDLE = tokenize.TSTRING_MIDDLE
TSTRING_END = tokenize.TSTRING_END
else: # pragma: <3.14 cover
TSTRING_START = TSTRING_MIDDLE = TSTRING_END = -1

View file

@ -3,4 +3,3 @@
This is the only submodule in Flake8 with a guaranteed stable API. All other This is the only submodule in Flake8 with a guaranteed stable API. All other
submodules are considered internal only and are subject to change. submodules are considered internal only and are subject to change.
""" """
from __future__ import annotations

View file

@ -3,17 +3,14 @@
Previously, users would import :func:`get_style_guide` from ``flake8.engine``. Previously, users would import :func:`get_style_guide` from ``flake8.engine``.
In 3.0 we no longer have an "engine" module but we maintain the API from it. In 3.0 we no longer have an "engine" module but we maintain the API from it.
""" """
from __future__ import annotations
import argparse import argparse
import logging import logging
import os.path import os.path
from typing import Any
from flake8.discover_files import expand_paths import flake8
from flake8.formatting import base as formatter from flake8.formatting import base as formatter
from flake8.main import application as app from flake8.main import application as app
from flake8.options.parse_args import parse_args from flake8.options import config
LOG = logging.getLogger(__name__) LOG = logging.getLogger(__name__)
@ -21,48 +18,46 @@ LOG = logging.getLogger(__name__)
__all__ = ("get_style_guide",) __all__ = ("get_style_guide",)
class Report: def get_style_guide(**kwargs):
"""Public facing object that mimic's Flake8 2.0's API. r"""Provision a StyleGuide for use.
.. note:: :param \*\*kwargs:
Keyword arguments that provide some options for the StyleGuide.
There are important changes in how this object behaves compared to :returns:
the object provided in Flake8 2.x. An initialized StyleGuide
:rtype:
.. warning:: :class:`StyleGuide`
This should not be instantiated by users.
.. versionchanged:: 3.0.0
""" """
application = app.Application()
prelim_opts, remaining_args = application.parse_preliminary_options([])
flake8.configure_logging(prelim_opts.verbose, prelim_opts.output_file)
config_finder = config.ConfigFileFinder(
application.program,
prelim_opts.append_config,
config_file=prelim_opts.config,
ignore_config_files=prelim_opts.isolated,
)
def __init__(self, application: app.Application) -> None: application.find_plugins(config_finder)
"""Initialize the Report for the user. application.register_plugin_options()
application.parse_configuration_and_cli(
.. warning:: This should not be instantiated by users. config_finder,
""" remaining_args,
assert application.guide is not None )
self._application = application # We basically want application.initialize to be called but with these
self._style_guide = application.guide # options set instead before we make our formatter, notifier, internal
self._stats = self._style_guide.stats # style guide and file checker manager.
options = application.options
@property for key, value in kwargs.items():
def total_errors(self) -> int: try:
"""Return the total number of errors.""" getattr(options, key)
return self._application.result_count setattr(options, key, value)
except AttributeError:
def get_statistics(self, violation: str) -> list[str]: LOG.error('Could not update option "%s"', key)
"""Get the list of occurrences of a violation. application.make_formatter()
application.make_guide()
:returns: application.make_file_checker_manager()
List of occurrences of a violation formatted as: return StyleGuide(application)
{Count} {Error Code} {Message}, e.g.,
``8 E531 Some error message about the error``
"""
return [
f"{s.count} {s.error_code} {s.message}"
for s in self._stats.statistics_for(violation)
]
class StyleGuide: class StyleGuide:
@ -80,7 +75,7 @@ class StyleGuide:
.. versionchanged:: 3.0.0 .. versionchanged:: 3.0.0
""" """
def __init__(self, application: app.Application) -> None: def __init__(self, application):
"""Initialize our StyleGuide.""" """Initialize our StyleGuide."""
self._application = application self._application = application
self._file_checker_manager = application.file_checker_manager self._file_checker_manager = application.file_checker_manager
@ -91,126 +86,130 @@ class StyleGuide:
An instance of :class:`argparse.Namespace` containing parsed options. An instance of :class:`argparse.Namespace` containing parsed options.
""" """
assert self._application.options is not None
return self._application.options return self._application.options
@property @property
def paths(self) -> list[str]: def paths(self):
"""Return the extra arguments passed as paths.""" """Return the extra arguments passed as paths."""
assert self._application.options is not None return self._application.paths
return self._application.options.filenames
def check_files(self, paths: list[str] | None = None) -> Report: def check_files(self, paths=None):
"""Run collected checks on the files provided. """Run collected checks on the files provided.
This will check the files passed in and return a :class:`Report` This will check the files passed in and return a :class:`Report`
instance. instance.
:param paths: :param list paths:
List of filenames (or paths) to check. List of filenames (or paths) to check.
:returns: :returns:
Object that mimic's Flake8 2.0's Reporter class. Object that mimic's Flake8 2.0's Reporter class.
:rtype:
flake8.api.legacy.Report
""" """
assert self._application.options is not None self._application.run_checks(paths)
self._application.options.filenames = paths
self._application.run_checks()
self._application.report_errors() self._application.report_errors()
return Report(self._application) return Report(self._application)
def excluded(self, filename: str, parent: str | None = None) -> bool: def excluded(self, filename, parent=None):
"""Determine if a file is excluded. """Determine if a file is excluded.
:param filename: :param str filename:
Path to the file to check if it is excluded. Path to the file to check if it is excluded.
:param parent: :param str parent:
Name of the parent directory containing the file. Name of the parent directory containing the file.
:returns: :returns:
True if the filename is excluded, False otherwise. True if the filename is excluded, False otherwise.
:rtype:
bool
""" """
return self._file_checker_manager.is_path_excluded(filename) or (
def excluded(path: str) -> bool: parent
paths = tuple( and self._file_checker_manager.is_path_excluded(
expand_paths( os.path.join(parent, filename)
paths=[path],
stdin_display_name=self.options.stdin_display_name,
filename_patterns=self.options.filename,
exclude=self.options.exclude,
),
) )
return not paths
return excluded(filename) or (
parent is not None and excluded(os.path.join(parent, filename))
) )
def init_report( def init_report(self, reporter=None):
self,
reporter: type[formatter.BaseFormatter] | None = None,
) -> None:
"""Set up a formatter for this run of Flake8.""" """Set up a formatter for this run of Flake8."""
if reporter is None: if reporter is None:
return return
if not issubclass(reporter, formatter.BaseFormatter): if not issubclass(reporter, formatter.BaseFormatter):
raise ValueError( raise ValueError(
"Report should be subclass of " "Report should be subclass of "
"flake8.formatter.BaseFormatter.", "flake8.formatter.BaseFormatter."
) )
self._application.formatter = reporter(self.options) self._application.formatter = None
self._application.make_formatter(reporter)
self._application.guide = None self._application.guide = None
# NOTE(sigmavirus24): This isn't the intended use of # NOTE(sigmavirus24): This isn't the intended use of
# Application#make_guide but it works pretty well. # Application#make_guide but it works pretty well.
# Stop cringing... I know it's gross. # Stop cringing... I know it's gross.
self._application.make_guide() self._application.make_guide()
self._application.file_checker_manager = None self._application.file_checker_manager = None
self._application.make_file_checker_manager([]) self._application.make_file_checker_manager()
def input_file( def input_file(self, filename, lines=None, expected=None, line_offset=0):
self,
filename: str,
lines: Any | None = None,
expected: Any | None = None,
line_offset: Any | None = 0,
) -> Report:
"""Run collected checks on a single file. """Run collected checks on a single file.
This will check the file passed in and return a :class:`Report` This will check the file passed in and return a :class:`Report`
instance. instance.
:param filename: :param str filename:
The path to the file to check. The path to the file to check.
:param lines: :param list lines:
Ignored since Flake8 3.0. Ignored since Flake8 3.0.
:param expected: :param expected:
Ignored since Flake8 3.0. Ignored since Flake8 3.0.
:param line_offset: :param int line_offset:
Ignored since Flake8 3.0. Ignored since Flake8 3.0.
:returns: :returns:
Object that mimic's Flake8 2.0's Reporter class. Object that mimic's Flake8 2.0's Reporter class.
:rtype:
flake8.api.legacy.Report
""" """
return self.check_files([filename]) return self.check_files([filename])
def get_style_guide(**kwargs: Any) -> StyleGuide: class Report:
r"""Provision a StyleGuide for use. """Public facing object that mimic's Flake8 2.0's API.
:param \*\*kwargs: .. note::
Keyword arguments that provide some options for the StyleGuide.
:returns: There are important changes in how this object behaves compared to
An initialized StyleGuide the object provided in Flake8 2.x.
.. warning::
This should not be instantiated by users.
.. versionchanged:: 3.0.0
""" """
application = app.Application()
application.plugins, application.options = parse_args([]) def __init__(self, application):
# We basically want application.initialize to be called but with these """Initialize the Report for the user.
# options set instead before we make our formatter, notifier, internal
# style guide and file checker manager. .. warning:: This should not be instantiated by users.
options = application.options """
for key, value in kwargs.items(): self._application = application
try: self._style_guide = application.guide
getattr(options, key) self._stats = self._style_guide.stats
setattr(options, key, value)
except AttributeError: @property
LOG.error('Could not update option "%s"', key) def total_errors(self):
application.make_formatter() """Return the total number of errors."""
application.make_guide() return self._application.result_count
application.make_file_checker_manager([])
return StyleGuide(application) def get_statistics(self, violation):
"""Get the list of occurrences of a violation.
:returns:
List of occurrences of a violation formatted as:
{Count} {Error Code} {Message}, e.g.,
``8 E531 Some error message about the error``
:rtype:
list
"""
return [
f"{s.count} {s.error_code} {s.message}"
for s in self._stats.statistics_for(violation)
]

View file

@ -1,39 +1,33 @@
"""Checker Manager and Checker classes.""" """Checker Manager and Checker classes."""
from __future__ import annotations import collections
import argparse
import contextlib
import errno import errno
import itertools
import logging import logging
import multiprocessing.pool
import operator
import signal import signal
import tokenize import tokenize
from collections.abc import Generator from typing import Dict
from collections.abc import Sequence from typing import List
from typing import Any
from typing import Optional from typing import Optional
from typing import Tuple
from flake8 import defaults from flake8 import defaults
from flake8 import exceptions from flake8 import exceptions
from flake8 import processor from flake8 import processor
from flake8 import utils from flake8 import utils
from flake8._compat import FSTRING_START
from flake8._compat import TSTRING_START
from flake8.discover_files import expand_paths
from flake8.options.parse_args import parse_args
from flake8.plugins.finder import Checkers
from flake8.plugins.finder import LoadedPlugin
from flake8.style_guide import StyleGuideManager
Results = list[tuple[str, int, int, str, Optional[str]]] try:
import multiprocessing.pool
except ImportError:
multiprocessing = None # type: ignore
Results = List[Tuple[str, int, int, str, Optional[str]]]
LOG = logging.getLogger(__name__) LOG = logging.getLogger(__name__)
SERIAL_RETRY_ERRNOS = { SERIAL_RETRY_ERRNOS = {
# ENOSPC: Added by sigmavirus24 # ENOSPC: Added by sigmavirus24
# > On some operating systems (OSX), multiprocessing may cause an # > On some operating systems (OSX), multiprocessing may cause an
# > ENOSPC error while trying to create a Semaphore. # > ENOSPC error while trying to trying to create a Semaphore.
# > In those cases, we should replace the customized Queue Report # > In those cases, we should replace the customized Queue Report
# > class with pep8's StandardReport class to ensure users don't run # > class with pep8's StandardReport class to ensure users don't run
# > into this problem. # > into this problem.
@ -45,40 +39,10 @@ SERIAL_RETRY_ERRNOS = {
# noise in diffs. # noise in diffs.
} }
_mp: tuple[Checkers, argparse.Namespace] | None = None
def _multiprocessing_is_fork(): # type () -> bool
@contextlib.contextmanager """Class state is only preserved when using the `fork` strategy."""
def _mp_prefork( return multiprocessing and multiprocessing.get_start_method() == "fork"
plugins: Checkers, options: argparse.Namespace,
) -> Generator[None]:
# we can save significant startup work w/ `fork` multiprocessing
global _mp
_mp = plugins, options
try:
yield
finally:
_mp = None
def _mp_init(argv: Sequence[str]) -> None:
global _mp
# Ensure correct signaling of ^C using multiprocessing.Pool.
signal.signal(signal.SIGINT, signal.SIG_IGN)
# for `fork` this'll already be set
if _mp is None:
plugins, options = parse_args(argv)
_mp = plugins.checkers, options
def _mp_run(filename: str) -> tuple[str, Results, dict[str, int]]:
assert _mp is not None, _mp
plugins, options = _mp
return FileChecker(
filename=filename, plugins=plugins, options=options,
).run_checks()
class Manager: class Manager:
@ -100,44 +64,71 @@ class Manager:
together and make our output deterministic. together and make our output deterministic.
""" """
def __init__( def __init__(self, style_guide, arguments, checker_plugins):
self, """Initialize our Manager instance.
style_guide: StyleGuideManager,
plugins: Checkers, :param style_guide:
argv: Sequence[str], The instantiated style guide for this instance of Flake8.
) -> None: :type style_guide:
"""Initialize our Manager instance.""" flake8.style_guide.StyleGuide
:param list arguments:
The extra arguments parsed from the CLI (if any)
:param checker_plugins:
The plugins representing checks parsed from entry-points.
:type checker_plugins:
flake8.plugins.manager.Checkers
"""
self.arguments = arguments
self.style_guide = style_guide self.style_guide = style_guide
self.options = style_guide.options self.options = style_guide.options
self.plugins = plugins self.checks = checker_plugins
self.jobs = self._job_count() self.jobs = self._job_count()
self._all_checkers: List[FileChecker] = []
self.checkers: List[FileChecker] = []
self.statistics = { self.statistics = {
"files": 0, "files": 0,
"logical lines": 0, "logical lines": 0,
"physical lines": 0, "physical lines": 0,
"tokens": 0, "tokens": 0,
} }
self.exclude = (*self.options.exclude, *self.options.extend_exclude) self.exclude = tuple(
self.argv = argv itertools.chain(self.options.exclude, self.options.extend_exclude)
self.results: list[tuple[str, Results, dict[str, int]]] = [] )
def _process_statistics(self) -> None: def _process_statistics(self):
for _, _, statistics in self.results: for checker in self.checkers:
for statistic in defaults.STATISTIC_NAMES: for statistic in defaults.STATISTIC_NAMES:
self.statistics[statistic] += statistics[statistic] self.statistics[statistic] += checker.statistics[statistic]
self.statistics["files"] += len(self.filenames) self.statistics["files"] += len(self.checkers)
def _job_count(self) -> int: def _job_count(self) -> int:
# First we walk through all of our error cases: # First we walk through all of our error cases:
# - multiprocessing library is not present # - multiprocessing library is not present
# - we're running on windows in which case we know we have significant
# implementation issues
# - the user provided stdin and that's not something we can handle # - the user provided stdin and that's not something we can handle
# well # well
# - we're processing a diff, which again does not work well with
# multiprocessing and which really shouldn't require multiprocessing
# - the user provided some awful input # - the user provided some awful input
if not _multiprocessing_is_fork():
LOG.warning(
"The multiprocessing module is not available. "
"Ignoring --jobs arguments."
)
return 0
if utils.is_using_stdin(self.options.filenames): if utils.is_using_stdin(self.arguments):
LOG.warning( LOG.warning(
"The --jobs option is not compatible with supplying " "The --jobs option is not compatible with supplying "
"input using - . Ignoring --jobs arguments.", "input using - . Ignoring --jobs arguments."
)
return 0
if self.options.diff:
LOG.warning(
"The --diff option was specified with --jobs but "
"they are not compatible. Ignoring --jobs arguments."
) )
return 0 return 0
@ -157,10 +148,10 @@ class Manager:
# it to an integer # it to an integer
return jobs.n_jobs return jobs.n_jobs
def _handle_results(self, filename: str, results: Results) -> int: def _handle_results(self, filename, results):
style_guide = self.style_guide style_guide = self.style_guide
reported_results_count = 0 reported_results_count = 0
for error_code, line_number, column, text, physical_line in results: for (error_code, line_number, column, text, physical_line) in results:
reported_results_count += style_guide.handle_error( reported_results_count += style_guide.handle_error(
code=error_code, code=error_code,
filename=filename, filename=filename,
@ -171,7 +162,75 @@ class Manager:
) )
return reported_results_count return reported_results_count
def report(self) -> tuple[int, int]: def is_path_excluded(self, path: str) -> bool:
"""Check if a path is excluded.
:param str path:
Path to check against the exclude patterns.
:returns:
True if there are exclude patterns and the path matches,
otherwise False.
:rtype:
bool
"""
if path == "-":
if self.options.stdin_display_name == "stdin":
return False
path = self.options.stdin_display_name
return utils.matches_filename(
path,
patterns=self.exclude,
log_message='"%(path)s" has %(whether)sbeen excluded',
logger=LOG,
)
def make_checkers(self, paths: Optional[List[str]] = None) -> None:
"""Create checkers for each file."""
if paths is None:
paths = self.arguments
if not paths:
paths = ["."]
filename_patterns = self.options.filename
running_from_diff = self.options.diff
# NOTE(sigmavirus24): Yes this is a little unsightly, but it's our
# best solution right now.
def should_create_file_checker(filename, argument):
"""Determine if we should create a file checker."""
matches_filename_patterns = utils.fnmatch(
filename, filename_patterns
)
is_stdin = filename == "-"
# NOTE(sigmavirus24): If a user explicitly specifies something,
# e.g, ``flake8 bin/script`` then we should run Flake8 against
# that. Since should_create_file_checker looks to see if the
# filename patterns match the filename, we want to skip that in
# the event that the argument and the filename are identical.
# If it was specified explicitly, the user intended for it to be
# checked.
explicitly_provided = not running_from_diff and (
argument == filename
)
return (
explicitly_provided or matches_filename_patterns
) or is_stdin
checks = self.checks.to_dictionary()
self._all_checkers = [
FileChecker(filename, checks, self.options)
for argument in paths
for filename in utils.filenames_from(
argument, self.is_path_excluded
)
if should_create_file_checker(filename, argument)
]
self.checkers = [c for c in self._all_checkers if c.should_process]
LOG.info("Checking %d files", len(self.checkers))
def report(self) -> Tuple[int, int]:
"""Report all of the errors found in the managed file checkers. """Report all of the errors found in the managed file checkers.
This iterates over each of the checkers and reports the errors sorted This iterates over each of the checkers and reports the errors sorted
@ -179,11 +238,13 @@ class Manager:
:returns: :returns:
A tuple of the total results found and the results reported. A tuple of the total results found and the results reported.
:rtype:
tuple(int, int)
""" """
results_reported = results_found = 0 results_reported = results_found = 0
self.results.sort(key=operator.itemgetter(0)) for checker in self._all_checkers:
for filename, results, _ in self.results: results = sorted(checker.results, key=lambda tup: (tup[1], tup[2]))
results.sort(key=operator.itemgetter(1, 2)) filename = checker.display_name
with self.style_guide.processing_file(filename): with self.style_guide.processing_file(filename):
results_reported += self._handle_results(filename, results) results_reported += self._handle_results(filename, results)
results_found += len(results) results_found += len(results)
@ -191,8 +252,12 @@ class Manager:
def run_parallel(self) -> None: def run_parallel(self) -> None:
"""Run the checkers in parallel.""" """Run the checkers in parallel."""
with _mp_prefork(self.plugins, self.options): # fmt: off
pool = _try_initialize_processpool(self.jobs, self.argv) final_results: Dict[str, List[Tuple[str, int, int, str, Optional[str]]]] = collections.defaultdict(list) # noqa: E501
final_statistics: Dict[str, Dict[str, int]] = collections.defaultdict(dict) # noqa: E501
# fmt: on
pool = _try_initialize_processpool(self.jobs)
if pool is None: if pool is None:
self.run_serial() self.run_serial()
@ -200,7 +265,17 @@ class Manager:
pool_closed = False pool_closed = False
try: try:
self.results = list(pool.imap_unordered(_mp_run, self.filenames)) pool_map = pool.imap_unordered(
_run_checks,
self.checkers,
chunksize=calculate_pool_chunksize(
len(self.checkers), self.jobs
),
)
for ret in pool_map:
filename, results, statistics = ret
final_results[filename] = results
final_statistics[filename] = statistics
pool.close() pool.close()
pool.join() pool.join()
pool_closed = True pool_closed = True
@ -209,16 +284,15 @@ class Manager:
pool.terminate() pool.terminate()
pool.join() pool.join()
for checker in self.checkers:
filename = checker.display_name
checker.results = final_results[filename]
checker.statistics = final_statistics[filename]
def run_serial(self) -> None: def run_serial(self) -> None:
"""Run the checkers in serial.""" """Run the checkers in serial."""
self.results = [ for checker in self.checkers:
FileChecker( checker.run_checks()
filename=filename,
plugins=self.plugins,
options=self.options,
).run_checks()
for filename in self.filenames
]
def run(self) -> None: def run(self) -> None:
"""Run all the checkers. """Run all the checkers.
@ -227,10 +301,11 @@ class Manager:
or whether to run them in serial. or whether to run them in serial.
If running the checks in parallel causes a problem (e.g., If running the checks in parallel causes a problem (e.g.,
:issue:`117`) this also implements fallback to serial processing. https://github.com/pycqa/flake8/issues/117) this also implements
fallback to serial processing.
""" """
try: try:
if self.jobs > 1 and len(self.filenames) > 1: if self.jobs > 1 and len(self.checkers) > 1:
self.run_parallel() self.run_parallel()
else: else:
self.run_serial() self.run_serial()
@ -238,25 +313,17 @@ class Manager:
LOG.warning("Flake8 was interrupted by the user") LOG.warning("Flake8 was interrupted by the user")
raise exceptions.EarlyQuit("Early quit while running checks") raise exceptions.EarlyQuit("Early quit while running checks")
def start(self) -> None: def start(self, paths=None):
"""Start checking files. """Start checking files.
:param paths: :param list paths:
Path names to check. This is passed directly to Path names to check. This is passed directly to
:meth:`~Manager.make_checkers`. :meth:`~Manager.make_checkers`.
""" """
LOG.info("Making checkers") LOG.info("Making checkers")
self.filenames = tuple( self.make_checkers(paths)
expand_paths(
paths=self.options.filenames,
stdin_display_name=self.options.stdin_display_name,
filename_patterns=self.options.filename,
exclude=self.exclude,
),
)
self.jobs = min(len(self.filenames), self.jobs)
def stop(self) -> None: def stop(self):
"""Stop checking files.""" """Stop checking files."""
self._process_statistics() self._process_statistics()
@ -264,17 +331,23 @@ class Manager:
class FileChecker: class FileChecker:
"""Manage running checks for a file and aggregate the results.""" """Manage running checks for a file and aggregate the results."""
def __init__( def __init__(self, filename, checks, options):
self, """Initialize our file checker.
*,
filename: str, :param str filename:
plugins: Checkers, Name of the file to check.
options: argparse.Namespace, :param checks:
) -> None: The plugins registered to check the file.
"""Initialize our file checker.""" :type checks:
dict
:param options:
Parsed option values from config and command-line.
:type options:
argparse.Namespace
"""
self.options = options self.options = options
self.filename = filename self.filename = filename
self.plugins = plugins self.checks = checks
self.results: Results = [] self.results: Results = []
self.statistics = { self.statistics = {
"tokens": 0, "tokens": 0,
@ -293,7 +366,7 @@ class FileChecker:
"""Provide helpful debugging representation.""" """Provide helpful debugging representation."""
return f"FileChecker for {self.filename}" return f"FileChecker for {self.filename}"
def _make_processor(self) -> processor.FileProcessor | None: def _make_processor(self) -> Optional[processor.FileProcessor]:
try: try:
return processor.FileProcessor(self.filename, self.options) return processor.FileProcessor(self.filename, self.options)
except OSError as e: except OSError as e:
@ -308,7 +381,7 @@ class FileChecker:
def report( def report(
self, self,
error_code: str | None, error_code: Optional[str],
line_number: int, line_number: int,
column: int, column: int,
text: str, text: str,
@ -327,33 +400,33 @@ class FileChecker:
self.results.append((error_code, line_number, column, text, line)) self.results.append((error_code, line_number, column, text, line))
return error_code return error_code
def run_check(self, plugin: LoadedPlugin, **arguments: Any) -> Any: def run_check(self, plugin, **arguments):
"""Run the check in a single plugin.""" """Run the check in a single plugin."""
assert self.processor is not None, self.filename LOG.debug("Running %r with %r", plugin, arguments)
assert self.processor is not None
try: try:
params = self.processor.keyword_arguments_for( self.processor.keyword_arguments_for(
plugin.parameters, arguments, plugin["parameters"], arguments
) )
except AttributeError as ae: except AttributeError as ae:
LOG.error("Plugin requested unknown parameters.")
raise exceptions.PluginRequestedUnknownParameters( raise exceptions.PluginRequestedUnknownParameters(
plugin_name=plugin.display_name, exception=ae, plugin=plugin, exception=ae
) )
try: try:
return plugin.obj(**arguments, **params) return plugin["plugin"](**arguments)
except Exception as all_exc: except Exception as all_exc:
LOG.critical( LOG.critical(
"Plugin %s raised an unexpected exception", "Plugin %s raised an unexpected exception",
plugin.display_name, plugin["name"],
exc_info=True, exc_info=True,
) )
raise exceptions.PluginExecutionFailed( raise exceptions.PluginExecutionFailed(
filename=self.filename, plugin=plugin, exception=all_exc
plugin_name=plugin.display_name,
exception=all_exc,
) )
@staticmethod @staticmethod
def _extract_syntax_information(exception: Exception) -> tuple[int, int]: def _extract_syntax_information(exception: Exception) -> Tuple[int, int]:
if ( if (
len(exception.args) > 1 len(exception.args) > 1
and exception.args[1] and exception.args[1]
@ -372,14 +445,51 @@ class FileChecker:
token = () token = ()
row, column = (1, 0) row, column = (1, 0)
if (
column > 0
and token
and isinstance(exception, SyntaxError)
and len(token) == 4 # Python 3.9 or earlier
):
# NOTE(sigmavirus24): SyntaxErrors report 1-indexed column
# numbers. We need to decrement the column number by 1 at
# least.
column_offset = 1
row_offset = 0
# See also: https://github.com/pycqa/flake8/issues/169,
# https://github.com/PyCQA/flake8/issues/1372
# On Python 3.9 and earlier, token will be a 4-item tuple with the
# last item being the string. Starting with 3.10, they added to
# the tuple so now instead of it ending with the code that failed
# to parse, it ends with the end of the section of code that
# failed to parse. Luckily the absolute position in the tuple is
# stable across versions so we can use that here
physical_line = token[3]
# NOTE(sigmavirus24): Not all "tokens" have a string as the last
# argument. In this event, let's skip trying to find the correct
# column and row values.
if physical_line is not None:
# NOTE(sigmavirus24): SyntaxErrors also don't exactly have a
# "physical" line so much as what was accumulated by the point
# tokenizing failed.
# See also: https://github.com/pycqa/flake8/issues/169
lines = physical_line.rstrip("\n").split("\n")
row_offset = len(lines) - 1
logical_line = lines[0]
logical_line_length = len(logical_line)
if column > logical_line_length:
column = logical_line_length
row -= row_offset
column -= column_offset
return row, column return row, column
def run_ast_checks(self) -> None: def run_ast_checks(self) -> None:
"""Run all checks expecting an abstract syntax tree.""" """Run all checks expecting an abstract syntax tree."""
assert self.processor is not None, self.filename assert self.processor is not None
ast = self.processor.build_ast() ast = self.processor.build_ast()
for plugin in self.plugins.tree: for plugin in self.checks["ast_plugins"]:
checker = self.run_check(plugin, tree=ast) checker = self.run_check(plugin, tree=ast)
# If the plugin uses a class, call the run method of it, otherwise # If the plugin uses a class, call the run method of it, otherwise
# the call should return something iterable itself # the call should return something iterable itself
@ -387,7 +497,7 @@ class FileChecker:
runner = checker.run() runner = checker.run()
except AttributeError: except AttributeError:
runner = checker runner = checker
for line_number, offset, text, _ in runner: for (line_number, offset, text, _) in runner:
self.report( self.report(
error_code=None, error_code=None,
line_number=line_number, line_number=line_number,
@ -395,7 +505,7 @@ class FileChecker:
text=text, text=text,
) )
def run_logical_checks(self) -> None: def run_logical_checks(self):
"""Run all checks expecting a logical line.""" """Run all checks expecting a logical line."""
assert self.processor is not None assert self.processor is not None
comments, logical_line, mapping = self.processor.build_logical_line() comments, logical_line, mapping = self.processor.build_logical_line()
@ -405,7 +515,7 @@ class FileChecker:
LOG.debug('Logical line: "%s"', logical_line.rstrip()) LOG.debug('Logical line: "%s"', logical_line.rstrip())
for plugin in self.plugins.logical_line: for plugin in self.checks["logical_line_plugins"]:
self.processor.update_checker_state_for(plugin) self.processor.update_checker_state_for(plugin)
results = self.run_check(plugin, logical_line=logical_line) or () results = self.run_check(plugin, logical_line=logical_line) or ()
for offset, text in results: for offset, text in results:
@ -421,13 +531,13 @@ class FileChecker:
self.processor.next_logical_line() self.processor.next_logical_line()
def run_physical_checks(self, physical_line: str) -> None: def run_physical_checks(self, physical_line):
"""Run all checks for a given physical line. """Run all checks for a given physical line.
A single physical check may return multiple errors. A single physical check may return multiple errors.
""" """
assert self.processor is not None assert self.processor is not None
for plugin in self.plugins.physical_line: for plugin in self.checks["physical_line_plugins"]:
self.processor.update_checker_state_for(plugin) self.processor.update_checker_state_for(plugin)
result = self.run_check(plugin, physical_line=physical_line) result = self.run_check(plugin, physical_line=physical_line)
@ -452,7 +562,7 @@ class FileChecker:
text=text, text=text,
) )
def process_tokens(self) -> None: def process_tokens(self):
"""Process tokens and trigger checks. """Process tokens and trigger checks.
Instead of using this directly, you should use Instead of using this directly, you should use
@ -467,6 +577,7 @@ class FileChecker:
statistics["tokens"] += 1 statistics["tokens"] += 1
self.check_physical_eol(token, prev_physical) self.check_physical_eol(token, prev_physical)
token_type, text = token[0:2] token_type, text = token[0:2]
processor.log_token(LOG, token)
if token_type == tokenize.OP: if token_type == tokenize.OP:
parens = processor.count_parentheses(parens, text) parens = processor.count_parentheses(parens, text)
elif parens == 0: elif parens == 0:
@ -479,11 +590,9 @@ class FileChecker:
self.run_physical_checks(file_processor.lines[-1]) self.run_physical_checks(file_processor.lines[-1])
self.run_logical_checks() self.run_logical_checks()
def run_checks(self) -> tuple[str, Results, dict[str, int]]: def run_checks(self) -> Tuple[str, Results, Dict[str, int]]:
"""Run checks against the file.""" """Run checks against the file."""
if self.processor is None or not self.should_process: assert self.processor is not None
return self.display_name, self.results, self.statistics
try: try:
self.run_ast_checks() self.run_ast_checks()
self.process_tokens() self.process_tokens()
@ -491,13 +600,13 @@ class FileChecker:
code = "E902" if isinstance(e, tokenize.TokenError) else "E999" code = "E902" if isinstance(e, tokenize.TokenError) else "E999"
row, column = self._extract_syntax_information(e) row, column = self._extract_syntax_information(e)
self.report(code, row, column, f"{type(e).__name__}: {e.args[0]}") self.report(code, row, column, f"{type(e).__name__}: {e.args[0]}")
return self.display_name, self.results, self.statistics return self.filename, self.results, self.statistics
logical_lines = self.processor.statistics["logical lines"] logical_lines = self.processor.statistics["logical lines"]
self.statistics["logical lines"] = logical_lines self.statistics["logical lines"] = logical_lines
return self.display_name, self.results, self.statistics return self.filename, self.results, self.statistics
def handle_newline(self, token_type: int) -> None: def handle_newline(self, token_type):
"""Handle the logic when encountering a newline token.""" """Handle the logic when encountering a newline token."""
assert self.processor is not None assert self.processor is not None
if token_type == tokenize.NEWLINE: if token_type == tokenize.NEWLINE:
@ -511,23 +620,19 @@ class FileChecker:
self.run_logical_checks() self.run_logical_checks()
def check_physical_eol( def check_physical_eol(
self, token: tokenize.TokenInfo, prev_physical: str, self, token: processor._Token, prev_physical: str
) -> None: ) -> None:
"""Run physical checks if and only if it is at the end of the line.""" """Run physical checks if and only if it is at the end of the line."""
assert self.processor is not None assert self.processor is not None
if token.type == FSTRING_START: # pragma: >=3.12 cover
self.processor.fstring_start(token.start[0])
elif token.type == TSTRING_START: # pragma: >=3.14 cover
self.processor.tstring_start(token.start[0])
# a newline token ends a single physical line. # a newline token ends a single physical line.
elif processor.is_eol_token(token): if processor.is_eol_token(token):
# if the file does not end with a newline, the NEWLINE # if the file does not end with a newline, the NEWLINE
# token is inserted by the parser, but it does not contain # token is inserted by the parser, but it does not contain
# the previous physical line in `token[4]` # the previous physical line in `token[4]`
if token.line == "": if token[4] == "":
self.run_physical_checks(prev_physical) self.run_physical_checks(prev_physical)
else: else:
self.run_physical_checks(token.line) self.run_physical_checks(token[4])
elif processor.is_multiline_string(token): elif processor.is_multiline_string(token):
# Less obviously, a string that contains newlines is a # Less obviously, a string that contains newlines is a
# multiline string, either triple-quoted or with internal # multiline string, either triple-quoted or with internal
@ -540,17 +645,23 @@ class FileChecker:
# - have to wind self.line_number back because initially it # - have to wind self.line_number back because initially it
# points to the last line of the string, and we want # points to the last line of the string, and we want
# check_physical() to give accurate feedback # check_physical() to give accurate feedback
for line in self.processor.multiline_string(token): line_no = token[2][0]
self.run_physical_checks(line) with self.processor.inside_multiline(line_number=line_no):
for line in self.processor.split_line(token):
self.run_physical_checks(line + "\n")
def _pool_init() -> None:
"""Ensure correct signaling of ^C using multiprocessing.Pool."""
signal.signal(signal.SIGINT, signal.SIG_IGN)
def _try_initialize_processpool( def _try_initialize_processpool(
job_count: int, job_count: int,
argv: Sequence[str], ) -> Optional[multiprocessing.pool.Pool]:
) -> multiprocessing.pool.Pool | None:
"""Return a new process pool instance if we are able to create one.""" """Return a new process pool instance if we are able to create one."""
try: try:
return multiprocessing.Pool(job_count, _mp_init, initargs=(argv,)) return multiprocessing.Pool(job_count, _pool_init)
except OSError as err: except OSError as err:
if err.errno not in SERIAL_RETRY_ERRNOS: if err.errno not in SERIAL_RETRY_ERRNOS:
raise raise
@ -560,9 +671,25 @@ def _try_initialize_processpool(
return None return None
def calculate_pool_chunksize(num_checkers, num_jobs):
"""Determine the chunksize for the multiprocessing Pool.
- For chunksize, see: https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool.imap # noqa
- This formula, while not perfect, aims to give each worker two batches of
work.
- See: https://github.com/pycqa/flake8/issues/829#note_18878876
- See: https://github.com/pycqa/flake8/issues/197
"""
return max(num_checkers // (num_jobs * 2), 1)
def _run_checks(checker):
return checker.run_checks()
def find_offset( def find_offset(
offset: int, mapping: processor._LogicalMapping, offset: int, mapping: processor._LogicalMapping
) -> tuple[int, int]: ) -> Tuple[int, int]:
"""Find the offset tuple for a single offset.""" """Find the offset tuple for a single offset."""
if isinstance(offset, tuple): if isinstance(offset, tuple):
return offset return offset

View file

@ -1,6 +1,4 @@
"""Constants that define defaults.""" """Constants that define defaults."""
from __future__ import annotations
import re import re
EXCLUDE = ( EXCLUDE = (
@ -11,14 +9,16 @@ EXCLUDE = (
".git", ".git",
"__pycache__", "__pycache__",
".tox", ".tox",
".nox",
".eggs", ".eggs",
"*.egg", "*.egg",
) )
IGNORE = ("E121", "E123", "E126", "E226", "E24", "E704", "W503", "W504") IGNORE = ("E121", "E123", "E126", "E226", "E24", "E704", "W503", "W504")
SELECT = ("E", "F", "W", "C90")
MAX_LINE_LENGTH = 79 MAX_LINE_LENGTH = 79
INDENT_SIZE = 4 INDENT_SIZE = 4
TRUTHY_VALUES = {"true", "1", "t"}
# Other constants # Other constants
WHITESPACE = frozenset(" \t") WHITESPACE = frozenset(" \t")
@ -36,10 +36,9 @@ NOQA_INLINE_REGEXP = re.compile(
# We do not want to capture the ``: `` that follows ``noqa`` # We do not want to capture the ``: `` that follows ``noqa``
# We do not care about the casing of ``noqa`` # We do not care about the casing of ``noqa``
# We want a comma-separated list of errors # We want a comma-separated list of errors
# https://regex101.com/r/4XUuax/2 full explanation of the regex
r"# noqa(?::[\s]?(?P<codes>([A-Z]+[0-9]+(?:[,\s]+)?)+))?", r"# noqa(?::[\s]?(?P<codes>([A-Z]+[0-9]+(?:[,\s]+)?)+))?",
re.IGNORECASE, re.IGNORECASE,
) )
NOQA_FILE = re.compile(r"\s*# flake8[:=]\s*noqa", re.I) NOQA_FILE = re.compile(r"\s*# flake8[:=]\s*noqa", re.I)
VALID_CODE_PREFIX = re.compile("^[A-Z]{1,3}[0-9]{0,3}$", re.ASCII)

View file

@ -1,89 +0,0 @@
"""Functions related to discovering paths."""
from __future__ import annotations
import logging
import os.path
from collections.abc import Callable
from collections.abc import Generator
from collections.abc import Sequence
from flake8 import utils
LOG = logging.getLogger(__name__)
def _filenames_from(
arg: str,
*,
predicate: Callable[[str], bool],
) -> Generator[str]:
"""Generate filenames from an argument.
:param arg:
Parameter from the command-line.
:param predicate:
Predicate to use to filter out filenames. If the predicate
returns ``True`` we will exclude the filename, otherwise we
will yield it. By default, we include every filename
generated.
:returns:
Generator of paths
"""
if predicate(arg):
return
if os.path.isdir(arg):
for root, sub_directories, files in os.walk(arg):
# NOTE(sigmavirus24): os.walk() will skip a directory if you
# remove it from the list of sub-directories.
for directory in tuple(sub_directories):
joined = os.path.join(root, directory)
if predicate(joined):
sub_directories.remove(directory)
for filename in files:
joined = os.path.join(root, filename)
if not predicate(joined):
yield joined
else:
yield arg
def expand_paths(
*,
paths: Sequence[str],
stdin_display_name: str,
filename_patterns: Sequence[str],
exclude: Sequence[str],
) -> Generator[str]:
"""Expand out ``paths`` from commandline to the lintable files."""
if not paths:
paths = ["."]
def is_excluded(arg: str) -> bool:
if arg == "-":
# if the stdin_display_name is the default, always include it
if stdin_display_name == "stdin":
return False
arg = stdin_display_name
return utils.matches_filename(
arg,
patterns=exclude,
log_message='"%(path)s" has %(whether)sbeen excluded',
logger=LOG,
)
return (
filename
for path in paths
for filename in _filenames_from(path, predicate=is_excluded)
if (
# always lint `-`
filename == "-"
# always lint explicitly passed (even if not matching filter)
or path == filename
# otherwise, check the file against filtered patterns
or utils.fnmatch(filename, filename_patterns)
)
)

View file

@ -1,5 +1,5 @@
"""Exception classes for all of Flake8.""" """Exception classes for all of Flake8."""
from __future__ import annotations from typing import Dict
class Flake8Exception(Exception): class Flake8Exception(Exception):
@ -38,16 +38,16 @@ class PluginRequestedUnknownParameters(Flake8Exception):
FORMAT = '"%(name)s" requested unknown parameters causing %(exc)s' FORMAT = '"%(name)s" requested unknown parameters causing %(exc)s'
def __init__(self, plugin_name: str, exception: Exception) -> None: def __init__(self, plugin: Dict[str, str], exception: Exception) -> None:
"""Pop certain keyword arguments for initialization.""" """Pop certain keyword arguments for initialization."""
self.plugin_name = plugin_name self.plugin = plugin
self.original_exception = exception self.original_exception = exception
super().__init__(plugin_name, exception) super().__init__(plugin, exception)
def __str__(self) -> str: def __str__(self) -> str:
"""Format our exception message.""" """Format our exception message."""
return self.FORMAT % { return self.FORMAT % {
"name": self.plugin_name, "name": self.plugin["plugin_name"],
"exc": self.original_exception, "exc": self.original_exception,
} }
@ -55,24 +55,17 @@ class PluginRequestedUnknownParameters(Flake8Exception):
class PluginExecutionFailed(Flake8Exception): class PluginExecutionFailed(Flake8Exception):
"""The plugin failed during execution.""" """The plugin failed during execution."""
FORMAT = '{fname}: "{plugin}" failed during execution due to {exc!r}' FORMAT = '"%(name)s" failed during execution due to "%(exc)s"'
def __init__( def __init__(self, plugin: Dict[str, str], exception: Exception) -> None:
self,
filename: str,
plugin_name: str,
exception: Exception,
) -> None:
"""Utilize keyword arguments for message generation.""" """Utilize keyword arguments for message generation."""
self.filename = filename self.plugin = plugin
self.plugin_name = plugin_name
self.original_exception = exception self.original_exception = exception
super().__init__(filename, plugin_name, exception) super().__init__(plugin, exception)
def __str__(self) -> str: def __str__(self) -> str:
"""Format our exception message.""" """Format our exception message."""
return self.FORMAT.format( return self.FORMAT % {
fname=self.filename, "name": self.plugin["plugin_name"],
plugin=self.plugin_name, "exc": self.original_exception,
exc=self.original_exception, }
)

View file

@ -1,2 +1 @@
"""Submodule containing the default formatters for Flake8.""" """Submodule containing the default formatters for Flake8."""
from __future__ import annotations

View file

@ -1,61 +0,0 @@
"""ctypes hackery to enable color processing on windows.
See: https://github.com/pre-commit/pre-commit/blob/cb40e96/pre_commit/color.py
"""
from __future__ import annotations
import sys
if sys.platform == "win32": # pragma: no cover (windows)
def _enable() -> None:
from ctypes import POINTER
from ctypes import windll
from ctypes import WinError
from ctypes import WINFUNCTYPE
from ctypes.wintypes import BOOL
from ctypes.wintypes import DWORD
from ctypes.wintypes import HANDLE
STD_ERROR_HANDLE = -12
ENABLE_VIRTUAL_TERMINAL_PROCESSING = 4
def bool_errcheck(result, func, args):
if not result:
raise WinError()
return args
GetStdHandle = WINFUNCTYPE(HANDLE, DWORD)(
("GetStdHandle", windll.kernel32),
((1, "nStdHandle"),),
)
GetConsoleMode = WINFUNCTYPE(BOOL, HANDLE, POINTER(DWORD))(
("GetConsoleMode", windll.kernel32),
((1, "hConsoleHandle"), (2, "lpMode")),
)
GetConsoleMode.errcheck = bool_errcheck
SetConsoleMode = WINFUNCTYPE(BOOL, HANDLE, DWORD)(
("SetConsoleMode", windll.kernel32),
((1, "hConsoleHandle"), (1, "dwMode")),
)
SetConsoleMode.errcheck = bool_errcheck
# As of Windows 10, the Windows console supports (some) ANSI escape
# sequences, but it needs to be enabled using `SetConsoleMode` first.
#
# More info on the escape sequences supported:
# https://msdn.microsoft.com/en-us/library/windows/desktop/mt638032(v=vs.85).aspx
stderr = GetStdHandle(STD_ERROR_HANDLE)
flags = GetConsoleMode(stderr)
SetConsoleMode(stderr, flags | ENABLE_VIRTUAL_TERMINAL_PROCESSING)
try:
_enable()
except OSError:
terminal_supports_color = False
else:
terminal_supports_color = True
else: # pragma: win32 no cover
terminal_supports_color = True

View file

@ -1,14 +1,16 @@
"""The base class and interface for all formatting plugins.""" """The base class and interface for all formatting plugins."""
from __future__ import annotations
import argparse import argparse
import os import os
import sys import sys
from typing import IO from typing import IO
from typing import List
from typing import Optional
from typing import Tuple
from typing import TYPE_CHECKING
from flake8.formatting import _windows_color if TYPE_CHECKING:
from flake8.statistics import Statistics from flake8.statistics import Statistics
from flake8.violation import Violation from flake8.style_guide import Violation
class BaseFormatter: class BaseFormatter:
@ -42,16 +44,13 @@ class BaseFormatter:
:param options: :param options:
User specified configuration parsed from both configuration files User specified configuration parsed from both configuration files
and the command-line interface. and the command-line interface.
:type options:
:class:`argparse.Namespace`
""" """
self.options = options self.options = options
self.filename = options.output_file self.filename = options.output_file
self.output_fd: IO[str] | None = None self.output_fd: Optional[IO[str]] = None
self.newline = "\n" self.newline = "\n"
self.color = options.color == "always" or (
options.color == "auto"
and sys.stdout.isatty()
and _windows_color.terminal_supports_color
)
self.after_init() self.after_init()
def after_init(self) -> None: def after_init(self) -> None:
@ -60,7 +59,7 @@ class BaseFormatter:
def beginning(self, filename: str) -> None: def beginning(self, filename: str) -> None:
"""Notify the formatter that we're starting to process a file. """Notify the formatter that we're starting to process a file.
:param filename: :param str filename:
The name of the file that Flake8 is beginning to report results The name of the file that Flake8 is beginning to report results
from. from.
""" """
@ -68,7 +67,7 @@ class BaseFormatter:
def finished(self, filename: str) -> None: def finished(self, filename: str) -> None:
"""Notify the formatter that we've finished processing a file. """Notify the formatter that we've finished processing a file.
:param filename: :param str filename:
The name of the file that Flake8 has finished reporting results The name of the file that Flake8 has finished reporting results
from. from.
""" """
@ -83,7 +82,7 @@ class BaseFormatter:
os.makedirs(dirname, exist_ok=True) os.makedirs(dirname, exist_ok=True)
self.output_fd = open(self.filename, "a") self.output_fd = open(self.filename, "a")
def handle(self, error: Violation) -> None: def handle(self, error: "Violation") -> None:
"""Handle an error reported by Flake8. """Handle an error reported by Flake8.
This defaults to calling :meth:`format`, :meth:`show_source`, and This defaults to calling :meth:`format`, :meth:`show_source`, and
@ -92,28 +91,34 @@ class BaseFormatter:
:param error: :param error:
This will be an instance of This will be an instance of
:class:`~flake8.violation.Violation`. :class:`~flake8.style_guide.Violation`.
:type error:
flake8.style_guide.Violation
""" """
line = self.format(error) line = self.format(error)
source = self.show_source(error) source = self.show_source(error)
self.write(line, source) self.write(line, source)
def format(self, error: Violation) -> str | None: def format(self, error: "Violation") -> Optional[str]:
"""Format an error reported by Flake8. """Format an error reported by Flake8.
This method **must** be implemented by subclasses. This method **must** be implemented by subclasses.
:param error: :param error:
This will be an instance of This will be an instance of
:class:`~flake8.violation.Violation`. :class:`~flake8.style_guide.Violation`.
:type error:
flake8.style_guide.Violation
:returns: :returns:
The formatted error string. The formatted error string.
:rtype:
str
""" """
raise NotImplementedError( raise NotImplementedError(
"Subclass of BaseFormatter did not implement" " format.", "Subclass of BaseFormatter did not implement" " format."
) )
def show_statistics(self, statistics: Statistics) -> None: def show_statistics(self, statistics: "Statistics") -> None:
"""Format and print the statistics.""" """Format and print the statistics."""
for error_code in statistics.error_codes(): for error_code in statistics.error_codes():
stats_for_error_code = statistics.statistics_for(error_code) stats_for_error_code = statistics.statistics_for(error_code)
@ -122,7 +127,7 @@ class BaseFormatter:
count += sum(stat.count for stat in stats_for_error_code) count += sum(stat.count for stat in stats_for_error_code)
self._write(f"{count:<5} {error_code} {statistic.message}") self._write(f"{count:<5} {error_code} {statistic.message}")
def show_benchmarks(self, benchmarks: list[tuple[str, float]]) -> None: def show_benchmarks(self, benchmarks: List[Tuple[str, float]]) -> None:
"""Format and print the benchmarks.""" """Format and print the benchmarks."""
# NOTE(sigmavirus24): The format strings are a little confusing, even # NOTE(sigmavirus24): The format strings are a little confusing, even
# to me, so here's a quick explanation: # to me, so here's a quick explanation:
@ -143,7 +148,7 @@ class BaseFormatter:
benchmark = float_format(statistic=statistic, value=value) benchmark = float_format(statistic=statistic, value=value)
self._write(benchmark) self._write(benchmark)
def show_source(self, error: Violation) -> str | None: def show_source(self, error: "Violation") -> Optional[str]:
"""Show the physical line generating the error. """Show the physical line generating the error.
This also adds an indicator for the particular part of the line that This also adds an indicator for the particular part of the line that
@ -151,11 +156,15 @@ class BaseFormatter:
:param error: :param error:
This will be an instance of This will be an instance of
:class:`~flake8.violation.Violation`. :class:`~flake8.style_guide.Violation`.
:type error:
flake8.style_guide.Violation
:returns: :returns:
The formatted error string if the user wants to show the source. The formatted error string if the user wants to show the source.
If the user does not want to show the source, this will return If the user does not want to show the source, this will return
``None``. ``None``.
:rtype:
str
""" """
if not self.options.show_source or error.physical_line is None: if not self.options.show_source or error.physical_line is None:
return "" return ""
@ -177,16 +186,16 @@ class BaseFormatter:
if self.output_fd is None or self.options.tee: if self.output_fd is None or self.options.tee:
sys.stdout.buffer.write(output.encode() + self.newline.encode()) sys.stdout.buffer.write(output.encode() + self.newline.encode())
def write(self, line: str | None, source: str | None) -> None: def write(self, line: Optional[str], source: Optional[str]) -> None:
"""Write the line either to the output file or stdout. """Write the line either to the output file or stdout.
This handles deciding whether to write to a file or print to standard This handles deciding whether to write to a file or print to standard
out for subclasses. Override this if you want behaviour that differs out for subclasses. Override this if you want behaviour that differs
from the default. from the default.
:param line: :param str line:
The formatted string to print or write. The formatted string to print or write.
:param source: :param str source:
The source code that has been formatted and associated with the The source code that has been formatted and associated with the
line of output. line of output.
""" """

View file

@ -1,22 +1,12 @@
"""Default formatting class for Flake8.""" """Default formatting class for Flake8."""
from __future__ import annotations from typing import Optional
from typing import Set
from typing import TYPE_CHECKING
from flake8.formatting import base from flake8.formatting import base
from flake8.violation import Violation
COLORS = { if TYPE_CHECKING:
"bold": "\033[1m", from flake8.style_guide import Violation
"black": "\033[30m",
"red": "\033[31m",
"green": "\033[32m",
"yellow": "\033[33m",
"blue": "\033[34m",
"magenta": "\033[35m",
"cyan": "\033[36m",
"white": "\033[37m",
"reset": "\033[m",
}
COLORS_OFF = {k: "" for k in COLORS}
class SimpleFormatter(base.BaseFormatter): class SimpleFormatter(base.BaseFormatter):
@ -37,7 +27,7 @@ class SimpleFormatter(base.BaseFormatter):
error_format: str error_format: str
def format(self, error: Violation) -> str | None: def format(self, error: "Violation") -> Optional[str]:
"""Format and write error out. """Format and write error out.
If an output filename is specified, write formatted errors to that If an output filename is specified, write formatted errors to that
@ -49,7 +39,6 @@ class SimpleFormatter(base.BaseFormatter):
"path": error.filename, "path": error.filename,
"row": error.line_number, "row": error.line_number,
"col": error.column_number, "col": error.column_number,
**(COLORS if self.color else COLORS_OFF),
} }
@ -60,11 +49,7 @@ class Default(SimpleFormatter):
format string. format string.
""" """
error_format = ( error_format = "%(path)s:%(row)d:%(col)d: %(code)s %(text)s"
"%(bold)s%(path)s%(reset)s"
"%(cyan)s:%(reset)s%(row)d%(cyan)s:%(reset)s%(col)d%(cyan)s:%(reset)s "
"%(bold)s%(red)s%(code)s%(reset)s %(text)s"
)
def after_init(self) -> None: def after_init(self) -> None:
"""Check for a custom format string.""" """Check for a custom format string."""
@ -85,12 +70,12 @@ class FilenameOnly(SimpleFormatter):
def after_init(self) -> None: def after_init(self) -> None:
"""Initialize our set of filenames.""" """Initialize our set of filenames."""
self.filenames_already_printed: set[str] = set() self.filenames_already_printed: Set[str] = set()
def show_source(self, error: Violation) -> str | None: def show_source(self, error: "Violation") -> Optional[str]:
"""Do not include the source code.""" """Do not include the source code."""
def format(self, error: Violation) -> str | None: def format(self, error: "Violation") -> Optional[str]:
"""Ensure we only print each error once.""" """Ensure we only print each error once."""
if error.filename not in self.filenames_already_printed: if error.filename not in self.filenames_already_printed:
self.filenames_already_printed.add(error.filename) self.filenames_already_printed.add(error.filename)
@ -102,8 +87,8 @@ class FilenameOnly(SimpleFormatter):
class Nothing(base.BaseFormatter): class Nothing(base.BaseFormatter):
"""Print absolutely nothing.""" """Print absolutely nothing."""
def format(self, error: Violation) -> str | None: def format(self, error: "Violation") -> Optional[str]:
"""Do nothing.""" """Do nothing."""
def show_source(self, error: Violation) -> str | None: def show_source(self, error: "Violation") -> Optional[str]:
"""Do not print the source.""" """Do not print the source."""

View file

@ -1,2 +1 @@
"""Module containing the logic for the Flake8 entry-points.""" """Module containing the logic for the Flake8 entry-points."""
from __future__ import annotations

View file

@ -1,22 +1,30 @@
"""Module containing the application logic for Flake8.""" """Module containing the application logic for Flake8."""
from __future__ import annotations
import argparse import argparse
import json
import logging import logging
import sys
import time import time
from collections.abc import Sequence from typing import Dict
from typing import List
from typing import Optional
from typing import Set
from typing import Tuple
from typing import Type
from typing import TYPE_CHECKING
import flake8 import flake8
from flake8 import checker from flake8 import checker
from flake8 import defaults from flake8 import defaults
from flake8 import exceptions from flake8 import exceptions
from flake8 import style_guide from flake8 import style_guide
from flake8.formatting.base import BaseFormatter from flake8 import utils
from flake8.main import debug from flake8.main import options
from flake8.options.parse_args import parse_args from flake8.options import aggregator
from flake8.plugins import finder from flake8.options import config
from flake8.plugins import reporter from flake8.options import manager
from flake8.plugins import manager as plugin_manager
if TYPE_CHECKING:
from flake8.formatting.base import BaseFormatter
LOG = logging.getLogger(__name__) LOG = logging.getLogger(__name__)
@ -25,26 +33,56 @@ LOG = logging.getLogger(__name__)
class Application: class Application:
"""Abstract our application into a class.""" """Abstract our application into a class."""
def __init__(self) -> None: def __init__(self, program="flake8", version=flake8.__version__):
"""Initialize our application.""" """Initialize our application.
:param str program:
The name of the program/application that we're executing.
:param str version:
The version of the program/application we're executing.
"""
#: The timestamp when the Application instance was instantiated. #: The timestamp when the Application instance was instantiated.
self.start_time = time.time() self.start_time = time.time()
#: The timestamp when the Application finished reported errors. #: The timestamp when the Application finished reported errors.
self.end_time: float | None = None self.end_time: Optional[float] = None
#: The name of the program being run
self.program = program
#: The version of the program being run
self.version = version
#: The prelimary argument parser for handling options required for
#: obtaining and parsing the configuration file.
self.prelim_arg_parser = argparse.ArgumentParser(add_help=False)
options.register_preliminary_options(self.prelim_arg_parser)
#: The instance of :class:`flake8.options.manager.OptionManager` used
#: to parse and handle the options and arguments passed by the user
self.option_manager = manager.OptionManager(
prog="flake8",
version=flake8.__version__,
parents=[self.prelim_arg_parser],
)
options.register_default_options(self.option_manager)
self.plugins: finder.Plugins | None = None #: The instance of :class:`flake8.plugins.manager.Checkers`
self.check_plugins: Optional[plugin_manager.Checkers] = None
#: The instance of :class:`flake8.plugins.manager.ReportFormatters`
self.formatting_plugins: Optional[
plugin_manager.ReportFormatters
] = None
#: The user-selected formatter from :attr:`formatting_plugins` #: The user-selected formatter from :attr:`formatting_plugins`
self.formatter: BaseFormatter | None = None self.formatter: Optional[BaseFormatter] = None
#: The :class:`flake8.style_guide.StyleGuideManager` built from the #: The :class:`flake8.style_guide.StyleGuideManager` built from the
#: user's options #: user's options
self.guide: style_guide.StyleGuideManager | None = None self.guide: Optional[style_guide.StyleGuideManager] = None
#: The :class:`flake8.checker.Manager` that will handle running all of #: The :class:`flake8.checker.Manager` that will handle running all of
#: the checks selected by the user. #: the checks selected by the user.
self.file_checker_manager: checker.Manager | None = None self.file_checker_manager: Optional[checker.Manager] = None
#: The user-supplied options parsed into an instance of #: The user-supplied options parsed into an instance of
#: :class:`argparse.Namespace` #: :class:`argparse.Namespace`
self.options: argparse.Namespace | None = None self.options: Optional[argparse.Namespace] = None
#: The left over arguments that were not parsed by
#: :attr:`option_manager`
self.args: Optional[List[str]] = None
#: The number of errors, warnings, and other messages after running #: The number of errors, warnings, and other messages after running
#: flake8 and taking into account ignored errors and lines. #: flake8 and taking into account ignored errors and lines.
self.result_count = 0 self.result_count = 0
@ -55,50 +93,180 @@ class Application:
#: with a non-zero status code #: with a non-zero status code
self.catastrophic_failure = False self.catastrophic_failure = False
def exit_code(self) -> int: #: Whether the program is processing a diff or not
"""Return the program exit code.""" self.running_against_diff = False
if self.catastrophic_failure: #: The parsed diff information
return 1 self.parsed_diff: Dict[str, Set[int]] = {}
assert self.options is not None
if self.options.exit_zero:
return 0
else:
return int(self.result_count > 0)
def make_formatter(self) -> None: def parse_preliminary_options(
"""Initialize a formatter based on the parsed options.""" self, argv: List[str]
assert self.plugins is not None ) -> Tuple[argparse.Namespace, List[str]]:
"""Get preliminary options from the CLI, pre-plugin-loading.
We need to know the values of a few standard options so that we can
locate configuration files and configure logging.
Since plugins aren't loaded yet, there may be some as-yet-unknown
options; we ignore those for now, they'll be parsed later when we do
real option parsing.
:param list argv:
Command-line arguments passed in directly.
:returns:
Populated namespace and list of remaining argument strings.
:rtype:
(argparse.Namespace, list)
"""
args, rest = self.prelim_arg_parser.parse_known_args(argv)
# XXX (ericvw): Special case "forwarding" the output file option so
# that it can be reparsed again for the BaseFormatter.filename.
if args.output_file:
rest.extend(("--output-file", args.output_file))
return args, rest
def exit(self) -> None:
"""Handle finalization and exiting the program.
This should be the last thing called on the application instance. It
will check certain options and exit appropriately.
"""
assert self.options is not None assert self.options is not None
self.formatter = reporter.make(self.plugins.reporters, self.options) if self.options.count:
print(self.result_count)
if self.options.exit_zero:
raise SystemExit(self.catastrophic_failure)
else:
raise SystemExit(
(self.result_count > 0) or self.catastrophic_failure
)
def find_plugins(self, config_finder: config.ConfigFileFinder) -> None:
"""Find and load the plugins for this application.
Set the :attr:`check_plugins` and :attr:`formatting_plugins` attributes
based on the discovered plugins found.
:param config.ConfigFileFinder config_finder:
The finder for finding and reading configuration files.
"""
local_plugins = config.get_local_plugins(config_finder)
sys.path.extend(local_plugins.paths)
self.check_plugins = plugin_manager.Checkers(local_plugins.extension)
self.formatting_plugins = plugin_manager.ReportFormatters(
local_plugins.report
)
self.check_plugins.load_plugins()
self.formatting_plugins.load_plugins()
def register_plugin_options(self) -> None:
"""Register options provided by plugins to our option manager."""
assert self.check_plugins is not None
self.check_plugins.register_options(self.option_manager)
self.check_plugins.register_plugin_versions(self.option_manager)
assert self.formatting_plugins is not None
self.formatting_plugins.register_options(self.option_manager)
def parse_configuration_and_cli(
self,
config_finder: config.ConfigFileFinder,
argv: List[str],
) -> None:
"""Parse configuration files and the CLI options.
:param config.ConfigFileFinder config_finder:
The finder for finding and reading configuration files.
:param list argv:
Command-line arguments passed in directly.
"""
self.options, self.args = aggregator.aggregate_options(
self.option_manager,
config_finder,
argv,
)
self.running_against_diff = self.options.diff
if self.running_against_diff:
self.parsed_diff = utils.parse_unified_diff()
if not self.parsed_diff:
self.exit()
assert self.check_plugins is not None
self.check_plugins.provide_options(
self.option_manager, self.options, self.args
)
assert self.formatting_plugins is not None
self.formatting_plugins.provide_options(
self.option_manager, self.options, self.args
)
def formatter_for(self, formatter_plugin_name):
"""Retrieve the formatter class by plugin name."""
assert self.formatting_plugins is not None
default_formatter = self.formatting_plugins["default"]
formatter_plugin = self.formatting_plugins.get(formatter_plugin_name)
if formatter_plugin is None:
LOG.warning(
'"%s" is an unknown formatter. Falling back to default.',
formatter_plugin_name,
)
formatter_plugin = default_formatter
return formatter_plugin.execute
def make_formatter(
self, formatter_class: Optional[Type["BaseFormatter"]] = None
) -> None:
"""Initialize a formatter based on the parsed options."""
assert self.options is not None
format_plugin = self.options.format
if 1 <= self.options.quiet < 2:
format_plugin = "quiet-filename"
elif 2 <= self.options.quiet:
format_plugin = "quiet-nothing"
if formatter_class is None:
formatter_class = self.formatter_for(format_plugin)
self.formatter = formatter_class(self.options)
def make_guide(self) -> None: def make_guide(self) -> None:
"""Initialize our StyleGuide.""" """Initialize our StyleGuide."""
assert self.formatter is not None assert self.formatter is not None
assert self.options is not None assert self.options is not None
self.guide = style_guide.StyleGuideManager( self.guide = style_guide.StyleGuideManager(
self.options, self.formatter, self.options, self.formatter
) )
def make_file_checker_manager(self, argv: Sequence[str]) -> None: if self.running_against_diff:
self.guide.add_diff_ranges(self.parsed_diff)
def make_file_checker_manager(self) -> None:
"""Initialize our FileChecker Manager.""" """Initialize our FileChecker Manager."""
assert self.guide is not None
assert self.plugins is not None
self.file_checker_manager = checker.Manager( self.file_checker_manager = checker.Manager(
style_guide=self.guide, style_guide=self.guide,
plugins=self.plugins.checkers, arguments=self.args,
argv=argv, checker_plugins=self.check_plugins,
) )
def run_checks(self) -> None: def run_checks(self, files: Optional[List[str]] = None) -> None:
"""Run the actual checks with the FileChecker Manager. """Run the actual checks with the FileChecker Manager.
This method encapsulates the logic to make a This method encapsulates the logic to make a
:class:`~flake8.checker.Manger` instance run the checks it is :class:`~flake8.checker.Manger` instance run the checks it is
managing. managing.
:param list files:
List of filenames to process
""" """
assert self.file_checker_manager is not None assert self.file_checker_manager is not None
if self.running_against_diff:
self.file_checker_manager.start() files = sorted(self.parsed_diff)
self.file_checker_manager.start(files)
try: try:
self.file_checker_manager.run() self.file_checker_manager.run()
except exceptions.PluginExecutionFailed as plugin_failed: except exceptions.PluginExecutionFailed as plugin_failed:
@ -109,7 +277,7 @@ class Application:
self.file_checker_manager.stop() self.file_checker_manager.stop()
self.end_time = time.time() self.end_time = time.time()
def report_benchmarks(self) -> None: def report_benchmarks(self):
"""Aggregate, calculate, and report benchmarks for this run.""" """Aggregate, calculate, and report benchmarks for this run."""
assert self.options is not None assert self.options is not None
if not self.options.benchmark: if not self.options.benchmark:
@ -146,7 +314,7 @@ class Application:
self.result_count, self.result_count,
) )
def report_statistics(self) -> None: def report_statistics(self):
"""Aggregate and report statistics from this run.""" """Aggregate and report statistics from this run."""
assert self.options is not None assert self.options is not None
if not self.options.statistics: if not self.options.statistics:
@ -156,24 +324,33 @@ class Application:
assert self.guide is not None assert self.guide is not None
self.formatter.show_statistics(self.guide.stats) self.formatter.show_statistics(self.guide.stats)
def initialize(self, argv: Sequence[str]) -> None: def initialize(self, argv: List[str]) -> None:
"""Initialize the application to be run. """Initialize the application to be run.
This finds the plugins, registers their options, and parses the This finds the plugins, registers their options, and parses the
command-line arguments. command-line arguments.
""" """
self.plugins, self.options = parse_args(argv) # NOTE(sigmavirus24): When updating this, make sure you also update
# our legacy API calls to these same methods.
if self.options.bug_report: prelim_opts, remaining_args = self.parse_preliminary_options(argv)
info = debug.information(flake8.__version__, self.plugins) flake8.configure_logging(prelim_opts.verbose, prelim_opts.output_file)
print(json.dumps(info, indent=2, sort_keys=True)) config_finder = config.ConfigFileFinder(
raise SystemExit(0) self.program,
prelim_opts.append_config,
config_file=prelim_opts.config,
ignore_config_files=prelim_opts.isolated,
)
self.find_plugins(config_finder)
self.register_plugin_options()
self.parse_configuration_and_cli(
config_finder,
remaining_args,
)
self.make_formatter() self.make_formatter()
self.make_guide() self.make_guide()
self.make_file_checker_manager(argv) self.make_file_checker_manager()
def report(self) -> None: def report(self):
"""Report errors, statistics, and benchmarks.""" """Report errors, statistics, and benchmarks."""
assert self.formatter is not None assert self.formatter is not None
self.formatter.start() self.formatter.start()
@ -182,12 +359,12 @@ class Application:
self.report_benchmarks() self.report_benchmarks()
self.formatter.stop() self.formatter.stop()
def _run(self, argv: Sequence[str]) -> None: def _run(self, argv: List[str]) -> None:
self.initialize(argv) self.initialize(argv)
self.run_checks() self.run_checks()
self.report() self.report()
def run(self, argv: Sequence[str]) -> None: def run(self, argv: List[str]) -> None:
"""Run our application. """Run our application.
This method will also handle KeyboardInterrupt exceptions for the This method will also handle KeyboardInterrupt exceptions for the
@ -209,7 +386,3 @@ class Application:
except exceptions.EarlyQuit: except exceptions.EarlyQuit:
self.catastrophic_failure = True self.catastrophic_failure = True
print("... stopped while processing files") print("... stopped while processing files")
else:
assert self.options is not None
if self.options.count:
print(self.result_count)

View file

@ -1,19 +1,18 @@
"""Command-line implementation of flake8.""" """Command-line implementation of flake8."""
from __future__ import annotations
import sys import sys
from collections.abc import Sequence from typing import List
from typing import Optional
from flake8.main import application from flake8.main import application
def main(argv: Sequence[str] | None = None) -> int: def main(argv: Optional[List[str]] = None) -> None:
"""Execute the main bit of the application. """Execute the main bit of the application.
This handles the creation of an instance of :class:`Application`, runs it, This handles the creation of an instance of :class:`Application`, runs it,
and then exits the application. and then exits the application.
:param argv: :param list argv:
The arguments to be passed to the application for parsing. The arguments to be passed to the application for parsing.
""" """
if argv is None: if argv is None:
@ -21,4 +20,4 @@ def main(argv: Sequence[str] | None = None) -> int:
app = application.Application() app = application.Application()
app.run(argv) app.run(argv)
return app.exit_code() app.exit()

View file

@ -1,30 +1,64 @@
"""Module containing the logic for our debugging logic.""" """Module containing the logic for our debugging logic."""
from __future__ import annotations import argparse
import json
import platform import platform
from typing import Any from typing import Dict
from typing import List
from flake8.plugins.finder import Plugins
def information(version: str, plugins: Plugins) -> dict[str, Any]: class DebugAction(argparse.Action):
"""argparse action to print debug information."""
def __init__(self, *args, **kwargs):
"""Initialize the action.
This takes an extra `option_manager` keyword argument which will be
used to delay response.
"""
self._option_manager = kwargs.pop("option_manager")
super().__init__(*args, **kwargs)
def __call__(self, parser, namespace, values, option_string=None):
"""Perform the argparse action for printing debug information."""
# NOTE(sigmavirus24): Flake8 parses options twice. The first time, we
# will not have any registered plugins. We can skip this one and only
# take action on the second time we're called.
if not self._option_manager.registered_plugins:
return
print(
json.dumps(
information(self._option_manager), indent=2, sort_keys=True
)
)
raise SystemExit(0)
def information(option_manager):
"""Generate the information to be printed for the bug report.""" """Generate the information to be printed for the bug report."""
versions = sorted(
{
(loaded.plugin.package, loaded.plugin.version)
for loaded in plugins.all_plugins()
if loaded.plugin.package not in {"flake8", "local"}
},
)
return { return {
"version": version, "version": option_manager.version,
"plugins": [ "plugins": plugins_from(option_manager),
{"plugin": plugin, "version": version} "dependencies": dependencies(),
for plugin, version in versions
],
"platform": { "platform": {
"python_implementation": platform.python_implementation(), "python_implementation": platform.python_implementation(),
"python_version": platform.python_version(), "python_version": platform.python_version(),
"system": platform.system(), "system": platform.system(),
}, },
} }
def plugins_from(option_manager):
"""Generate the list of plugins installed."""
return [
{
"plugin": plugin.name,
"version": plugin.version,
"is_local": plugin.local,
}
for plugin in sorted(option_manager.registered_plugins)
]
def dependencies() -> List[Dict[str, str]]:
"""Generate the list of dependencies we care about."""
return []

View file

@ -1,13 +1,12 @@
"""Contains the logic for all of the default options for Flake8.""" """Contains the logic for all of the default options for Flake8."""
from __future__ import annotations
import argparse import argparse
import functools
from flake8 import defaults from flake8 import defaults
from flake8.options.manager import OptionManager from flake8.main import debug
def stage1_arg_parser() -> argparse.ArgumentParser: def register_preliminary_options(parser: argparse.ArgumentParser) -> None:
"""Register the preliminary options on our OptionManager. """Register the preliminary options on our OptionManager.
The preliminary options include: The preliminary options include:
@ -17,37 +16,35 @@ def stage1_arg_parser() -> argparse.ArgumentParser:
- ``--append-config`` - ``--append-config``
- ``--config`` - ``--config``
- ``--isolated`` - ``--isolated``
- ``--enable-extensions``
""" """
parser = argparse.ArgumentParser(add_help=False) add_argument = parser.add_argument
parser.add_argument( add_argument(
"-v", "-v",
"--verbose", "--verbose",
default=0, default=0,
action="count", action="count",
help="Print more information about what is happening in flake8. " help="Print more information about what is happening in flake8."
"This option is repeatable and will increase verbosity each " " This option is repeatable and will increase verbosity each "
"time it is repeated.", "time it is repeated.",
) )
parser.add_argument( add_argument(
"--output-file", default=None, help="Redirect report to a file.", "--output-file", default=None, help="Redirect report to a file."
) )
# Config file options # Config file options
parser.add_argument( add_argument(
"--append-config", "--append-config",
action="append", action="append",
default=[],
help="Provide extra config files to parse in addition to the files " help="Provide extra config files to parse in addition to the files "
"found by Flake8 by default. These files are the last ones read " "found by Flake8 by default. These files are the last ones read "
"and so they take the highest precedence when multiple files " "and so they take the highest precedence when multiple files "
"provide the same option.", "provide the same option.",
) )
parser.add_argument( add_argument(
"--config", "--config",
default=None, default=None,
help="Path to the config file that will be the authoritative config " help="Path to the config file that will be the authoritative config "
@ -55,28 +52,13 @@ def stage1_arg_parser() -> argparse.ArgumentParser:
"configuration files.", "configuration files.",
) )
parser.add_argument( add_argument(
"--isolated", "--isolated",
default=False, default=False,
action="store_true", action="store_true",
help="Ignore all configuration files.", help="Ignore all configuration files.",
) )
# Plugin enablement options
parser.add_argument(
"--enable-extensions",
help="Enable plugins and extensions that are otherwise disabled "
"by default",
)
parser.add_argument(
"--require-plugins",
help="Require specific plugins to be installed before running",
)
return parser
class JobsArgument: class JobsArgument:
"""Type callback for the --jobs argument.""" """Type callback for the --jobs argument."""
@ -84,7 +66,8 @@ class JobsArgument:
def __init__(self, arg: str) -> None: def __init__(self, arg: str) -> None:
"""Parse and validate the --jobs argument. """Parse and validate the --jobs argument.
:param arg: The argument passed by argparse for validation :param str arg:
The argument passed by argparse for validation
""" """
self.is_auto = False self.is_auto = False
self.n_jobs = -1 self.n_jobs = -1
@ -97,23 +80,19 @@ class JobsArgument:
f"{arg!r} must be 'auto' or an integer.", f"{arg!r} must be 'auto' or an integer.",
) )
def __repr__(self) -> str: def __str__(self):
"""Representation for debugging."""
return f"{type(self).__name__}({str(self)!r})"
def __str__(self) -> str:
"""Format our JobsArgument class.""" """Format our JobsArgument class."""
return "auto" if self.is_auto else str(self.n_jobs) return "auto" if self.is_auto else str(self.n_jobs)
def register_default_options(option_manager: OptionManager) -> None: def register_default_options(option_manager):
"""Register the default options on our OptionManager. """Register the default options on our OptionManager.
The default options include: The default options include:
- ``-q``/``--quiet`` - ``-q``/``--quiet``
- ``--color``
- ``--count`` - ``--count``
- ``--diff``
- ``--exclude`` - ``--exclude``
- ``--extend-exclude`` - ``--extend-exclude``
- ``--filename`` - ``--filename``
@ -130,6 +109,7 @@ def register_default_options(option_manager: OptionManager) -> None:
- ``--disable-noqa`` - ``--disable-noqa``
- ``--show-source`` - ``--show-source``
- ``--statistics`` - ``--statistics``
- ``--enable-extensions``
- ``--exit-zero`` - ``--exit-zero``
- ``-j``/``--jobs`` - ``-j``/``--jobs``
- ``--tee`` - ``--tee``
@ -138,6 +118,7 @@ def register_default_options(option_manager: OptionManager) -> None:
""" """
add_option = option_manager.add_option add_option = option_manager.add_option
# pep8 options
add_option( add_option(
"-q", "-q",
"--quiet", "--quiet",
@ -147,19 +128,19 @@ def register_default_options(option_manager: OptionManager) -> None:
help="Report only file names, or nothing. This option is repeatable.", help="Report only file names, or nothing. This option is repeatable.",
) )
add_option(
"--color",
choices=("auto", "always", "never"),
default="auto",
help="Whether to use color in output. Defaults to `%(default)s`.",
)
add_option( add_option(
"--count", "--count",
action="store_true", action="store_true",
parse_from_config=True, parse_from_config=True,
help="Print total number of errors to standard output after " help="Print total number of errors and warnings to standard error and"
"all other output.", " set the exit code to 1 if total is not empty.",
)
add_option(
"--diff",
action="store_true",
help="Report changes only within line number ranges in the unified "
"diff provided on standard in by the user.",
) )
add_option( add_option(
@ -169,8 +150,8 @@ def register_default_options(option_manager: OptionManager) -> None:
comma_separated_list=True, comma_separated_list=True,
parse_from_config=True, parse_from_config=True,
normalize_paths=True, normalize_paths=True,
help="Comma-separated list of files or directories to exclude. " help="Comma-separated list of files or directories to exclude."
"(Default: %(default)s)", " (Default: %(default)s)",
) )
add_option( add_option(
@ -180,8 +161,8 @@ def register_default_options(option_manager: OptionManager) -> None:
parse_from_config=True, parse_from_config=True,
comma_separated_list=True, comma_separated_list=True,
normalize_paths=True, normalize_paths=True,
help="Comma-separated list of files or directories to add to the list " help="Comma-separated list of files or directories to add to the list"
"of excluded ones.", " of excluded ones.",
) )
add_option( add_option(
@ -197,9 +178,9 @@ def register_default_options(option_manager: OptionManager) -> None:
add_option( add_option(
"--stdin-display-name", "--stdin-display-name",
default="stdin", default="stdin",
help="The name used when reporting errors from code passed via stdin. " help="The name used when reporting errors from code passed via stdin."
"This is useful for editors piping the file contents to flake8. " " This is useful for editors piping the file contents to flake8."
"(Default: %(default)s)", " (Default: %(default)s)",
) )
# TODO(sigmavirus24): Figure out --first/--repeat # TODO(sigmavirus24): Figure out --first/--repeat
@ -212,44 +193,35 @@ def register_default_options(option_manager: OptionManager) -> None:
metavar="format", metavar="format",
default="default", default="default",
parse_from_config=True, parse_from_config=True,
help=( help="Format errors according to the chosen formatter.",
f"Format errors according to the chosen formatter "
f"({', '.join(sorted(option_manager.formatter_names))}) "
f"or a format string containing %%-style "
f"mapping keys (code, col, path, row, text). "
f"For example, "
f"``--format=pylint`` or ``--format='%%(path)s %%(code)s'``. "
f"(Default: %(default)s)"
),
) )
add_option( add_option(
"--hang-closing", "--hang-closing",
action="store_true", action="store_true",
parse_from_config=True, parse_from_config=True,
help="Hang closing bracket instead of matching indentation of opening " help="Hang closing bracket instead of matching indentation of opening"
"bracket's line.", " bracket's line.",
) )
add_option( add_option(
"--ignore", "--ignore",
metavar="errors", metavar="errors",
default=",".join(defaults.IGNORE),
parse_from_config=True, parse_from_config=True,
comma_separated_list=True, comma_separated_list=True,
help=( help="Comma-separated list of errors and warnings to ignore (or skip)."
f"Comma-separated list of error codes to ignore (or skip). " " For example, ``--ignore=E4,E51,W234``. (Default: %(default)s)",
f"For example, ``--ignore=E4,E51,W234``. "
f"(Default: {','.join(defaults.IGNORE)})"
),
) )
add_option( add_option(
"--extend-ignore", "--extend-ignore",
metavar="errors", metavar="errors",
default="",
parse_from_config=True, parse_from_config=True,
comma_separated_list=True, comma_separated_list=True,
help="Comma-separated list of error codes to add to the list of " help="Comma-separated list of errors and warnings to add to the list"
"ignored ones. For example, ``--extend-ignore=E4,E51,W234``.", " of ignored ones. For example, ``--extend-ignore=E4,E51,W234``.",
) )
add_option( add_option(
@ -294,27 +266,22 @@ def register_default_options(option_manager: OptionManager) -> None:
add_option( add_option(
"--select", "--select",
metavar="errors", metavar="errors",
default=",".join(defaults.SELECT),
parse_from_config=True, parse_from_config=True,
comma_separated_list=True, comma_separated_list=True,
help=( help="Comma-separated list of errors and warnings to enable."
"Limit the reported error codes to codes prefix-matched by this " " For example, ``--select=E4,E51,W234``. (Default: %(default)s)",
"list. "
"You usually do not need to specify this option as the default "
"includes all installed plugin codes. "
"For example, ``--select=E4,E51,W234``."
),
) )
add_option( add_option(
"--extend-select", "--extend-select",
metavar="errors", metavar="errors",
default="",
parse_from_config=True, parse_from_config=True,
comma_separated_list=True, comma_separated_list=True,
help=( help=(
"Add additional error codes to the default ``--select``. " "Comma-separated list of errors and warnings to add to the list "
"You usually do not need to specify this option as the default " "of selected ones. For example, ``--extend-select=E4,E51,W234``."
"includes all installed plugin codes. "
"For example, ``--extend-select=E4,E51,W234``."
), ),
) )
@ -347,10 +314,18 @@ def register_default_options(option_manager: OptionManager) -> None:
"--statistics", "--statistics",
action="store_true", action="store_true",
parse_from_config=True, parse_from_config=True,
help="Count errors.", help="Count errors and warnings.",
) )
# Flake8 options # Flake8 options
add_option(
"--enable-extensions",
default="",
parse_from_config=True,
comma_separated_list=True,
help="Enable plugins and extensions that are otherwise disabled "
"by default",
)
add_option( add_option(
"--exit-zero", "--exit-zero",
@ -366,8 +341,8 @@ def register_default_options(option_manager: OptionManager) -> None:
type=JobsArgument, type=JobsArgument,
help="Number of subprocesses to use to run checks in parallel. " help="Number of subprocesses to use to run checks in parallel. "
'This is ignored on Windows. The default, "auto", will ' 'This is ignored on Windows. The default, "auto", will '
"auto-detect the number of processors available to use. " "auto-detect the number of processors available to use."
"(Default: %(default)s)", " (Default: %(default)s)",
) )
add_option( add_option(
@ -391,6 +366,9 @@ def register_default_options(option_manager: OptionManager) -> None:
add_option( add_option(
"--bug-report", "--bug-report",
action="store_true", action=functools.partial(
debug.DebugAction, option_manager=option_manager
),
nargs=0,
help="Print information necessary when preparing a bug report", help="Print information necessary when preparing a bug report",
) )

View file

@ -10,4 +10,3 @@
to aggregate configuration into one object used by plugins and Flake8. to aggregate configuration into one object used by plugins and Flake8.
""" """
from __future__ import annotations

View file

@ -3,12 +3,10 @@
This holds the logic that uses the collected and merged config files and This holds the logic that uses the collected and merged config files and
applies the user-specified command-line configuration on top of it. applies the user-specified command-line configuration on top of it.
""" """
from __future__ import annotations
import argparse import argparse
import configparser
import logging import logging
from collections.abc import Sequence from typing import List
from typing import Tuple
from flake8.options import config from flake8.options import config
from flake8.options.manager import OptionManager from flake8.options.manager import OptionManager
@ -18,20 +16,54 @@ LOG = logging.getLogger(__name__)
def aggregate_options( def aggregate_options(
manager: OptionManager, manager: OptionManager,
cfg: configparser.RawConfigParser, config_finder: config.ConfigFileFinder,
cfg_dir: str, argv: List[str],
argv: Sequence[str] | None, ) -> Tuple[argparse.Namespace, List[str]]:
) -> argparse.Namespace: """Aggregate and merge CLI and config file options.
"""Aggregate and merge CLI and config file options."""
:param flake8.options.manager.OptionManager manager:
The instance of the OptionManager that we're presently using.
:param flake8.options.config.ConfigFileFinder config_finder:
The config file finder to use.
:param list argv:
The list of remaining command-line arguments that were unknown during
preliminary option parsing to pass to ``manager.parse_args``.
:returns:
Tuple of the parsed options and extra arguments returned by
``manager.parse_args``.
:rtype:
tuple(argparse.Namespace, list)
"""
# Get defaults from the option parser # Get defaults from the option parser
default_values = manager.parse_args([]) default_values, _ = manager.parse_args([])
# Make our new configuration file mergerator
config_parser = config.ConfigParser(
option_manager=manager, config_finder=config_finder
)
# Get the parsed config # Get the parsed config
parsed_config = config.parse_config(manager, cfg, cfg_dir) parsed_config = config_parser.parse()
# store the plugin-set extended default ignore / select # Extend the default ignore value with the extended default ignore list,
default_values.extended_default_ignore = manager.extended_default_ignore # registered by plugins.
default_values.extended_default_select = manager.extended_default_select extended_default_ignore = manager.extended_default_ignore.copy()
# Let's store our extended default ignore for use by the decision engine
default_values.extended_default_ignore = (
manager.extended_default_ignore.copy()
)
LOG.debug(
"Extended default ignore list: %s", list(extended_default_ignore)
)
extended_default_ignore.update(default_values.ignore)
default_values.ignore = list(extended_default_ignore)
LOG.debug("Merged default ignore list: %s", default_values.ignore)
extended_default_select = manager.extended_default_select.copy()
LOG.debug(
"Extended default select list: %s", list(extended_default_select)
)
default_values.extended_default_select = extended_default_select
# Merge values parsed from config onto the default values returned # Merge values parsed from config onto the default values returned
for config_name, value in parsed_config.items(): for config_name, value in parsed_config.items():
@ -39,9 +71,7 @@ def aggregate_options(
# If the config name is somehow different from the destination name, # If the config name is somehow different from the destination name,
# fetch the destination name from our Option # fetch the destination name from our Option
if not hasattr(default_values, config_name): if not hasattr(default_values, config_name):
dest_val = manager.config_options_dict[config_name].dest dest_name = config_parser.config_options[config_name].dest
assert isinstance(dest_val, str)
dest_name = dest_val
LOG.debug( LOG.debug(
'Overriding default value of (%s) for "%s" with (%s)', 'Overriding default value of (%s) for "%s" with (%s)',

View file

@ -1,140 +1,318 @@
"""Config handling logic for Flake8.""" """Config handling logic for Flake8."""
from __future__ import annotations import collections
import configparser import configparser
import logging import logging
import os.path import os.path
from typing import Any from typing import List
from typing import Optional
from typing import Tuple
from flake8 import exceptions from flake8 import utils
from flake8.defaults import VALID_CODE_PREFIX
from flake8.options.manager import OptionManager
LOG = logging.getLogger(__name__) LOG = logging.getLogger(__name__)
__all__ = ("ConfigFileFinder", "ConfigParser")
def _stat_key(s: str) -> tuple[int, int]:
# same as what's used by samefile / samestat
st = os.stat(s)
return st.st_ino, st.st_dev
def _find_config_file(path: str) -> str | None: class ConfigFileFinder:
# on windows if the homedir isn't detected this returns back `~` """Encapsulate the logic for finding and reading config files."""
home = os.path.expanduser("~")
try:
home_stat = _stat_key(home) if home != "~" else None
except OSError: # FileNotFoundError / PermissionError / etc.
home_stat = None
dir_stat = _stat_key(path) def __init__(
while True: self,
for candidate in ("setup.cfg", "tox.ini", ".flake8"): program_name: str,
cfg = configparser.RawConfigParser() extra_config_files: Optional[List[str]] = None,
cfg_path = os.path.join(path, candidate) config_file: Optional[str] = None,
ignore_config_files: bool = False,
) -> None:
"""Initialize object to find config files.
:param str program_name:
Name of the current program (e.g., flake8).
:param list extra_config_files:
Extra configuration files specified by the user to read.
:param str config_file:
Configuration file override to only read configuration from.
:param bool ignore_config_files:
Determine whether to ignore configuration files or not.
"""
# The values of --append-config from the CLI
if extra_config_files is None:
extra_config_files = []
self.extra_config_files = utils.normalize_paths(extra_config_files)
# The value of --config from the CLI.
self.config_file = config_file
# The value of --isolated from the CLI.
self.ignore_config_files = ignore_config_files
# User configuration file.
self.program_name = program_name
# List of filenames to find in the local/project directory
self.project_filenames = ("setup.cfg", "tox.ini", f".{program_name}")
self.local_directory = os.path.abspath(os.curdir)
@staticmethod
def _read_config(
*files: str,
) -> Tuple[configparser.RawConfigParser, List[str]]:
config = configparser.RawConfigParser()
found_files = []
for filename in files:
try: try:
cfg.read(cfg_path, encoding="UTF-8") found_files.extend(config.read(filename))
except (UnicodeDecodeError, configparser.ParsingError) as e: except UnicodeDecodeError:
LOG.warning("ignoring unparseable config %s: %s", cfg_path, e) LOG.exception(
else: "There was an error decoding a config file."
# only consider it a config if it contains flake8 sections "The file with a problem was %s.",
if "flake8" in cfg or "flake8:local-plugins" in cfg: filename,
return cfg_path )
except configparser.ParsingError:
LOG.exception(
"There was an error trying to parse a config "
"file. The file with a problem was %s.",
filename,
)
return (config, found_files)
new_path = os.path.dirname(path) def cli_config(self, files: str) -> configparser.RawConfigParser:
new_dir_stat = _stat_key(new_path) """Read and parse the config file specified on the command-line."""
if new_dir_stat == dir_stat or new_dir_stat == home_stat: config, found_files = self._read_config(files)
break if found_files:
else: LOG.debug("Found cli configuration files: %s", found_files)
path = new_path return config
dir_stat = new_dir_stat
# did not find any configuration file def generate_possible_local_files(self):
return None """Find and generate all local config files."""
parent = tail = os.getcwd()
found_config_files = False
while tail and not found_config_files:
for project_filename in self.project_filenames:
filename = os.path.abspath(
os.path.join(parent, project_filename)
)
if os.path.exists(filename):
yield filename
found_config_files = True
self.local_directory = parent
(parent, tail) = os.path.split(parent)
def local_config_files(self):
"""Find all local config files which actually exist.
Filter results from
:meth:`~ConfigFileFinder.generate_possible_local_files` based
on whether the filename exists or not.
:returns:
List of files that exist that are local project config files with
extra config files appended to that list (which also exist).
:rtype:
[str]
"""
exists = os.path.exists
return [
filename for filename in self.generate_possible_local_files()
] + [f for f in self.extra_config_files if exists(f)]
def local_configs_with_files(self):
"""Parse all local config files into one config object.
Return (config, found_config_files) tuple.
"""
config, found_files = self._read_config(*self.local_config_files())
if found_files:
LOG.debug("Found local configuration files: %s", found_files)
return (config, found_files)
def local_configs(self):
"""Parse all local config files into one config object."""
return self.local_configs_with_files()[0]
def load_config( class ConfigParser:
config: str | None, """Encapsulate merging different types of configuration files.
extra: list[str],
*,
isolated: bool = False,
) -> tuple[configparser.RawConfigParser, str]:
"""Load the configuration given the user options.
- in ``isolated`` mode, return an empty configuration This parses out the options registered that were specified in the
- if a config file is given in ``config`` use that, otherwise attempt to configuration files, handles extra configuration files, and returns
discover a configuration using ``tox.ini`` / ``setup.cfg`` / ``.flake8`` dictionaries with the parsed values.
- finally, load any ``extra`` configuration files
""" """
pwd = os.path.abspath(".")
if isolated: #: Set of actions that should use the
return configparser.RawConfigParser(), pwd #: :meth:`~configparser.RawConfigParser.getbool` method.
GETBOOL_ACTIONS = {"store_true", "store_false"}
if config is None: def __init__(self, option_manager, config_finder):
config = _find_config_file(pwd) """Initialize the ConfigParser instance.
cfg = configparser.RawConfigParser() :param flake8.options.manager.OptionManager option_manager:
if config is not None: Initialized OptionManager.
if not cfg.read(config, encoding="UTF-8"): :param flake8.options.config.ConfigFileFinder config_finder:
raise exceptions.ExecutionError( Initialized ConfigFileFinder.
f"The specified config file does not exist: {config}", """
#: Our instance of flake8.options.manager.OptionManager
self.option_manager = option_manager
#: The prog value for the cli parser
self.program_name = option_manager.program_name
#: Mapping of configuration option names to
#: :class:`~flake8.options.manager.Option` instances
self.config_options = option_manager.config_options_dict
#: Our instance of our :class:`~ConfigFileFinder`
self.config_finder = config_finder
def _normalize_value(self, option, value, parent=None):
if parent is None:
parent = self.config_finder.local_directory
final_value = option.normalize(value, parent)
LOG.debug(
'%r has been normalized to %r for option "%s"',
value,
final_value,
option.config_name,
)
return final_value
def _parse_config(self, config_parser, parent=None):
config_dict = {}
for option_name in config_parser.options(self.program_name):
if option_name not in self.config_options:
LOG.debug(
'Option "%s" is not registered. Ignoring.', option_name
)
continue
option = self.config_options[option_name]
# Use the appropriate method to parse the config value
method = config_parser.get
if option.type is int or option.action == "count":
method = config_parser.getint
elif option.action in self.GETBOOL_ACTIONS:
method = config_parser.getboolean
value = method(self.program_name, option_name)
LOG.debug('Option "%s" returned value: %r', option_name, value)
final_value = self._normalize_value(option, value, parent)
config_dict[option.config_name] = final_value
return config_dict
def is_configured_by(self, config):
"""Check if the specified config parser has an appropriate section."""
return config.has_section(self.program_name)
def parse_local_config(self):
"""Parse and return the local configuration files."""
config = self.config_finder.local_configs()
if not self.is_configured_by(config):
LOG.debug(
"Local configuration files have no %s section",
self.program_name,
) )
cfg_dir = os.path.dirname(config) return {}
LOG.debug("Parsing local configuration files.")
return self._parse_config(config)
def parse_cli_config(self, config_path):
"""Parse and return the file specified by --config."""
config = self.config_finder.cli_config(config_path)
if not self.is_configured_by(config):
LOG.debug(
"CLI configuration files have no %s section",
self.program_name,
)
return {}
LOG.debug("Parsing CLI configuration files.")
return self._parse_config(config, os.path.dirname(config_path))
def parse(self):
"""Parse and return the local config files.
:returns:
Dictionary of parsed configuration options
:rtype:
dict
"""
if self.config_finder.ignore_config_files:
LOG.debug(
"Refusing to parse configuration files due to user-"
"requested isolation"
)
return {}
if self.config_finder.config_file:
LOG.debug(
"Ignoring user and locally found configuration files. "
'Reading only configuration from "%s" specified via '
"--config by the user",
self.config_finder.config_file,
)
return self.parse_cli_config(self.config_finder.config_file)
return self.parse_local_config()
def get_local_plugins(config_finder):
"""Get local plugins lists from config files.
:param flake8.options.config.ConfigFileFinder config_finder:
The config file finder to use.
:returns:
LocalPlugins namedtuple containing two lists of plugin strings,
one for extension (checker) plugins and one for report plugins.
:rtype:
flake8.options.config.LocalPlugins
"""
local_plugins = LocalPlugins(extension=[], report=[], paths=[])
if config_finder.ignore_config_files:
LOG.debug(
"Refusing to look for local plugins in configuration"
"files due to user-requested isolation"
)
return local_plugins
if config_finder.config_file:
LOG.debug(
'Reading local plugins only from "%s" specified via '
"--config by the user",
config_finder.config_file,
)
config = config_finder.cli_config(config_finder.config_file)
config_files = [config_finder.config_file]
else: else:
cfg_dir = pwd config, config_files = config_finder.local_configs_with_files()
# TODO: remove this and replace it with configuration modifying plugins base_dirs = {os.path.dirname(cf) for cf in config_files}
# read the additional configs afterwards
for filename in extra: section = f"{config_finder.program_name}:local-plugins"
if not cfg.read(filename, encoding="UTF-8"): for plugin_type in ["extension", "report"]:
raise exceptions.ExecutionError( if config.has_option(section, plugin_type):
f"The specified config file does not exist: {filename}", local_plugins_string = config.get(section, plugin_type).strip()
plugin_type_list = getattr(local_plugins, plugin_type)
plugin_type_list.extend(
utils.parse_comma_separated_list(
local_plugins_string, regexp=utils.LOCAL_PLUGIN_LIST_RE
)
) )
if config.has_option(section, "paths"):
return cfg, cfg_dir raw_paths = utils.parse_comma_separated_list(
config.get(section, "paths").strip()
)
norm_paths: List[str] = []
for base_dir in base_dirs:
norm_paths.extend(
path
for path in utils.normalize_paths(raw_paths, parent=base_dir)
if os.path.exists(path)
)
local_plugins.paths.extend(norm_paths)
return local_plugins
def parse_config( LocalPlugins = collections.namedtuple("LocalPlugins", "extension report paths")
option_manager: OptionManager,
cfg: configparser.RawConfigParser,
cfg_dir: str,
) -> dict[str, Any]:
"""Parse and normalize the typed configuration options."""
if "flake8" not in cfg:
return {}
config_dict = {}
for option_name in cfg["flake8"]:
option = option_manager.config_options_dict.get(option_name)
if option is None:
LOG.debug('Option "%s" is not registered. Ignoring.', option_name)
continue
# Use the appropriate method to parse the config value
value: Any
if option.type is int or option.action == "count":
value = cfg.getint("flake8", option_name)
elif option.action in {"store_true", "store_false"}:
value = cfg.getboolean("flake8", option_name)
else:
value = cfg.get("flake8", option_name)
LOG.debug('Option "%s" returned value: %r', option_name, value)
final_value = option.normalize(value, cfg_dir)
if option_name in {"ignore", "extend-ignore"}:
for error_code in final_value:
if not VALID_CODE_PREFIX.match(error_code):
raise ValueError(
f"Error code {error_code!r} "
f"supplied to {option_name!r} option "
f"does not match {VALID_CODE_PREFIX.pattern!r}",
)
assert option.config_name is not None
config_dict[option.config_name] = final_value
return config_dict

View file

@ -1,16 +1,29 @@
"""Option handling and Option management logic.""" """Option handling and Option management logic."""
from __future__ import annotations
import argparse import argparse
import collections
import contextlib
import enum import enum
import functools import functools
import logging import logging
from collections.abc import Callable
from collections.abc import Sequence
from typing import Any from typing import Any
from typing import Callable
from typing import cast
from typing import Dict
from typing import Generator
from typing import List
from typing import Mapping
from typing import Optional
from typing import Sequence
from typing import Set
from typing import Tuple
from typing import Type
from typing import TYPE_CHECKING
from typing import Union
from flake8 import utils from flake8 import utils
from flake8.plugins.finder import Plugins
if TYPE_CHECKING:
from typing import NoReturn
LOG = logging.getLogger(__name__) LOG = logging.getLogger(__name__)
@ -19,13 +32,57 @@ LOG = logging.getLogger(__name__)
_ARG = enum.Enum("_ARG", "NO") _ARG = enum.Enum("_ARG", "NO")
_optparse_callable_map: Dict[str, Union[Type[Any], _ARG]] = {
"int": int,
"long": int,
"string": str,
"float": float,
"complex": complex,
"choice": _ARG.NO,
# optparse allows this but does not document it
"str": str,
}
class _CallbackAction(argparse.Action):
"""Shim for optparse-style callback actions."""
def __init__(self, *args: Any, **kwargs: Any) -> None:
self._callback = kwargs.pop("callback")
self._callback_args = kwargs.pop("callback_args", ())
self._callback_kwargs = kwargs.pop("callback_kwargs", {})
super().__init__(*args, **kwargs)
def __call__(
self,
parser: argparse.ArgumentParser,
namespace: argparse.Namespace,
values: Optional[Union[Sequence[str], str]],
option_string: Optional[str] = None,
) -> None:
if not values:
values = None
elif isinstance(values, list) and len(values) > 1:
values = tuple(values)
self._callback(
self,
option_string,
values,
parser,
*self._callback_args,
**self._callback_kwargs,
)
def _flake8_normalize( def _flake8_normalize(
value: str, value: str, *args: str, **kwargs: bool
*args: str, ) -> Union[str, List[str]]:
comma_separated_list: bool = False, comma_separated_list = kwargs.pop("comma_separated_list", False)
normalize_paths: bool = False, normalize_paths = kwargs.pop("normalize_paths", False)
) -> str | list[str]: if kwargs:
ret: str | list[str] = value raise TypeError(f"Unexpected keyword args: {kwargs}")
ret: Union[str, List[str]] = value
if comma_separated_list and isinstance(ret, str): if comma_separated_list and isinstance(ret, str):
ret = utils.parse_comma_separated_list(value) ret = utils.parse_comma_separated_list(value)
@ -43,19 +100,24 @@ class Option:
def __init__( def __init__(
self, self,
short_option_name: str | _ARG = _ARG.NO, short_option_name: Union[str, _ARG] = _ARG.NO,
long_option_name: str | _ARG = _ARG.NO, long_option_name: Union[str, _ARG] = _ARG.NO,
# Options below here are taken from the optparse.Option class
action: Union[str, Type[argparse.Action], _ARG] = _ARG.NO,
default: Union[Any, _ARG] = _ARG.NO,
type: Union[str, Callable[..., Any], _ARG] = _ARG.NO,
dest: Union[str, _ARG] = _ARG.NO,
nargs: Union[int, str, _ARG] = _ARG.NO,
const: Union[Any, _ARG] = _ARG.NO,
choices: Union[Sequence[Any], _ARG] = _ARG.NO,
help: Union[str, _ARG] = _ARG.NO,
metavar: Union[str, _ARG] = _ARG.NO,
# deprecated optparse-only options
callback: Union[Callable[..., Any], _ARG] = _ARG.NO,
callback_args: Union[Sequence[Any], _ARG] = _ARG.NO,
callback_kwargs: Union[Mapping[str, Any], _ARG] = _ARG.NO,
# Options below are taken from argparse.ArgumentParser.add_argument # Options below are taken from argparse.ArgumentParser.add_argument
action: str | type[argparse.Action] | _ARG = _ARG.NO, required: Union[bool, _ARG] = _ARG.NO,
default: Any | _ARG = _ARG.NO,
type: Callable[..., Any] | _ARG = _ARG.NO,
dest: str | _ARG = _ARG.NO,
nargs: int | str | _ARG = _ARG.NO,
const: Any | _ARG = _ARG.NO,
choices: Sequence[Any] | _ARG = _ARG.NO,
help: str | _ARG = _ARG.NO,
metavar: str | _ARG = _ARG.NO,
required: bool | _ARG = _ARG.NO,
# Options below here are specific to Flake8 # Options below here are specific to Flake8
parse_from_config: bool = False, parse_from_config: bool = False,
comma_separated_list: bool = False, comma_separated_list: bool = False,
@ -65,10 +127,10 @@ class Option:
The following are all passed directly through to argparse. The following are all passed directly through to argparse.
:param short_option_name: :param str short_option_name:
The short name of the option (e.g., ``-x``). This will be the The short name of the option (e.g., ``-x``). This will be the
first argument passed to ``ArgumentParser.add_argument`` first argument passed to ``ArgumentParser.add_argument``
:param long_option_name: :param str long_option_name:
The long name of the option (e.g., ``--xtra-long-option``). This The long name of the option (e.g., ``--xtra-long-option``). This
will be the second argument passed to will be the second argument passed to
``ArgumentParser.add_argument`` ``ArgumentParser.add_argument``
@ -81,13 +143,13 @@ class Option:
:param const: :param const:
Constant value to store on a common destination. Usually used in Constant value to store on a common destination. Usually used in
conjunction with ``action="store_const"``. conjunction with ``action="store_const"``.
:param choices: :param iterable choices:
Possible values for the option. Possible values for the option.
:param help: :param str help:
Help text displayed in the usage information. Help text displayed in the usage information.
:param metavar: :param str metavar:
Name to use instead of the long option name for help text. Name to use instead of the long option name for help text.
:param required: :param bool required:
Whether this option is required or not. Whether this option is required or not.
The following options may be passed directly through to :mod:`argparse` The following options may be passed directly through to :mod:`argparse`
@ -95,18 +157,30 @@ class Option:
:param type: :param type:
A callable to normalize the type (as is the case in A callable to normalize the type (as is the case in
:mod:`argparse`). :mod:`argparse`). Deprecated: you can also pass through type
:param action: strings such as ``'int'`` which are handled by :mod:`optparse`.
Any action allowed by :mod:`argparse`. :param str action:
Any action allowed by :mod:`argparse`. Deprecated: this also
understands the ``action='callback'`` action from :mod:`optparse`.
:param callable callback:
Callback used if the action is ``"callback"``. Deprecated: please
use ``action=`` instead.
:param iterable callback_args:
Additional positional arguments to the callback callable.
Deprecated: please use ``action=`` instead (probably with
``functools.partial``).
:param dictionary callback_kwargs:
Keyword arguments to the callback callable. Deprecated: please
use ``action=`` instead (probably with ``functools.partial``).
The following parameters are for Flake8's option handling alone. The following parameters are for Flake8's option handling alone.
:param parse_from_config: :param bool parse_from_config:
Whether or not this option should be parsed out of config files. Whether or not this option should be parsed out of config files.
:param comma_separated_list: :param bool comma_separated_list:
Whether the option is a comma separated list when parsing from a Whether the option is a comma separated list when parsing from a
config file. config file.
:param normalize_paths: :param bool normalize_paths:
Whether the option is expecting a path or list of paths and should Whether the option is expecting a path or list of paths and should
attempt to normalize the paths to absolute paths. attempt to normalize the paths to absolute paths.
""" """
@ -117,6 +191,37 @@ class Option:
): ):
short_option_name, long_option_name = _ARG.NO, short_option_name short_option_name, long_option_name = _ARG.NO, short_option_name
# optparse -> argparse `%default` => `%(default)s`
if help is not _ARG.NO and "%default" in help:
LOG.warning(
"option %s: please update `help=` text to use %%(default)s "
"instead of %%default -- this will be an error in the future",
long_option_name,
)
help = help.replace("%default", "%(default)s")
# optparse -> argparse for `callback`
if action == "callback":
LOG.warning(
"option %s: please update from optparse `action='callback'` "
"to argparse action classes -- this will be an error in the "
"future",
long_option_name,
)
action = _CallbackAction
if type is _ARG.NO:
nargs = 0
# optparse -> argparse for `type`
if isinstance(type, str):
LOG.warning(
"option %s: please update from optparse string `type=` to "
"argparse callable `type=` -- this will be an error in the "
"future",
long_option_name,
)
type = _optparse_callable_map[type]
# flake8 special type normalization # flake8 special type normalization
if comma_separated_list or normalize_paths: if comma_separated_list or normalize_paths:
type = functools.partial( type = functools.partial(
@ -139,10 +244,13 @@ class Option:
self.nargs = nargs self.nargs = nargs
self.const = const self.const = const
self.choices = choices self.choices = choices
self.callback = callback
self.callback_args = callback_args
self.callback_kwargs = callback_kwargs
self.help = help self.help = help
self.metavar = metavar self.metavar = metavar
self.required = required self.required = required
self.option_kwargs: dict[str, Any | _ARG] = { self.option_kwargs: Dict[str, Union[Any, _ARG]] = {
"action": self.action, "action": self.action,
"default": self.default, "default": self.default,
"type": self.type, "type": self.type,
@ -150,6 +258,9 @@ class Option:
"nargs": self.nargs, "nargs": self.nargs,
"const": self.const, "const": self.const,
"choices": self.choices, "choices": self.choices,
"callback": self.callback,
"callback_args": self.callback_args,
"callback_kwargs": self.callback_kwargs,
"help": self.help, "help": self.help,
"metavar": self.metavar, "metavar": self.metavar,
"required": self.required, "required": self.required,
@ -160,19 +271,19 @@ class Option:
self.comma_separated_list = comma_separated_list self.comma_separated_list = comma_separated_list
self.normalize_paths = normalize_paths self.normalize_paths = normalize_paths
self.config_name: str | None = None self.config_name: Optional[str] = None
if parse_from_config: if parse_from_config:
if long_option_name is _ARG.NO: if long_option_name is _ARG.NO:
raise ValueError( raise ValueError(
"When specifying parse_from_config=True, " "When specifying parse_from_config=True, "
"a long_option_name must also be specified.", "a long_option_name must also be specified."
) )
self.config_name = long_option_name[2:].replace("-", "_") self.config_name = long_option_name[2:].replace("-", "_")
self._opt = None self._opt = None
@property @property
def filtered_option_kwargs(self) -> dict[str, Any]: def filtered_option_kwargs(self) -> Dict[str, Any]:
"""Return any actually-specified arguments.""" """Return any actually-specified arguments."""
return { return {
k: v for k, v in self.option_kwargs.items() if v is not _ARG.NO k: v for k, v in self.option_kwargs.items() if v is not _ARG.NO
@ -199,70 +310,93 @@ class Option:
return value return value
def to_argparse(self) -> tuple[list[str], dict[str, Any]]: def normalize_from_setuptools(
self, value: str
) -> Union[int, float, complex, bool, str]:
"""Normalize the value received from setuptools."""
value = self.normalize(value)
if self.type is int or self.action == "count":
return int(value)
elif self.type is float:
return float(value)
elif self.type is complex:
return complex(value)
if self.action in ("store_true", "store_false"):
value = str(value).upper()
if value in ("1", "T", "TRUE", "ON"):
return True
if value in ("0", "F", "FALSE", "OFF"):
return False
return value
def to_argparse(self) -> Tuple[List[str], Dict[str, Any]]:
"""Convert a Flake8 Option to argparse ``add_argument`` arguments.""" """Convert a Flake8 Option to argparse ``add_argument`` arguments."""
return self.option_args, self.filtered_option_kwargs return self.option_args, self.filtered_option_kwargs
@property
def to_optparse(self) -> "NoReturn":
"""No longer functional."""
raise AttributeError("to_optparse: flake8 now uses argparse")
PluginVersion = collections.namedtuple(
"PluginVersion", ["name", "version", "local"]
)
class OptionManager: class OptionManager:
"""Manage Options and OptionParser while adding post-processing.""" """Manage Options and OptionParser while adding post-processing."""
def __init__( def __init__(
self, self,
*, prog: str,
version: str, version: str,
plugin_versions: str, usage: str = "%(prog)s [options] file file ...",
parents: list[argparse.ArgumentParser], parents: Optional[List[argparse.ArgumentParser]] = None,
formatter_names: list[str], ) -> None: # noqa: E501
) -> None: """Initialize an instance of an OptionManager.
"""Initialize an instance of an OptionManager."""
self.formatter_names = formatter_names :param str prog:
self.parser = argparse.ArgumentParser( Name of the actual program (e.g., flake8).
prog="flake8", :param str version:
usage="%(prog)s [options] file file ...", Version string for the program.
parents=parents, :param str usage:
epilog=f"Installed plugins: {plugin_versions}", Basic usage string used by the OptionParser.
:param argparse.ArgumentParser parents:
A list of ArgumentParser objects whose arguments should also be
included.
"""
if parents is None:
parents = []
self.parser: argparse.ArgumentParser = argparse.ArgumentParser(
prog=prog, usage=usage, parents=parents
) )
self.parser.add_argument( self._current_group: Optional[argparse._ArgumentGroup] = None
"--version", self.version_action = cast(
action="version", "argparse._VersionAction",
version=( self.parser.add_argument(
f"{version} ({plugin_versions}) " "--version", action="version", version=version
f"{utils.get_python_version()}"
), ),
) )
self.parser.add_argument("filenames", nargs="*", metavar="filename") self.parser.add_argument("filenames", nargs="*", metavar="filename")
self.config_options_dict: Dict[str, Option] = {}
self.options: List[Option] = []
self.program_name = prog
self.version = version
self.registered_plugins: Set[PluginVersion] = set()
self.extended_default_ignore: Set[str] = set()
self.extended_default_select: Set[str] = set()
self.config_options_dict: dict[str, Option] = {} @contextlib.contextmanager
self.options: list[Option] = [] def group(self, name: str) -> Generator[None, None, None]:
self.extended_default_ignore: list[str] = [] """Attach options to an argparse group during this context."""
self.extended_default_select: list[str] = [] group = self.parser.add_argument_group(name)
self._current_group, orig_group = group, self._current_group
self._current_group: argparse._ArgumentGroup | None = None try:
yield
# TODO: maybe make this a free function to reduce api surface area finally:
def register_plugins(self, plugins: Plugins) -> None: self._current_group = orig_group
"""Register the plugin options (if needed)."""
groups: dict[str, argparse._ArgumentGroup] = {}
def _set_group(name: str) -> None:
try:
self._current_group = groups[name]
except KeyError:
group = self.parser.add_argument_group(name)
self._current_group = groups[name] = group
for loaded in plugins.all_plugins():
add_options = getattr(loaded.obj, "add_options", None)
if add_options:
_set_group(loaded.plugin.package)
add_options(self)
if loaded.plugin.entry_point.group == "flake8.extension":
self.extend_default_select([loaded.entry_name])
# isn't strictly necessary, but seems cleaner
self._current_group = None
def add_option(self, *args: Any, **kwargs: Any) -> None: def add_option(self, *args: Any, **kwargs: Any) -> None:
"""Create and register a new option. """Create and register a new option.
@ -284,37 +418,108 @@ class OptionManager:
self.options.append(option) self.options.append(option)
if option.parse_from_config: if option.parse_from_config:
name = option.config_name name = option.config_name
assert name is not None assert name is not None # nosec (for mypy)
self.config_options_dict[name] = option self.config_options_dict[name] = option
self.config_options_dict[name.replace("_", "-")] = option self.config_options_dict[name.replace("_", "-")] = option
LOG.debug('Registered option "%s".', option) LOG.debug('Registered option "%s".', option)
def remove_from_default_ignore(self, error_codes: Sequence[str]) -> None:
"""Remove specified error codes from the default ignore list.
:param list error_codes:
List of strings that are the error/warning codes to attempt to
remove from the extended default ignore list.
"""
LOG.debug("Removing %r from the default ignore list", error_codes)
for error_code in error_codes:
try:
self.extended_default_ignore.remove(error_code)
except (ValueError, KeyError):
LOG.debug(
"Attempted to remove %s from default ignore"
" but it was not a member of the list.",
error_code,
)
def extend_default_ignore(self, error_codes: Sequence[str]) -> None: def extend_default_ignore(self, error_codes: Sequence[str]) -> None:
"""Extend the default ignore list with the error codes provided. """Extend the default ignore list with the error codes provided.
:param error_codes: :param list error_codes:
List of strings that are the error/warning codes with which to List of strings that are the error/warning codes with which to
extend the default ignore list. extend the default ignore list.
""" """
LOG.debug("Extending default ignore list with %r", error_codes) LOG.debug("Extending default ignore list with %r", error_codes)
self.extended_default_ignore.extend(error_codes) self.extended_default_ignore.update(error_codes)
def extend_default_select(self, error_codes: Sequence[str]) -> None: def extend_default_select(self, error_codes: Sequence[str]) -> None:
"""Extend the default select list with the error codes provided. """Extend the default select list with the error codes provided.
:param error_codes: :param list error_codes:
List of strings that are the error/warning codes with which List of strings that are the error/warning codes with which
to extend the default select list. to extend the default select list.
""" """
LOG.debug("Extending default select list with %r", error_codes) LOG.debug("Extending default select list with %r", error_codes)
self.extended_default_select.extend(error_codes) self.extended_default_select.update(error_codes)
def generate_versions(
self, format_str: str = "%(name)s: %(version)s", join_on: str = ", "
) -> str:
"""Generate a comma-separated list of versions of plugins."""
return join_on.join(
format_str % plugin._asdict()
for plugin in sorted(self.registered_plugins)
)
def update_version_string(self) -> None:
"""Update the flake8 version string."""
self.version_action.version = "{} ({}) {}".format(
self.version, self.generate_versions(), utils.get_python_version()
)
def generate_epilog(self) -> None:
"""Create an epilog with the version and name of each of plugin."""
plugin_version_format = "%(name)s: %(version)s"
self.parser.epilog = "Installed plugins: " + self.generate_versions(
plugin_version_format
)
def parse_args( def parse_args(
self, self,
args: Sequence[str] | None = None, args: Optional[List[str]] = None,
values: argparse.Namespace | None = None, values: Optional[argparse.Namespace] = None,
) -> argparse.Namespace: ) -> Tuple[argparse.Namespace, List[str]]:
"""Proxy to calling the OptionParser's parse_args method.""" """Proxy to calling the OptionParser's parse_args method."""
self.generate_epilog()
self.update_version_string()
if values: if values:
self.parser.set_defaults(**vars(values)) self.parser.set_defaults(**vars(values))
return self.parser.parse_args(args) parsed_args = self.parser.parse_args(args)
# TODO: refactor callers to not need this
return parsed_args, parsed_args.filenames
def parse_known_args(
self, args: Optional[List[str]] = None
) -> Tuple[argparse.Namespace, List[str]]:
"""Parse only the known arguments from the argument values.
Replicate a little argparse behaviour while we're still on
optparse.
"""
self.generate_epilog()
self.update_version_string()
return self.parser.parse_known_args(args)
def register_plugin(
self, name: str, version: str, local: bool = False
) -> None:
"""Register a plugin relying on the OptionManager.
:param str name:
The name of the checker itself. This will be the ``name``
attribute of the class or function loaded from the entry-point.
:param str version:
The version of the checker that we're using.
:param bool local:
Whether the plugin is local to the project/repository or not.
"""
self.registered_plugins.add(PluginVersion(name, version, local))

View file

@ -1,70 +0,0 @@
"""Procedure for parsing args, config, loading plugins."""
from __future__ import annotations
import argparse
from collections.abc import Sequence
import flake8
from flake8.main import options
from flake8.options import aggregator
from flake8.options import config
from flake8.options import manager
from flake8.plugins import finder
def parse_args(
argv: Sequence[str],
) -> tuple[finder.Plugins, argparse.Namespace]:
"""Procedure for parsing args, config, loading plugins."""
prelim_parser = options.stage1_arg_parser()
args0, rest = prelim_parser.parse_known_args(argv)
# XXX (ericvw): Special case "forwarding" the output file option so
# that it can be reparsed again for the BaseFormatter.filename.
if args0.output_file:
rest.extend(("--output-file", args0.output_file))
flake8.configure_logging(args0.verbose, args0.output_file)
cfg, cfg_dir = config.load_config(
config=args0.config,
extra=args0.append_config,
isolated=args0.isolated,
)
plugin_opts = finder.parse_plugin_options(
cfg,
cfg_dir,
enable_extensions=args0.enable_extensions,
require_plugins=args0.require_plugins,
)
raw_plugins = finder.find_plugins(cfg, plugin_opts)
plugins = finder.load_plugins(raw_plugins, plugin_opts)
option_manager = manager.OptionManager(
version=flake8.__version__,
plugin_versions=plugins.versions_str(),
parents=[prelim_parser],
formatter_names=list(plugins.reporters),
)
options.register_default_options(option_manager)
option_manager.register_plugins(plugins)
opts = aggregator.aggregate_options(option_manager, cfg, cfg_dir, rest)
for loaded in plugins.all_plugins():
parse_options = getattr(loaded.obj, "parse_options", None)
if parse_options is None:
continue
# XXX: ideally we wouldn't have two forms of parse_options
try:
parse_options(
option_manager,
opts,
opts.filenames,
)
except TypeError:
parse_options(opts)
return plugins, opts

View file

@ -1,2 +1 @@
"""Submodule of built-in plugins and plugin managers.""" """Submodule of built-in plugins and plugin managers."""
from __future__ import annotations

View file

@ -1,365 +0,0 @@
"""Functions related to finding and loading plugins."""
from __future__ import annotations
import configparser
import importlib.metadata
import inspect
import itertools
import logging
import sys
from collections.abc import Generator
from collections.abc import Iterable
from typing import Any
from typing import NamedTuple
from flake8 import utils
from flake8.defaults import VALID_CODE_PREFIX
from flake8.exceptions import ExecutionError
from flake8.exceptions import FailedToLoadPlugin
LOG = logging.getLogger(__name__)
FLAKE8_GROUPS = frozenset(("flake8.extension", "flake8.report"))
BANNED_PLUGINS = {
"flake8-colors": "5.0",
"flake8-per-file-ignores": "3.7",
}
class Plugin(NamedTuple):
"""A plugin before loading."""
package: str
version: str
entry_point: importlib.metadata.EntryPoint
class LoadedPlugin(NamedTuple):
"""Represents a plugin after being imported."""
plugin: Plugin
obj: Any
parameters: dict[str, bool]
@property
def entry_name(self) -> str:
"""Return the name given in the packaging metadata."""
return self.plugin.entry_point.name
@property
def display_name(self) -> str:
"""Return the name for use in user-facing / error messages."""
return f"{self.plugin.package}[{self.entry_name}]"
class Checkers(NamedTuple):
"""Classified plugins needed for checking."""
tree: list[LoadedPlugin]
logical_line: list[LoadedPlugin]
physical_line: list[LoadedPlugin]
class Plugins(NamedTuple):
"""Classified plugins."""
checkers: Checkers
reporters: dict[str, LoadedPlugin]
disabled: list[LoadedPlugin]
def all_plugins(self) -> Generator[LoadedPlugin]:
"""Return an iterator over all :class:`LoadedPlugin`s."""
yield from self.checkers.tree
yield from self.checkers.logical_line
yield from self.checkers.physical_line
yield from self.reporters.values()
def versions_str(self) -> str:
"""Return a user-displayed list of plugin versions."""
return ", ".join(
sorted(
{
f"{loaded.plugin.package}: {loaded.plugin.version}"
for loaded in self.all_plugins()
if loaded.plugin.package not in {"flake8", "local"}
},
),
)
class PluginOptions(NamedTuple):
"""Options related to plugin loading."""
local_plugin_paths: tuple[str, ...]
enable_extensions: frozenset[str]
require_plugins: frozenset[str]
@classmethod
def blank(cls) -> PluginOptions:
"""Make a blank PluginOptions, mostly used for tests."""
return cls(
local_plugin_paths=(),
enable_extensions=frozenset(),
require_plugins=frozenset(),
)
def _parse_option(
cfg: configparser.RawConfigParser,
cfg_opt_name: str,
opt: str | None,
) -> list[str]:
# specified on commandline: use that
if opt is not None:
return utils.parse_comma_separated_list(opt)
else:
# ideally this would reuse our config parsing framework but we need to
# parse this from preliminary options before plugins are enabled
for opt_name in (cfg_opt_name, cfg_opt_name.replace("_", "-")):
val = cfg.get("flake8", opt_name, fallback=None)
if val is not None:
return utils.parse_comma_separated_list(val)
else:
return []
def parse_plugin_options(
cfg: configparser.RawConfigParser,
cfg_dir: str,
*,
enable_extensions: str | None,
require_plugins: str | None,
) -> PluginOptions:
"""Parse plugin loading related options."""
paths_s = cfg.get("flake8:local-plugins", "paths", fallback="").strip()
paths = utils.parse_comma_separated_list(paths_s)
paths = utils.normalize_paths(paths, cfg_dir)
return PluginOptions(
local_plugin_paths=tuple(paths),
enable_extensions=frozenset(
_parse_option(cfg, "enable_extensions", enable_extensions),
),
require_plugins=frozenset(
_parse_option(cfg, "require_plugins", require_plugins),
),
)
def _flake8_plugins(
eps: Iterable[importlib.metadata.EntryPoint],
name: str,
version: str,
) -> Generator[Plugin]:
pyflakes_meta = importlib.metadata.distribution("pyflakes").metadata
pycodestyle_meta = importlib.metadata.distribution("pycodestyle").metadata
for ep in eps:
if ep.group not in FLAKE8_GROUPS:
continue
if ep.name == "F":
yield Plugin(pyflakes_meta["name"], pyflakes_meta["version"], ep)
elif ep.name in "EW":
# pycodestyle provides both `E` and `W` -- but our default select
# handles those
# ideally pycodestyle's plugin entrypoints would exactly represent
# the codes they produce...
yield Plugin(
pycodestyle_meta["name"], pycodestyle_meta["version"], ep,
)
else:
yield Plugin(name, version, ep)
def _find_importlib_plugins() -> Generator[Plugin]:
# some misconfigured pythons (RHEL) have things on `sys.path` twice
seen = set()
for dist in importlib.metadata.distributions():
# assigned to prevent continual reparsing
eps = dist.entry_points
# perf: skip parsing `.metadata` (slow) if no entry points match
if not any(ep.group in FLAKE8_GROUPS for ep in eps):
continue
# assigned to prevent continual reparsing
meta = dist.metadata
if meta["name"] in seen:
continue
else:
seen.add(meta["name"])
if meta["name"] in BANNED_PLUGINS:
LOG.warning(
"%s plugin is obsolete in flake8>=%s",
meta["name"],
BANNED_PLUGINS[meta["name"]],
)
continue
elif meta["name"] == "flake8":
# special case flake8 which provides plugins for pyflakes /
# pycodestyle
yield from _flake8_plugins(eps, meta["name"], meta["version"])
continue
for ep in eps:
if ep.group in FLAKE8_GROUPS:
yield Plugin(meta["name"], meta["version"], ep)
def _find_local_plugins(
cfg: configparser.RawConfigParser,
) -> Generator[Plugin]:
for plugin_type in ("extension", "report"):
group = f"flake8.{plugin_type}"
for plugin_s in utils.parse_comma_separated_list(
cfg.get("flake8:local-plugins", plugin_type, fallback="").strip(),
regexp=utils.LOCAL_PLUGIN_LIST_RE,
):
name, _, entry_str = plugin_s.partition("=")
name, entry_str = name.strip(), entry_str.strip()
ep = importlib.metadata.EntryPoint(name, entry_str, group)
yield Plugin("local", "local", ep)
def _check_required_plugins(
plugins: list[Plugin],
expected: frozenset[str],
) -> None:
plugin_names = {
utils.normalize_pypi_name(plugin.package) for plugin in plugins
}
expected_names = {utils.normalize_pypi_name(name) for name in expected}
missing_plugins = expected_names - plugin_names
if missing_plugins:
raise ExecutionError(
f"required plugins were not installed!\n"
f"- installed: {', '.join(sorted(plugin_names))}\n"
f"- expected: {', '.join(sorted(expected_names))}\n"
f"- missing: {', '.join(sorted(missing_plugins))}",
)
def find_plugins(
cfg: configparser.RawConfigParser,
opts: PluginOptions,
) -> list[Plugin]:
"""Discovers all plugins (but does not load them)."""
ret = [*_find_importlib_plugins(), *_find_local_plugins(cfg)]
# for determinism, sort the list
ret.sort()
_check_required_plugins(ret, opts.require_plugins)
return ret
def _parameters_for(func: Any) -> dict[str, bool]:
"""Return the parameters for the plugin.
This will inspect the plugin and return either the function parameters
if the plugin is a function or the parameters for ``__init__`` after
``self`` if the plugin is a class.
:returns:
A dictionary mapping the parameter name to whether or not it is
required (a.k.a., is positional only/does not have a default).
"""
is_class = not inspect.isfunction(func)
if is_class:
func = func.__init__
parameters = {
parameter.name: parameter.default is inspect.Parameter.empty
for parameter in inspect.signature(func).parameters.values()
if parameter.kind is inspect.Parameter.POSITIONAL_OR_KEYWORD
}
if is_class:
parameters.pop("self", None)
return parameters
def _load_plugin(plugin: Plugin) -> LoadedPlugin:
try:
obj = plugin.entry_point.load()
except Exception as e:
raise FailedToLoadPlugin(plugin.package, e)
if not callable(obj):
err = TypeError("expected loaded plugin to be callable")
raise FailedToLoadPlugin(plugin.package, err)
return LoadedPlugin(plugin, obj, _parameters_for(obj))
def _import_plugins(
plugins: list[Plugin],
opts: PluginOptions,
) -> list[LoadedPlugin]:
sys.path.extend(opts.local_plugin_paths)
return [_load_plugin(p) for p in plugins]
def _classify_plugins(
plugins: list[LoadedPlugin],
opts: PluginOptions,
) -> Plugins:
tree = []
logical_line = []
physical_line = []
reporters = {}
disabled = []
for loaded in plugins:
if (
getattr(loaded.obj, "off_by_default", False)
and loaded.plugin.entry_point.name not in opts.enable_extensions
):
disabled.append(loaded)
elif loaded.plugin.entry_point.group == "flake8.report":
reporters[loaded.entry_name] = loaded
elif "tree" in loaded.parameters:
tree.append(loaded)
elif "logical_line" in loaded.parameters:
logical_line.append(loaded)
elif "physical_line" in loaded.parameters:
physical_line.append(loaded)
else:
raise NotImplementedError(f"what plugin type? {loaded}")
for loaded in itertools.chain(tree, logical_line, physical_line):
if not VALID_CODE_PREFIX.match(loaded.entry_name):
raise ExecutionError(
f"plugin code for `{loaded.display_name}` does not match "
f"{VALID_CODE_PREFIX.pattern}",
)
return Plugins(
checkers=Checkers(
tree=tree,
logical_line=logical_line,
physical_line=physical_line,
),
reporters=reporters,
disabled=disabled,
)
def load_plugins(
plugins: list[Plugin],
opts: PluginOptions,
) -> Plugins:
"""Load and classify all flake8 plugins.
- first: extends ``sys.path`` with ``paths`` (to import local plugins)
- next: converts the ``Plugin``s to ``LoadedPlugins``
- finally: classifies plugins into their specific types
"""
return _classify_plugins(_import_plugins(plugins, opts), opts)

View file

@ -0,0 +1,533 @@
"""Plugin loading and management logic and classes."""
import logging
from typing import Any
from typing import Dict
from typing import List
from typing import Optional
from typing import Set
from flake8 import exceptions
from flake8 import utils
from flake8._compat import importlib_metadata
LOG = logging.getLogger(__name__)
__all__ = ("Checkers", "Plugin", "PluginManager", "ReportFormatters")
NO_GROUP_FOUND = object()
class Plugin:
"""Wrap an EntryPoint from setuptools and other logic."""
def __init__(self, name, entry_point, local=False):
"""Initialize our Plugin.
:param str name:
Name of the entry-point as it was registered with setuptools.
:param entry_point:
EntryPoint returned by setuptools.
:type entry_point:
setuptools.EntryPoint
:param bool local:
Is this a repo-local plugin?
"""
self.name = name
self.entry_point = entry_point
self.local = local
self._plugin: Any = None
self._parameters = None
self._parameter_names: Optional[List[str]] = None
self._group = None
self._plugin_name = None
self._version = None
def __repr__(self) -> str:
"""Provide an easy to read description of the current plugin."""
return 'Plugin(name="{}", entry_point="{}")'.format(
self.name, self.entry_point.value
)
def to_dictionary(self):
"""Convert this plugin to a dictionary."""
return {
"name": self.name,
"parameters": self.parameters,
"parameter_names": self.parameter_names,
"plugin": self.plugin,
"plugin_name": self.plugin_name,
}
def is_in_a_group(self):
"""Determine if this plugin is in a group.
:returns:
True if the plugin is in a group, otherwise False.
:rtype:
bool
"""
return self.group() is not None
def group(self):
"""Find and parse the group the plugin is in."""
if self._group is None:
name = self.name.split(".", 1)
if len(name) > 1:
self._group = name[0]
else:
self._group = NO_GROUP_FOUND
if self._group is NO_GROUP_FOUND:
return None
return self._group
@property
def parameters(self):
"""List of arguments that need to be passed to the plugin."""
if self._parameters is None:
self._parameters = utils.parameters_for(self)
return self._parameters
@property
def parameter_names(self) -> List[str]:
"""List of argument names that need to be passed to the plugin."""
if self._parameter_names is None:
self._parameter_names = list(self.parameters)
return self._parameter_names
@property
def plugin(self):
"""Load and return the plugin associated with the entry-point.
This property implicitly loads the plugin and then caches it.
"""
self.load_plugin()
return self._plugin
@property
def version(self) -> str:
"""Return the version of the plugin."""
version = self._version
if version is None:
if self.is_in_a_group():
version = self._version = version_for(self)
else:
version = self._version = self.plugin.version
return version
@property
def plugin_name(self):
"""Return the name of the plugin."""
if self._plugin_name is None:
if self.is_in_a_group():
self._plugin_name = self.group()
else:
self._plugin_name = self.plugin.name
return self._plugin_name
@property
def off_by_default(self):
"""Return whether the plugin is ignored by default."""
return getattr(self.plugin, "off_by_default", False)
def execute(self, *args, **kwargs):
r"""Call the plugin with \*args and \*\*kwargs."""
return self.plugin(*args, **kwargs) # pylint: disable=not-callable
def _load(self):
self._plugin = self.entry_point.load()
if not callable(self._plugin):
msg = (
f"Plugin {self._plugin!r} is not a callable. It might be "
f"written for an older version of flake8 and might not work "
f"with this version"
)
LOG.critical(msg)
raise TypeError(msg)
def load_plugin(self):
"""Retrieve the plugin for this entry-point.
This loads the plugin, stores it on the instance and then returns it.
It does not reload it after the first time, it merely returns the
cached plugin.
:returns:
Nothing
"""
if self._plugin is None:
LOG.info('Loading plugin "%s" from entry-point.', self.name)
try:
self._load()
except Exception as load_exception:
LOG.exception(load_exception)
failed_to_load = exceptions.FailedToLoadPlugin(
plugin_name=self.name, exception=load_exception
)
LOG.critical(str(failed_to_load))
raise failed_to_load
def enable(self, optmanager, options=None):
"""Remove plugin name from the default ignore list."""
optmanager.remove_from_default_ignore([self.name])
optmanager.extend_default_select([self.name])
if not options:
return
try:
options.ignore.remove(self.name)
except (ValueError, KeyError):
LOG.debug(
"Attempted to remove %s from the ignore list but it was "
"not a member of the list.",
self.name,
)
def disable(self, optmanager):
"""Add the plugin name to the default ignore list."""
optmanager.extend_default_ignore([self.name])
def provide_options(self, optmanager, options, extra_args):
"""Pass the parsed options and extra arguments to the plugin."""
parse_options = getattr(self.plugin, "parse_options", None)
if parse_options is not None:
LOG.debug('Providing options to plugin "%s".', self.name)
try:
parse_options(optmanager, options, extra_args)
except TypeError:
parse_options(options)
if self.name in options.enable_extensions:
self.enable(optmanager, options)
def register_options(self, optmanager):
"""Register the plugin's command-line options on the OptionManager.
:param optmanager:
Instantiated OptionManager to register options on.
:type optmanager:
flake8.options.manager.OptionManager
:returns:
Nothing
"""
add_options = getattr(self.plugin, "add_options", None)
if add_options is not None:
LOG.debug(
'Registering options from plugin "%s" on OptionManager %r',
self.name,
optmanager,
)
with optmanager.group(self.plugin_name):
add_options(optmanager)
if self.off_by_default:
self.disable(optmanager)
class PluginManager: # pylint: disable=too-few-public-methods
"""Find and manage plugins consistently."""
def __init__(
self, namespace: str, local_plugins: Optional[List[str]] = None
) -> None:
"""Initialize the manager.
:param str namespace:
Namespace of the plugins to manage, e.g., 'flake8.extension'.
:param list local_plugins:
Plugins from config (as "X = path.to:Plugin" strings).
"""
self.namespace = namespace
self.plugins: Dict[str, Plugin] = {}
self.names: List[str] = []
self._load_local_plugins(local_plugins or [])
self._load_entrypoint_plugins()
def _load_local_plugins(self, local_plugins):
"""Load local plugins from config.
:param list local_plugins:
Plugins from config (as "X = path.to:Plugin" strings).
"""
for plugin_str in local_plugins:
name, _, entry_str = plugin_str.partition("=")
name, entry_str = name.strip(), entry_str.strip()
entry_point = importlib_metadata.EntryPoint(
name, entry_str, self.namespace
)
self._load_plugin_from_entrypoint(entry_point, local=True)
def _load_entrypoint_plugins(self):
LOG.info('Loading entry-points for "%s".', self.namespace)
eps = importlib_metadata.entry_points().get(self.namespace, ())
# python2.7 occasionally gives duplicate results due to redundant
# `local/lib` -> `../lib` symlink on linux in virtualenvs so we
# eliminate duplicates here
for entry_point in sorted(frozenset(eps)):
if entry_point.name == "per-file-ignores":
LOG.warning(
"flake8-per-file-ignores plugin is incompatible with "
"flake8>=3.7 (which implements per-file-ignores itself)."
)
continue
self._load_plugin_from_entrypoint(entry_point)
def _load_plugin_from_entrypoint(self, entry_point, local=False):
"""Load a plugin from a setuptools EntryPoint.
:param EntryPoint entry_point:
EntryPoint to load plugin from.
:param bool local:
Is this a repo-local plugin?
"""
name = entry_point.name
self.plugins[name] = Plugin(name, entry_point, local=local)
self.names.append(name)
LOG.debug('Loaded %r for plugin "%s".', self.plugins[name], name)
def map(self, func, *args, **kwargs):
r"""Call ``func`` with the plugin and \*args and \**kwargs after.
This yields the return value from ``func`` for each plugin.
:param collections.Callable func:
Function to call with each plugin. Signature should at least be:
.. code-block:: python
def myfunc(plugin):
pass
Any extra positional or keyword arguments specified with map will
be passed along to this function after the plugin. The plugin
passed is a :class:`~flake8.plugins.manager.Plugin`.
:param args:
Positional arguments to pass to ``func`` after each plugin.
:param kwargs:
Keyword arguments to pass to ``func`` after each plugin.
"""
for name in self.names:
yield func(self.plugins[name], *args, **kwargs)
def versions(self):
# () -> (str, str)
"""Generate the versions of plugins.
:returns:
Tuples of the plugin_name and version
:rtype:
tuple
"""
plugins_seen: Set[str] = set()
for entry_point_name in self.names:
plugin = self.plugins[entry_point_name]
plugin_name = plugin.plugin_name
if plugin.plugin_name in plugins_seen:
continue
plugins_seen.add(plugin_name)
yield (plugin_name, plugin.version)
def version_for(plugin):
# (Plugin) -> Optional[str]
"""Determine the version of a plugin by its module.
:param plugin:
The loaded plugin
:type plugin:
Plugin
:returns:
version string for the module
:rtype:
str
"""
module_name = plugin.plugin.__module__
try:
module = __import__(module_name)
except ImportError:
return None
return getattr(module, "__version__", None)
class PluginTypeManager:
"""Parent class for most of the specific plugin types."""
namespace: str
def __init__(self, local_plugins=None):
"""Initialize the plugin type's manager.
:param list local_plugins:
Plugins from config file instead of entry-points
"""
self.manager = PluginManager(
self.namespace, local_plugins=local_plugins
)
self.plugins_loaded = False
def __contains__(self, name):
"""Check if the entry-point name is in this plugin type manager."""
LOG.debug('Checking for "%s" in plugin type manager.', name)
return name in self.plugins
def __getitem__(self, name):
"""Retrieve a plugin by its name."""
LOG.debug('Retrieving plugin for "%s".', name)
return self.plugins[name]
def get(self, name, default=None):
"""Retrieve the plugin referred to by ``name`` or return the default.
:param str name:
Name of the plugin to retrieve.
:param default:
Default value to return.
:returns:
Plugin object referred to by name, if it exists.
:rtype:
:class:`Plugin`
"""
if name in self:
return self[name]
return default
@property
def names(self):
"""Proxy attribute to underlying manager."""
return self.manager.names
@property
def plugins(self):
"""Proxy attribute to underlying manager."""
return self.manager.plugins
@staticmethod
def _generate_call_function(method_name, optmanager, *args, **kwargs):
def generated_function(plugin):
method = getattr(plugin, method_name, None)
if method is not None and callable(method):
return method(optmanager, *args, **kwargs)
return generated_function
def load_plugins(self):
"""Load all plugins of this type that are managed by this manager."""
if self.plugins_loaded:
return
def load_plugin(plugin):
"""Call each plugin's load_plugin method."""
return plugin.load_plugin()
plugins = list(self.manager.map(load_plugin))
# Do not set plugins_loaded if we run into an exception
self.plugins_loaded = True
return plugins
def register_plugin_versions(self, optmanager):
"""Register the plugins and their versions with the OptionManager."""
self.load_plugins()
for (plugin_name, version) in self.manager.versions():
optmanager.register_plugin(name=plugin_name, version=version)
def register_options(self, optmanager):
"""Register all of the checkers' options to the OptionManager."""
self.load_plugins()
call_register_options = self._generate_call_function(
"register_options", optmanager
)
list(self.manager.map(call_register_options))
def provide_options(self, optmanager, options, extra_args):
"""Provide parsed options and extra arguments to the plugins."""
call_provide_options = self._generate_call_function(
"provide_options", optmanager, options, extra_args
)
list(self.manager.map(call_provide_options))
class Checkers(PluginTypeManager):
"""All of the checkers registered through entry-points or config."""
namespace = "flake8.extension"
def checks_expecting(self, argument_name):
"""Retrieve checks that expect an argument with the specified name.
Find all checker plugins that are expecting a specific argument.
"""
for plugin in self.plugins.values():
if argument_name == plugin.parameter_names[0]:
yield plugin
def to_dictionary(self):
"""Return a dictionary of AST and line-based plugins."""
return {
"ast_plugins": [
plugin.to_dictionary() for plugin in self.ast_plugins
],
"logical_line_plugins": [
plugin.to_dictionary() for plugin in self.logical_line_plugins
],
"physical_line_plugins": [
plugin.to_dictionary() for plugin in self.physical_line_plugins
],
}
def register_options(self, optmanager):
"""Register all of the checkers' options to the OptionManager.
This also ensures that plugins that are not part of a group and are
enabled by default are enabled on the option manager.
"""
# NOTE(sigmavirus24) We reproduce a little of
# PluginTypeManager.register_options to reduce the number of times
# that we loop over the list of plugins. Instead of looping twice,
# option registration and enabling the plugin, we loop once with one
# function to map over the plugins.
self.load_plugins()
call_register_options = self._generate_call_function(
"register_options", optmanager
)
def register_and_enable(plugin):
call_register_options(plugin)
if plugin.group() is None and not plugin.off_by_default:
plugin.enable(optmanager)
list(self.manager.map(register_and_enable))
@property
def ast_plugins(self):
"""List of plugins that expect the AST tree."""
plugins = getattr(self, "_ast_plugins", [])
if not plugins:
plugins = list(self.checks_expecting("tree"))
self._ast_plugins = plugins
return plugins
@property
def logical_line_plugins(self):
"""List of plugins that expect the logical lines."""
plugins = getattr(self, "_logical_line_plugins", [])
if not plugins:
plugins = list(self.checks_expecting("logical_line"))
self._logical_line_plugins = plugins
return plugins
@property
def physical_line_plugins(self):
"""List of plugins that expect the physical lines."""
plugins = getattr(self, "_physical_line_plugins", [])
if not plugins:
plugins = list(self.checks_expecting("physical_line"))
self._physical_line_plugins = plugins
return plugins
class ReportFormatters(PluginTypeManager):
"""All of the report formatters registered through entry-points/config."""
namespace = "flake8.report"

View file

@ -1,112 +0,0 @@
"""Generated using ./bin/gen-pycodestyle-plugin."""
# fmt: off
from __future__ import annotations
from collections.abc import Generator
from typing import Any
from pycodestyle import ambiguous_identifier as _ambiguous_identifier
from pycodestyle import bare_except as _bare_except
from pycodestyle import blank_lines as _blank_lines
from pycodestyle import break_after_binary_operator as _break_after_binary_operator # noqa: E501
from pycodestyle import break_before_binary_operator as _break_before_binary_operator # noqa: E501
from pycodestyle import comparison_negative as _comparison_negative
from pycodestyle import comparison_to_singleton as _comparison_to_singleton
from pycodestyle import comparison_type as _comparison_type
from pycodestyle import compound_statements as _compound_statements
from pycodestyle import continued_indentation as _continued_indentation
from pycodestyle import explicit_line_join as _explicit_line_join
from pycodestyle import extraneous_whitespace as _extraneous_whitespace
from pycodestyle import imports_on_separate_lines as _imports_on_separate_lines
from pycodestyle import indentation as _indentation
from pycodestyle import maximum_doc_length as _maximum_doc_length
from pycodestyle import maximum_line_length as _maximum_line_length
from pycodestyle import missing_whitespace as _missing_whitespace
from pycodestyle import missing_whitespace_after_keyword as _missing_whitespace_after_keyword # noqa: E501
from pycodestyle import module_imports_on_top_of_file as _module_imports_on_top_of_file # noqa: E501
from pycodestyle import python_3000_invalid_escape_sequence as _python_3000_invalid_escape_sequence # noqa: E501
from pycodestyle import tabs_obsolete as _tabs_obsolete
from pycodestyle import tabs_or_spaces as _tabs_or_spaces
from pycodestyle import trailing_blank_lines as _trailing_blank_lines
from pycodestyle import trailing_whitespace as _trailing_whitespace
from pycodestyle import whitespace_around_comma as _whitespace_around_comma
from pycodestyle import whitespace_around_keywords as _whitespace_around_keywords # noqa: E501
from pycodestyle import whitespace_around_named_parameter_equals as _whitespace_around_named_parameter_equals # noqa: E501
from pycodestyle import whitespace_around_operator as _whitespace_around_operator # noqa: E501
from pycodestyle import whitespace_before_comment as _whitespace_before_comment
from pycodestyle import whitespace_before_parameters as _whitespace_before_parameters # noqa: E501
def pycodestyle_logical(
blank_before: Any,
blank_lines: Any,
checker_state: Any,
hang_closing: Any,
indent_char: Any,
indent_level: Any,
indent_size: Any,
line_number: Any,
lines: Any,
logical_line: Any,
max_doc_length: Any,
noqa: Any,
previous_indent_level: Any,
previous_logical: Any,
previous_unindented_logical_line: Any,
tokens: Any,
verbose: Any,
) -> Generator[tuple[int, str]]:
"""Run pycodestyle logical checks."""
yield from _ambiguous_identifier(logical_line, tokens)
yield from _bare_except(logical_line, noqa)
yield from _blank_lines(logical_line, blank_lines, indent_level, line_number, blank_before, previous_logical, previous_unindented_logical_line, previous_indent_level, lines) # noqa: E501
yield from _break_after_binary_operator(logical_line, tokens)
yield from _break_before_binary_operator(logical_line, tokens)
yield from _comparison_negative(logical_line)
yield from _comparison_to_singleton(logical_line, noqa)
yield from _comparison_type(logical_line, noqa)
yield from _compound_statements(logical_line)
yield from _continued_indentation(logical_line, tokens, indent_level, hang_closing, indent_char, indent_size, noqa, verbose) # noqa: E501
yield from _explicit_line_join(logical_line, tokens)
yield from _extraneous_whitespace(logical_line)
yield from _imports_on_separate_lines(logical_line)
yield from _indentation(logical_line, previous_logical, indent_char, indent_level, previous_indent_level, indent_size) # noqa: E501
yield from _maximum_doc_length(logical_line, max_doc_length, noqa, tokens)
yield from _missing_whitespace(logical_line, tokens)
yield from _missing_whitespace_after_keyword(logical_line, tokens)
yield from _module_imports_on_top_of_file(logical_line, indent_level, checker_state, noqa) # noqa: E501
yield from _python_3000_invalid_escape_sequence(logical_line, tokens, noqa)
yield from _whitespace_around_comma(logical_line)
yield from _whitespace_around_keywords(logical_line)
yield from _whitespace_around_named_parameter_equals(logical_line, tokens)
yield from _whitespace_around_operator(logical_line)
yield from _whitespace_before_comment(logical_line, tokens)
yield from _whitespace_before_parameters(logical_line, tokens)
def pycodestyle_physical(
indent_char: Any,
line_number: Any,
lines: Any,
max_line_length: Any,
multiline: Any,
noqa: Any,
physical_line: Any,
total_lines: Any,
) -> Generator[tuple[int, str]]:
"""Run pycodestyle physical checks."""
ret = _maximum_line_length(physical_line, max_line_length, multiline, line_number, noqa) # noqa: E501
if ret is not None:
yield ret
ret = _tabs_obsolete(physical_line)
if ret is not None:
yield ret
ret = _tabs_or_spaces(physical_line, indent_char)
if ret is not None:
yield ret
ret = _trailing_blank_lines(physical_line, lines, line_number, total_lines)
if ret is not None:
yield ret
ret = _trailing_whitespace(physical_line)
if ret is not None:
yield ret

View file

@ -1,17 +1,10 @@
"""Plugin built-in to Flake8 to treat pyflakes as a plugin.""" """Plugin built-in to Flake8 to treat pyflakes as a plugin."""
from __future__ import annotations import os
from typing import List
import argparse
import ast
import logging
from collections.abc import Generator
from typing import Any
import pyflakes.checker import pyflakes.checker
from flake8.options.manager import OptionManager from flake8 import utils
LOG = logging.getLogger(__name__)
FLAKE8_PYFLAKES_CODES = { FLAKE8_PYFLAKES_CODES = {
"UnusedImport": "F401", "UnusedImport": "F401",
@ -36,7 +29,6 @@ FLAKE8_PYFLAKES_CODES = {
"StringDotFormatMissingArgument": "F524", "StringDotFormatMissingArgument": "F524",
"StringDotFormatMixingAutomatic": "F525", "StringDotFormatMixingAutomatic": "F525",
"FStringMissingPlaceholders": "F541", "FStringMissingPlaceholders": "F541",
"TStringMissingPlaceholders": "F542",
"MultiValueRepeatedKeyLiteral": "F601", "MultiValueRepeatedKeyLiteral": "F601",
"MultiValueRepeatedKeyVariable": "F602", "MultiValueRepeatedKeyVariable": "F602",
"TooManyExpressionsInStarredAssignment": "F621", "TooManyExpressionsInStarredAssignment": "F621",
@ -47,19 +39,21 @@ FLAKE8_PYFLAKES_CODES = {
"IfTuple": "F634", "IfTuple": "F634",
"BreakOutsideLoop": "F701", "BreakOutsideLoop": "F701",
"ContinueOutsideLoop": "F702", "ContinueOutsideLoop": "F702",
"ContinueInFinally": "F703",
"YieldOutsideFunction": "F704", "YieldOutsideFunction": "F704",
"ReturnWithArgsInsideGenerator": "F705",
"ReturnOutsideFunction": "F706", "ReturnOutsideFunction": "F706",
"DefaultExceptNotLast": "F707", "DefaultExceptNotLast": "F707",
"DoctestSyntaxError": "F721", "DoctestSyntaxError": "F721",
"ForwardAnnotationSyntaxError": "F722", "ForwardAnnotationSyntaxError": "F722",
"CommentAnnotationSyntaxError": "F723",
"RedefinedWhileUnused": "F811", "RedefinedWhileUnused": "F811",
"RedefinedInListComp": "F812",
"UndefinedName": "F821", "UndefinedName": "F821",
"UndefinedExport": "F822", "UndefinedExport": "F822",
"UndefinedLocal": "F823", "UndefinedLocal": "F823",
"UnusedIndirectAssignment": "F824",
"DuplicateArgument": "F831", "DuplicateArgument": "F831",
"UnusedVariable": "F841", "UnusedVariable": "F841",
"UnusedAnnotation": "F842",
"RaiseNotImplemented": "F901", "RaiseNotImplemented": "F901",
} }
@ -67,16 +61,45 @@ FLAKE8_PYFLAKES_CODES = {
class FlakesChecker(pyflakes.checker.Checker): class FlakesChecker(pyflakes.checker.Checker):
"""Subclass the Pyflakes checker to conform with the flake8 API.""" """Subclass the Pyflakes checker to conform with the flake8 API."""
name = "pyflakes"
version = pyflakes.__version__
with_doctest = False with_doctest = False
include_in_doctest: List[str] = []
exclude_from_doctest: List[str] = []
def __init__(self, tree: ast.AST, filename: str) -> None: def __init__(self, tree, file_tokens, filename):
"""Initialize the PyFlakes plugin with an AST tree and filename.""" """Initialize the PyFlakes plugin with an AST tree and filename."""
filename = utils.normalize_path(filename)
with_doctest = self.with_doctest
included_by = [
include
for include in self.include_in_doctest
if include != "" and filename.startswith(include)
]
if included_by:
with_doctest = True
for exclude in self.exclude_from_doctest:
if exclude != "" and filename.startswith(exclude):
with_doctest = False
overlaped_by = [
include
for include in included_by
if include.startswith(exclude)
]
if overlaped_by:
with_doctest = True
super().__init__( super().__init__(
tree, filename=filename, withDoctest=self.with_doctest, tree,
filename=filename,
withDoctest=with_doctest,
file_tokens=file_tokens,
) )
@classmethod @classmethod
def add_options(cls, parser: OptionManager) -> None: def add_options(cls, parser):
"""Register options for PyFlakes on the Flake8 OptionManager.""" """Register options for PyFlakes on the Flake8 OptionManager."""
parser.add_option( parser.add_option(
"--builtins", "--builtins",
@ -91,15 +114,64 @@ class FlakesChecker(pyflakes.checker.Checker):
parse_from_config=True, parse_from_config=True,
help="also check syntax of the doctests", help="also check syntax of the doctests",
) )
parser.add_option(
"--include-in-doctest",
default="",
dest="include_in_doctest",
parse_from_config=True,
comma_separated_list=True,
normalize_paths=True,
help="Run doctests only on these files",
)
parser.add_option(
"--exclude-from-doctest",
default="",
dest="exclude_from_doctest",
parse_from_config=True,
comma_separated_list=True,
normalize_paths=True,
help="Skip these files when running doctests",
)
@classmethod @classmethod
def parse_options(cls, options: argparse.Namespace) -> None: def parse_options(cls, options):
"""Parse option values from Flake8's OptionManager.""" """Parse option values from Flake8's OptionManager."""
if options.builtins: if options.builtins:
cls.builtIns = cls.builtIns.union(options.builtins) cls.builtIns = cls.builtIns.union(options.builtins)
cls.with_doctest = options.doctests cls.with_doctest = options.doctests
def run(self) -> Generator[tuple[int, int, str, type[Any]]]: included_files = []
for included_file in options.include_in_doctest:
if included_file == "":
continue
if not included_file.startswith((os.sep, "./", "~/")):
included_files.append(f"./{included_file}")
else:
included_files.append(included_file)
cls.include_in_doctest = utils.normalize_paths(included_files)
excluded_files = []
for excluded_file in options.exclude_from_doctest:
if excluded_file == "":
continue
if not excluded_file.startswith((os.sep, "./", "~/")):
excluded_files.append(f"./{excluded_file}")
else:
excluded_files.append(excluded_file)
cls.exclude_from_doctest = utils.normalize_paths(excluded_files)
inc_exc = set(cls.include_in_doctest).intersection(
cls.exclude_from_doctest
)
if inc_exc:
raise ValueError(
f"{inc_exc!r} was specified in both the "
f"include-in-doctest and exclude-from-doctest "
f"options. You are not allowed to specify it in "
f"both for doctesting."
)
def run(self):
"""Run the plugin.""" """Run the plugin."""
for message in self.messages: for message in self.messages:
col = getattr(message, "col", 0) col = getattr(message, "col", 0)

View file

@ -1,42 +0,0 @@
"""Functions for constructing the requested report plugin."""
from __future__ import annotations
import argparse
import logging
from flake8.formatting.base import BaseFormatter
from flake8.plugins.finder import LoadedPlugin
LOG = logging.getLogger(__name__)
def make(
reporters: dict[str, LoadedPlugin],
options: argparse.Namespace,
) -> BaseFormatter:
"""Make the formatter from the requested user options.
- if :option:`flake8 --quiet` is specified, return the ``quiet-filename``
formatter.
- if :option:`flake8 --quiet` is specified at least twice, return the
``quiet-nothing`` formatter.
- otherwise attempt to return the formatter by name.
- failing that, assume it is a format string and return the ``default``
formatter.
"""
format_name = options.format
if options.quiet == 1:
format_name = "quiet-filename"
elif options.quiet >= 2:
format_name = "quiet-nothing"
try:
format_plugin = reporters[format_name]
except KeyError:
LOG.warning(
"%r is an unknown formatter. Falling back to default.",
format_name,
)
format_plugin = reporters["default"]
return format_plugin.obj(options)

View file

@ -1,35 +1,35 @@
"""Module containing our file processor that tokenizes a file for checks.""" """Module containing our file processor that tokenizes a file for checks."""
from __future__ import annotations
import argparse import argparse
import ast import ast
import functools import contextlib
import logging import logging
import tokenize import tokenize
from collections.abc import Generator
from typing import Any from typing import Any
from typing import Dict
from typing import Generator
from typing import List
from typing import Optional
from typing import Tuple
import flake8
from flake8 import defaults from flake8 import defaults
from flake8 import utils from flake8 import utils
from flake8._compat import FSTRING_END
from flake8._compat import FSTRING_MIDDLE
from flake8._compat import TSTRING_END
from flake8._compat import TSTRING_MIDDLE
from flake8.plugins.finder import LoadedPlugin
LOG = logging.getLogger(__name__) LOG = logging.getLogger(__name__)
PyCF_ONLY_AST = 1024
NEWLINE = frozenset([tokenize.NL, tokenize.NEWLINE]) NEWLINE = frozenset([tokenize.NL, tokenize.NEWLINE])
SKIP_TOKENS = frozenset( SKIP_TOKENS = frozenset(
[tokenize.NL, tokenize.NEWLINE, tokenize.INDENT, tokenize.DEDENT], [tokenize.NL, tokenize.NEWLINE, tokenize.INDENT, tokenize.DEDENT]
) )
_LogicalMapping = list[tuple[int, tuple[int, int]]] _Token = Tuple[int, str, Tuple[int, int], Tuple[int, int], str]
_Logical = tuple[list[str], list[str], _LogicalMapping] _LogicalMapping = List[Tuple[int, Tuple[int, int]]]
_Logical = Tuple[List[str], List[str], _LogicalMapping]
class FileProcessor: class FileProcessor:
"""Processes a file and holds state. """Processes a file and holdes state.
This processes a file by generating tokens, logical and physical lines, This processes a file by generating tokens, logical and physical lines,
and AST trees. This also provides a way of passing state about the file and AST trees. This also provides a way of passing state about the file
@ -63,11 +63,12 @@ class FileProcessor:
self, self,
filename: str, filename: str,
options: argparse.Namespace, options: argparse.Namespace,
lines: list[str] | None = None, lines: Optional[List[str]] = None,
) -> None: ) -> None:
"""Initialize our file processor. """Initialice our file processor.
:param filename: Name of the file to process :param str filename:
Name of the file to process
""" """
self.options = options self.options = options
self.filename = filename self.filename = filename
@ -80,17 +81,19 @@ class FileProcessor:
#: Number of blank lines #: Number of blank lines
self.blank_lines = 0 self.blank_lines = 0
#: Checker states for each plugin? #: Checker states for each plugin?
self._checker_states: dict[str, dict[Any, Any]] = {} self._checker_states: Dict[str, Dict[Any, Any]] = {}
#: Current checker state #: Current checker state
self.checker_state: dict[Any, Any] = {} self.checker_state: Dict[Any, Any] = {}
#: User provided option for hang closing #: User provided option for hang closing
self.hang_closing = options.hang_closing self.hang_closing = options.hang_closing
#: Character used for indentation #: Character used for indentation
self.indent_char: str | None = None self.indent_char: Optional[str] = None
#: Current level of indentation #: Current level of indentation
self.indent_level = 0 self.indent_level = 0
#: Number of spaces used for indentation #: Number of spaces used for indentation
self.indent_size = options.indent_size self.indent_size = options.indent_size
#: String representing the space indentation (DEPRECATED)
self.indent_size_str = str(self.indent_size)
#: Line number in the file #: Line number in the file
self.line_number = 0 self.line_number = 0
#: Current logical line #: Current logical line
@ -108,45 +111,36 @@ class FileProcessor:
#: Previous unindented (i.e. top-level) logical line #: Previous unindented (i.e. top-level) logical line
self.previous_unindented_logical_line = "" self.previous_unindented_logical_line = ""
#: Current set of tokens #: Current set of tokens
self.tokens: list[tokenize.TokenInfo] = [] self.tokens: List[_Token] = []
#: Total number of lines in the file #: Total number of lines in the file
self.total_lines = len(self.lines) self.total_lines = len(self.lines)
#: Verbosity level of Flake8 #: Verbosity level of Flake8
self.verbose = options.verbose self.verbose = options.verbose
#: Statistics dictionary #: Statistics dictionary
self.statistics = {"logical lines": 0} self.statistics = {"logical lines": 0}
self._fstring_start = self._tstring_start = -1 self._file_tokens: Optional[List[_Token]] = None
# map from line number to the line we'll search for `noqa` in
self._noqa_line_mapping: Optional[Dict[int, str]] = None
@functools.cached_property @property
def file_tokens(self) -> list[tokenize.TokenInfo]: def file_tokens(self) -> List[_Token]:
"""Return the complete set of tokens for a file.""" """Return the complete set of tokens for a file."""
line_iter = iter(self.lines) if self._file_tokens is None:
return list(tokenize.generate_tokens(lambda: next(line_iter))) line_iter = iter(self.lines)
self._file_tokens = list(
tokenize.generate_tokens(lambda: next(line_iter))
)
def fstring_start(self, lineno: int) -> None: # pragma: >=3.12 cover return self._file_tokens
"""Signal the beginning of an fstring."""
self._fstring_start = lineno
def tstring_start(self, lineno: int) -> None: # pragma: >=3.14 cover
"""Signal the beginning of an tstring."""
self._tstring_start = lineno
def multiline_string(self, token: tokenize.TokenInfo) -> Generator[str]:
"""Iterate through the lines of a multiline string."""
if token.type == FSTRING_END: # pragma: >=3.12 cover
start = self._fstring_start
elif token.type == TSTRING_END: # pragma: >=3.14 cover
start = self._tstring_start
else:
start = token.start[0]
@contextlib.contextmanager
def inside_multiline(
self, line_number: int
) -> Generator[None, None, None]:
"""Context-manager to toggle the multiline attribute."""
self.line_number = line_number
self.multiline = True self.multiline = True
self.line_number = start yield
# intentionally don't include the last line, that line will be
# terminated later by a future end-of-line
for _ in range(start, token.end[0]):
yield self.lines[self.line_number - 1]
self.line_number += 1
self.multiline = False self.multiline = False
def reset_blank_before(self) -> None: def reset_blank_before(self) -> None:
@ -169,11 +163,11 @@ class FileProcessor:
if self.blank_before < self.blank_lines: if self.blank_before < self.blank_lines:
self.blank_before = self.blank_lines self.blank_before = self.blank_lines
def update_checker_state_for(self, plugin: LoadedPlugin) -> None: def update_checker_state_for(self, plugin: Dict[str, Any]) -> None:
"""Update the checker_state attribute for the plugin.""" """Update the checker_state attribute for the plugin."""
if "checker_state" in plugin.parameters: if "checker_state" in plugin["parameters"]:
self.checker_state = self._checker_states.setdefault( self.checker_state = self._checker_states.setdefault(
plugin.entry_name, {}, plugin["name"], {}
) )
def next_logical_line(self) -> None: def next_logical_line(self) -> None:
@ -189,7 +183,7 @@ class FileProcessor:
self.blank_lines = 0 self.blank_lines = 0
self.tokens = [] self.tokens = []
def build_logical_line_tokens(self) -> _Logical: # noqa: C901 def build_logical_line_tokens(self) -> _Logical:
"""Build the mapping, comments, and logical line lists.""" """Build the mapping, comments, and logical line lists."""
logical = [] logical = []
comments = [] comments = []
@ -206,18 +200,7 @@ class FileProcessor:
continue continue
if token_type == tokenize.STRING: if token_type == tokenize.STRING:
text = mutate_string(text) text = mutate_string(text)
elif token_type in { if previous_row:
FSTRING_MIDDLE,
TSTRING_MIDDLE,
}: # pragma: >=3.12 cover # noqa: E501
# A curly brace in an FSTRING_MIDDLE token must be an escaped
# curly brace. Both 'text' and 'end' will account for the
# escaped version of the token (i.e. a single brace) rather
# than the raw double brace version, so we must counteract this
brace_offset = text.count("{") + text.count("}")
text = "x" * (len(text) + brace_offset)
end = (end[0], end[1] + brace_offset)
if previous_row is not None and previous_column is not None:
(start_row, start_column) = start (start_row, start_column) = start
if previous_row != start_row: if previous_row != start_row:
row_index = previous_row - 1 row_index = previous_row - 1
@ -239,7 +222,7 @@ class FileProcessor:
"""Build an abstract syntax tree from the list of lines.""" """Build an abstract syntax tree from the list of lines."""
return ast.parse("".join(self.lines)) return ast.parse("".join(self.lines))
def build_logical_line(self) -> tuple[str, str, _LogicalMapping]: def build_logical_line(self) -> Tuple[str, str, _LogicalMapping]:
"""Build a logical line from the current tokens list.""" """Build a logical line from the current tokens list."""
comments, logical, mapping_list = self.build_logical_line_tokens() comments, logical, mapping_list = self.build_logical_line_tokens()
joined_comments = "".join(comments) joined_comments = "".join(comments)
@ -247,20 +230,31 @@ class FileProcessor:
self.statistics["logical lines"] += 1 self.statistics["logical lines"] += 1
return joined_comments, self.logical_line, mapping_list return joined_comments, self.logical_line, mapping_list
def split_line(self, token: _Token) -> Generator[str, None, None]:
"""Split a physical line's line based on new-lines.
This also auto-increments the line number for the caller.
"""
for line in token[1].split("\n")[:-1]:
yield line
self.line_number += 1
def keyword_arguments_for( def keyword_arguments_for(
self, self,
parameters: dict[str, bool], parameters: Dict[str, bool],
arguments: dict[str, Any], arguments: Optional[Dict[str, Any]] = None,
) -> dict[str, Any]: ) -> Dict[str, Any]:
"""Generate the keyword arguments for a list of parameters.""" """Generate the keyword arguments for a list of parameters."""
ret = {} if arguments is None:
arguments = {}
for param, required in parameters.items(): for param, required in parameters.items():
if param in arguments: if param in arguments:
continue continue
try: try:
ret[param] = getattr(self, param) arguments[param] = getattr(self, param)
except AttributeError: except AttributeError as exc:
if required: if required:
LOG.exception(exc)
raise raise
else: else:
LOG.warning( LOG.warning(
@ -268,9 +262,9 @@ class FileProcessor:
"but this is not an available parameter.", "but this is not an available parameter.",
param, param,
) )
return ret return arguments
def generate_tokens(self) -> Generator[tokenize.TokenInfo]: def generate_tokens(self) -> Generator[_Token, None, None]:
"""Tokenize the file and yield the tokens.""" """Tokenize the file and yield the tokens."""
for token in tokenize.generate_tokens(self.next_line): for token in tokenize.generate_tokens(self.next_line):
if token[2][0] > self.total_lines: if token[2][0] > self.total_lines:
@ -278,42 +272,46 @@ class FileProcessor:
self.tokens.append(token) self.tokens.append(token)
yield token yield token
def _noqa_line_range(self, min_line: int, max_line: int) -> dict[int, str]: def _noqa_line_range(self, min_line: int, max_line: int) -> Dict[int, str]:
line_range = range(min_line, max_line + 1) line_range = range(min_line, max_line + 1)
joined = "".join(self.lines[min_line - 1: max_line]) joined = "".join(self.lines[min_line - 1 : max_line])
return dict.fromkeys(line_range, joined) return dict.fromkeys(line_range, joined)
@functools.cached_property def noqa_line_for(self, line_number: int) -> Optional[str]:
def _noqa_line_mapping(self) -> dict[int, str]: """Retrieve the line which will be used to determine noqa."""
"""Map from line number to the line we'll search for `noqa` in.""" if self._noqa_line_mapping is None:
try: try:
file_tokens = self.file_tokens file_tokens = self.file_tokens
except (tokenize.TokenError, SyntaxError): except (tokenize.TokenError, SyntaxError):
# if we failed to parse the file tokens, we'll always fail in # if we failed to parse the file tokens, we'll always fail in
# the future, so set this so the code does not try again # the future, so set this so the code does not try again
return {} self._noqa_line_mapping = {}
else: else:
ret = {} ret = {}
min_line = len(self.lines) + 2 min_line = len(self.lines) + 2
max_line = -1 max_line = -1
for tp, _, (s_line, _), (e_line, _), _ in file_tokens: for tp, _, (s_line, _), (e_line, _), _ in file_tokens:
if tp == tokenize.ENDMARKER or tp == tokenize.DEDENT: if tp == tokenize.ENDMARKER:
continue break
min_line = min(min_line, s_line) min_line = min(min_line, s_line)
max_line = max(max_line, e_line) max_line = max(max_line, e_line)
if tp in (tokenize.NL, tokenize.NEWLINE): if tp in (tokenize.NL, tokenize.NEWLINE):
ret.update(self._noqa_line_range(min_line, max_line))
min_line = len(self.lines) + 2
max_line = -1
# in newer versions of python, a `NEWLINE` token is inserted
# at the end of the file even if it doesn't have one.
# on old pythons, they will not have hit a `NEWLINE`
if max_line != -1:
ret.update(self._noqa_line_range(min_line, max_line)) ret.update(self._noqa_line_range(min_line, max_line))
min_line = len(self.lines) + 2 self._noqa_line_mapping = ret
max_line = -1
return ret
def noqa_line_for(self, line_number: int) -> str | None:
"""Retrieve the line which will be used to determine noqa."""
# NOTE(sigmavirus24): Some plugins choose to report errors for empty # NOTE(sigmavirus24): Some plugins choose to report errors for empty
# files on Line 1. In those cases, we shouldn't bother trying to # files on Line 1. In those cases, we shouldn't bother trying to
# retrieve a physical line (since none exist). # retrieve a physical line (since none exist).
@ -329,16 +327,16 @@ class FileProcessor:
self.indent_char = line[0] self.indent_char = line[0]
return line return line
def read_lines(self) -> list[str]: def read_lines(self) -> List[str]:
"""Read the lines for this file checker.""" """Read the lines for this file checker."""
if self.filename == "-": if self.filename is None or self.filename == "-":
self.filename = self.options.stdin_display_name or "stdin" self.filename = self.options.stdin_display_name or "stdin"
lines = self.read_lines_from_stdin() lines = self.read_lines_from_stdin()
else: else:
lines = self.read_lines_from_filename() lines = self.read_lines_from_filename()
return lines return lines
def read_lines_from_filename(self) -> list[str]: def read_lines_from_filename(self) -> List[str]:
"""Read the lines for a file.""" """Read the lines for a file."""
try: try:
with tokenize.open(self.filename) as fd: with tokenize.open(self.filename) as fd:
@ -349,7 +347,7 @@ class FileProcessor:
with open(self.filename, encoding="latin-1") as fd: with open(self.filename, encoding="latin-1") as fd:
return fd.readlines() return fd.readlines()
def read_lines_from_stdin(self) -> list[str]: def read_lines_from_stdin(self) -> List[str]:
"""Read the lines from standard in.""" """Read the lines from standard in."""
return utils.stdin_get_lines() return utils.stdin_get_lines()
@ -359,6 +357,8 @@ class FileProcessor:
:returns: :returns:
True if a line matches :attr:`defaults.NOQA_FILE`, True if a line matches :attr:`defaults.NOQA_FILE`,
otherwise False otherwise False
:rtype:
bool
""" """
if not self.options.disable_noqa and any( if not self.options.disable_noqa and any(
defaults.NOQA_FILE.match(line) for line in self.lines defaults.NOQA_FILE.match(line) for line in self.lines
@ -367,7 +367,7 @@ class FileProcessor:
elif any(defaults.NOQA_FILE.search(line) for line in self.lines): elif any(defaults.NOQA_FILE.search(line) for line in self.lines):
LOG.warning( LOG.warning(
"Detected `flake8: noqa` on line with code. To ignore an " "Detected `flake8: noqa` on line with code. To ignore an "
"error on a line use `noqa` instead.", "error on a line use `noqa` instead."
) )
return False return False
else: else:
@ -379,26 +379,28 @@ class FileProcessor:
# If we have nothing to analyze quit early # If we have nothing to analyze quit early
return return
first_byte = ord(self.lines[0][0])
if first_byte not in (0xEF, 0xFEFF):
return
# If the first byte of the file is a UTF-8 BOM, strip it # If the first byte of the file is a UTF-8 BOM, strip it
if self.lines[0][:1] == "\uFEFF": if first_byte == 0xFEFF:
self.lines[0] = self.lines[0][1:] self.lines[0] = self.lines[0][1:]
elif self.lines[0][:3] == "\xEF\xBB\xBF": elif self.lines[0][:3] == "\xEF\xBB\xBF":
self.lines[0] = self.lines[0][3:] self.lines[0] = self.lines[0][3:]
def is_eol_token(token: tokenize.TokenInfo) -> bool: def is_eol_token(token: _Token) -> bool:
"""Check if the token is an end-of-line token.""" """Check if the token is an end-of-line token."""
return token[0] in NEWLINE or token[4][token[3][1]:].lstrip() == "\\\n" return token[0] in NEWLINE or token[4][token[3][1] :].lstrip() == "\\\n"
def is_multiline_string(token: tokenize.TokenInfo) -> bool: def is_multiline_string(token: _Token) -> bool:
"""Check if this is a multiline string.""" """Check if this is a multiline string."""
return token.type in {FSTRING_END, TSTRING_END} or ( return token[0] == tokenize.STRING and "\n" in token[1]
token.type == tokenize.STRING and "\n" in token.string
)
def token_is_newline(token: tokenize.TokenInfo) -> bool: def token_is_newline(token: _Token) -> bool:
"""Check if the token type is a newline token type.""" """Check if the token type is a newline token type."""
return token[0] in NEWLINE return token[0] in NEWLINE
@ -412,6 +414,19 @@ def count_parentheses(current_parentheses_count: int, token_text: str) -> int:
return current_parentheses_count return current_parentheses_count
def log_token(log: logging.Logger, token: _Token) -> None:
"""Log a token to a provided logging object."""
if token[2][0] == token[3][0]:
pos = "[{}:{}]".format(token[2][1] or "", token[3][1])
else:
pos = f"l.{token[3][0]}"
log.log(
flake8._EXTRA_VERBOSE,
"l.%s\t%s\t%s\t%r"
% (token[2][0], pos, tokenize.tok_name[token[0]], token[1]),
)
def expand_indent(line: str) -> int: def expand_indent(line: str) -> int:
r"""Return the amount of indentation. r"""Return the amount of indentation.

View file

@ -1,10 +1,13 @@
"""Statistic collection logic for Flake8.""" """Statistic collection logic for Flake8."""
from __future__ import annotations import collections
from typing import Dict
from typing import Generator
from typing import List
from typing import Optional
from typing import TYPE_CHECKING
from collections.abc import Generator if TYPE_CHECKING:
from typing import NamedTuple from flake8.style_guide import Violation
from flake8.violation import Violation
class Statistics: class Statistics:
@ -12,22 +15,26 @@ class Statistics:
def __init__(self) -> None: def __init__(self) -> None:
"""Initialize the underlying dictionary for our statistics.""" """Initialize the underlying dictionary for our statistics."""
self._store: dict[Key, Statistic] = {} self._store: Dict[Key, "Statistic"] = {}
def error_codes(self) -> list[str]: def error_codes(self) -> List[str]:
"""Return all unique error codes stored. """Return all unique error codes stored.
:returns: :returns:
Sorted list of error codes. Sorted list of error codes.
:rtype:
list(str)
""" """
return sorted({key.code for key in self._store}) return sorted({key.code for key in self._store})
def record(self, error: Violation) -> None: def record(self, error: "Violation") -> None:
"""Add the fact that the error was seen in the file. """Add the fact that the error was seen in the file.
:param error: :param error:
The Violation instance containing the information about the The Violation instance containing the information about the
violation. violation.
:type error:
flake8.style_guide.Violation
""" """
key = Key.create_from(error) key = Key.create_from(error)
if key not in self._store: if key not in self._store:
@ -35,8 +42,8 @@ class Statistics:
self._store[key].increment() self._store[key].increment()
def statistics_for( def statistics_for(
self, prefix: str, filename: str | None = None, self, prefix: str, filename: Optional[str] = None
) -> Generator[Statistic]: ) -> Generator["Statistic", None, None]:
"""Generate statistics for the prefix and filename. """Generate statistics for the prefix and filename.
If you have a :class:`Statistics` object that has recorded errors, If you have a :class:`Statistics` object that has recorded errors,
@ -52,9 +59,9 @@ class Statistics:
>>> stats.statistics_for('W') >>> stats.statistics_for('W')
<generator ...> <generator ...>
:param prefix: :param str prefix:
The error class or specific error code to find statistics for. The error class or specific error code to find statistics for.
:param filename: :param str filename:
(Optional) The filename to further filter results by. (Optional) The filename to further filter results by.
:returns: :returns:
Generator of instances of :class:`Statistic` Generator of instances of :class:`Statistic`
@ -66,7 +73,7 @@ class Statistics:
yield self._store[error_code] yield self._store[error_code]
class Key(NamedTuple): class Key(collections.namedtuple("Key", ["filename", "code"])):
"""Simple key structure for the Statistics dictionary. """Simple key structure for the Statistics dictionary.
To make things clearer, easier to read, and more understandable, we use a To make things clearer, easier to read, and more understandable, we use a
@ -74,25 +81,26 @@ class Key(NamedTuple):
Statistics object. Statistics object.
""" """
filename: str __slots__ = ()
code: str
@classmethod @classmethod
def create_from(cls, error: Violation) -> Key: def create_from(cls, error: "Violation") -> "Key":
"""Create a Key from :class:`flake8.violation.Violation`.""" """Create a Key from :class:`flake8.style_guide.Violation`."""
return cls(filename=error.filename, code=error.code) return cls(filename=error.filename, code=error.code)
def matches(self, prefix: str, filename: str | None) -> bool: def matches(self, prefix: str, filename: Optional[str]) -> bool:
"""Determine if this key matches some constraints. """Determine if this key matches some constraints.
:param prefix: :param str prefix:
The error code prefix that this key's error code should start with. The error code prefix that this key's error code should start with.
:param filename: :param str filename:
The filename that we potentially want to match on. This can be The filename that we potentially want to match on. This can be
None to only match on error prefix. None to only match on error prefix.
:returns: :returns:
True if the Key's code starts with the prefix and either filename True if the Key's code starts with the prefix and either filename
is None, or the Key's filename matches the value passed in. is None, or the Key's filename matches the value passed in.
:rtype:
bool
""" """
return self.code.startswith(prefix) and ( return self.code.startswith(prefix) and (
filename is None or self.filename == filename filename is None or self.filename == filename
@ -103,12 +111,12 @@ class Statistic:
"""Simple wrapper around the logic of each statistic. """Simple wrapper around the logic of each statistic.
Instead of maintaining a simple but potentially hard to reason about Instead of maintaining a simple but potentially hard to reason about
tuple, we create a class which has attributes and a couple tuple, we create a namedtuple which has attributes and a couple
convenience methods on it. convenience methods on it.
""" """
def __init__( def __init__(
self, error_code: str, filename: str, message: str, count: int, self, error_code: str, filename: str, message: str, count: int
) -> None: ) -> None:
"""Initialize our Statistic.""" """Initialize our Statistic."""
self.error_code = error_code self.error_code = error_code
@ -117,8 +125,8 @@ class Statistic:
self.count = count self.count = count
@classmethod @classmethod
def create_from(cls, error: Violation) -> Statistic: def create_from(cls, error: "Violation") -> "Statistic":
"""Create a Statistic from a :class:`flake8.violation.Violation`.""" """Create a Statistic from a :class:`flake8.style_guide.Violation`."""
return cls( return cls(
error_code=error.code, error_code=error.code,
filename=error.filename, filename=error.filename,

View file

@ -1,20 +1,27 @@
"""Implementation of the StyleGuide used by Flake8.""" """Implementation of the StyleGuide used by Flake8."""
from __future__ import annotations
import argparse import argparse
import collections
import contextlib import contextlib
import copy import copy
import enum import enum
import functools import functools
import itertools
import linecache
import logging import logging
from collections.abc import Generator from typing import Dict
from collections.abc import Sequence from typing import Generator
from typing import List
from typing import Match
from typing import Optional
from typing import Sequence
from typing import Set
from typing import Tuple
from typing import Union
from flake8 import defaults from flake8 import defaults
from flake8 import statistics from flake8 import statistics
from flake8 import utils from flake8 import utils
from flake8.formatting import base as base_formatter from flake8.formatting import base as base_formatter
from flake8.violation import Violation
__all__ = ("StyleGuide",) __all__ = ("StyleGuide",)
@ -42,28 +49,101 @@ class Decision(enum.Enum):
Selected = "selected error" Selected = "selected error"
def _explicitly_chosen( @functools.lru_cache(maxsize=512)
*, def find_noqa(physical_line: str) -> Optional[Match[str]]:
option: list[str] | None, return defaults.NOQA_INLINE_REGEXP.search(physical_line)
extend: list[str] | None,
) -> tuple[str, ...]:
ret = [*(option or []), *(extend or [])]
return tuple(sorted(ret, reverse=True))
def _select_ignore( class Violation(
*, collections.namedtuple(
option: list[str] | None, "Violation",
default: tuple[str, ...], [
extended_default: list[str], "code",
extend: list[str] | None, "filename",
) -> tuple[str, ...]: "line_number",
# option was explicitly set, ignore the default and extended default "column_number",
if option is not None: "text",
ret = [*option, *(extend or [])] "physical_line",
else: ],
ret = [*default, *extended_default, *(extend or [])] )
return tuple(sorted(ret, reverse=True)) ):
"""Class representing a violation reported by Flake8."""
def is_inline_ignored(self, disable_noqa: bool) -> bool:
"""Determine if a comment has been added to ignore this line.
:param bool disable_noqa:
Whether or not users have provided ``--disable-noqa``.
:returns:
True if error is ignored in-line, False otherwise.
:rtype:
bool
"""
physical_line = self.physical_line
# TODO(sigmavirus24): Determine how to handle stdin with linecache
if disable_noqa:
return False
if physical_line is None:
physical_line = linecache.getline(self.filename, self.line_number)
noqa_match = find_noqa(physical_line)
if noqa_match is None:
LOG.debug("%r is not inline ignored", self)
return False
codes_str = noqa_match.groupdict()["codes"]
if codes_str is None:
LOG.debug("%r is ignored by a blanket ``# noqa``", self)
return True
codes = set(utils.parse_comma_separated_list(codes_str))
if self.code in codes or self.code.startswith(tuple(codes)):
LOG.debug(
"%r is ignored specifically inline with ``# noqa: %s``",
self,
codes_str,
)
return True
LOG.debug(
"%r is not ignored inline with ``# noqa: %s``", self, codes_str
)
return False
def is_in(self, diff: Dict[str, Set[int]]) -> bool:
"""Determine if the violation is included in a diff's line ranges.
This function relies on the parsed data added via
:meth:`~StyleGuide.add_diff_ranges`. If that has not been called and
we are not evaluating files in a diff, then this will always return
True. If there are diff ranges, then this will return True if the
line number in the error falls inside one of the ranges for the file
(and assuming the file is part of the diff data). If there are diff
ranges, this will return False if the file is not part of the diff
data or the line number of the error is not in any of the ranges of
the diff.
:returns:
True if there is no diff or if the error is in the diff's line
number ranges. False if the error's line number falls outside
the diff's line number ranges.
:rtype:
bool
"""
if not diff:
return True
# NOTE(sigmavirus24): The parsed diff will be a defaultdict with
# a set as the default value (if we have received it from
# flake8.utils.parse_unified_diff). In that case ranges below
# could be an empty set (which is False-y) or if someone else
# is using this API, it could be None. If we could guarantee one
# or the other, we would check for it more explicitly.
line_numbers = diff.get(self.filename)
if not line_numbers:
return False
return self.line_number in line_numbers
class DecisionEngine: class DecisionEngine:
@ -75,34 +155,46 @@ class DecisionEngine:
def __init__(self, options: argparse.Namespace) -> None: def __init__(self, options: argparse.Namespace) -> None:
"""Initialize the engine.""" """Initialize the engine."""
self.cache: dict[str, Decision] = {} self.cache: Dict[str, Decision] = {}
self.selected = tuple(options.select)
self.selected_explicitly = _explicitly_chosen( self.extended_selected = tuple(
option=options.select, sorted(options.extended_default_select, reverse=True)
extend=options.extend_select,
) )
self.ignored_explicitly = _explicitly_chosen( self.enabled_extensions = tuple(options.enable_extensions)
option=options.ignore, self.all_selected = tuple(
extend=options.extend_ignore, sorted(
itertools.chain(
self.selected,
options.extend_select,
self.enabled_extensions,
),
reverse=True,
)
)
self.ignored = tuple(
sorted(
itertools.chain(options.ignore, options.extend_ignore),
reverse=True,
)
)
self.using_default_ignore = set(self.ignored) == set(
defaults.IGNORE
).union(options.extended_default_ignore)
self.using_default_select = set(self.selected) == set(defaults.SELECT)
def _in_all_selected(self, code: str) -> bool:
return bool(self.all_selected) and code.startswith(self.all_selected)
def _in_extended_selected(self, code: str) -> bool:
return bool(self.extended_selected) and code.startswith(
self.extended_selected
) )
self.selected = _select_ignore( def was_selected(self, code: str) -> Union[Selected, Ignored]:
option=options.select,
default=(),
extended_default=options.extended_default_select,
extend=options.extend_select,
)
self.ignored = _select_ignore(
option=options.ignore,
default=defaults.IGNORE,
extended_default=options.extended_default_ignore,
extend=options.extend_ignore,
)
def was_selected(self, code: str) -> Selected | Ignored:
"""Determine if the code has been selected by the user. """Determine if the code has been selected by the user.
:param code: The code for the check that has been run. :param str code:
The code for the check that has been run.
:returns: :returns:
Selected.Implicitly if the selected list is empty, Selected.Implicitly if the selected list is empty,
Selected.Explicitly if the selected list is not empty and a match Selected.Explicitly if the selected list is not empty and a match
@ -110,17 +202,21 @@ class DecisionEngine:
Ignored.Implicitly if the selected list is not empty but no match Ignored.Implicitly if the selected list is not empty but no match
was found. was found.
""" """
if code.startswith(self.selected_explicitly): if self._in_all_selected(code):
return Selected.Explicitly return Selected.Explicitly
elif code.startswith(self.selected):
return Selected.Implicitly
else:
return Ignored.Implicitly
def was_ignored(self, code: str) -> Selected | Ignored: if not self.all_selected and self._in_extended_selected(code):
# If it was not explicitly selected, it may have been implicitly
# selected because the check comes from a plugin that is enabled by
# default
return Selected.Implicitly
return Ignored.Implicitly
def was_ignored(self, code: str) -> Union[Selected, Ignored]:
"""Determine if the code has been ignored by the user. """Determine if the code has been ignored by the user.
:param code: :param str code:
The code for the check that has been run. The code for the check that has been run.
:returns: :returns:
Selected.Implicitly if the ignored list is empty, Selected.Implicitly if the ignored list is empty,
@ -129,54 +225,84 @@ class DecisionEngine:
Selected.Implicitly if the ignored list is not empty but no match Selected.Implicitly if the ignored list is not empty but no match
was found. was found.
""" """
if code.startswith(self.ignored_explicitly): if self.ignored and code.startswith(self.ignored):
return Ignored.Explicitly return Ignored.Explicitly
elif code.startswith(self.ignored):
return Ignored.Implicitly return Selected.Implicitly
else:
return Selected.Implicitly def more_specific_decision_for(self, code: str) -> Decision:
select = find_first_match(code, self.all_selected)
extra_select = find_first_match(code, self.extended_selected)
ignore = find_first_match(code, self.ignored)
if select and ignore:
# If the violation code appears in both the select and ignore
# lists (in some fashion) then if we're using the default ignore
# list and a custom select list we should select the code. An
# example usage looks like this:
# A user has a code that would generate an E126 violation which
# is in our default ignore list and they specify select=E.
# We should be reporting that violation. This logic changes,
# however, if they specify select and ignore such that both match.
# In that case we fall through to our find_more_specific call.
# If, however, the user hasn't specified a custom select, and
# we're using the defaults for both select and ignore then the
# more specific rule must win. In most cases, that will be to
# ignore the violation since our default select list is very
# high-level and our ignore list is highly specific.
if self.using_default_ignore and not self.using_default_select:
return Decision.Selected
return find_more_specific(select, ignore)
if extra_select and ignore:
# At this point, select is false-y. Now we need to check if the
# code is in our extended select list and our ignore list. This is
# a *rare* case as we see little usage of the extended select list
# that plugins can use, so I suspect this section may change to
# look a little like the block above in which we check if we're
# using our default ignore list.
return find_more_specific(extra_select, ignore)
if select or (extra_select and self.using_default_select):
# Here, ignore was false-y and the user has either selected
# explicitly the violation or the violation is covered by
# something in the extended select list and we're using the
# default select list. In either case, we want the violation to be
# selected.
return Decision.Selected
if select is None and (
extra_select is None or not self.using_default_ignore
):
return Decision.Ignored
if (select is None and not self.using_default_select) and (
ignore is None and self.using_default_ignore
):
return Decision.Ignored
return Decision.Selected
def make_decision(self, code: str) -> Decision: def make_decision(self, code: str) -> Decision:
"""Decide if code should be ignored or selected.""" """Decide if code should be ignored or selected."""
LOG.debug('Deciding if "%s" should be reported', code)
selected = self.was_selected(code) selected = self.was_selected(code)
ignored = self.was_ignored(code) ignored = self.was_ignored(code)
LOG.debug( LOG.debug(
"The user configured %r to be %r, %r", 'The user configured "%s" to be "%s", "%s"',
code, code,
selected, selected,
ignored, ignored,
) )
if isinstance(selected, Selected) and isinstance(ignored, Selected): if (
return Decision.Selected selected is Selected.Explicitly or selected is Selected.Implicitly
elif isinstance(selected, Ignored) and isinstance(ignored, Ignored): ) and ignored is Selected.Implicitly:
return Decision.Ignored decision = Decision.Selected
elif (
selected is Selected.Explicitly
and ignored is not Ignored.Explicitly
):
return Decision.Selected
elif (
selected is not Selected.Explicitly
and ignored is Ignored.Explicitly
):
return Decision.Ignored
elif selected is Ignored.Implicitly and ignored is Selected.Implicitly:
return Decision.Ignored
elif ( elif (
selected is Selected.Explicitly and ignored is Ignored.Explicitly selected is Selected.Explicitly and ignored is Ignored.Explicitly
) or ( ) or (
selected is Selected.Implicitly and ignored is Ignored.Implicitly selected is Ignored.Implicitly and ignored is Selected.Implicitly
): ):
# we only get here if it was in both lists: longest prefix wins decision = self.more_specific_decision_for(code)
select = next(s for s in self.selected if code.startswith(s)) elif selected is Ignored.Implicitly or ignored is Ignored.Explicitly:
ignore = next(s for s in self.ignored if code.startswith(s)) decision = Decision.Ignored # pylint: disable=R0204
if len(select) > len(ignore): return decision
return Decision.Selected
else:
return Decision.Ignored
else:
raise AssertionError(f"unreachable {code} {selected} {ignored}")
def decision_for(self, code: str) -> Decision: def decision_for(self, code: str) -> Decision:
"""Return the decision for a specific code. """Return the decision for a specific code.
@ -189,7 +315,8 @@ class DecisionEngine:
This method does not look at whether the specific line is being This method does not look at whether the specific line is being
ignored in the file itself. ignored in the file itself.
:param code: The code for the check that has been run. :param str code:
The code for the check that has been run.
""" """
decision = self.cache.get(code) decision = self.cache.get(code)
if decision is None: if decision is None:
@ -206,7 +333,7 @@ class StyleGuideManager:
self, self,
options: argparse.Namespace, options: argparse.Namespace,
formatter: base_formatter.BaseFormatter, formatter: base_formatter.BaseFormatter,
decider: DecisionEngine | None = None, decider: Optional[DecisionEngine] = None,
) -> None: ) -> None:
"""Initialize our StyleGuide. """Initialize our StyleGuide.
@ -216,42 +343,52 @@ class StyleGuideManager:
self.formatter = formatter self.formatter = formatter
self.stats = statistics.Statistics() self.stats = statistics.Statistics()
self.decider = decider or DecisionEngine(options) self.decider = decider or DecisionEngine(options)
self.style_guides: list[StyleGuide] = [] self.style_guides: List[StyleGuide] = []
self.default_style_guide = StyleGuide( self.default_style_guide = StyleGuide(
options, formatter, self.stats, decider=decider, options, formatter, self.stats, decider=decider
)
self.style_guides = list(
itertools.chain(
[self.default_style_guide],
self.populate_style_guides_with(options),
)
) )
self.style_guides = [
self.default_style_guide,
*self.populate_style_guides_with(options),
]
self.style_guide_for = functools.cache(self._style_guide_for)
def populate_style_guides_with( def populate_style_guides_with(
self, options: argparse.Namespace, self, options: argparse.Namespace
) -> Generator[StyleGuide]: ) -> Generator["StyleGuide", None, None]:
"""Generate style guides from the per-file-ignores option. """Generate style guides from the per-file-ignores option.
:param options: :param options:
The original options parsed from the CLI and config file. The original options parsed from the CLI and config file.
:type options:
:class:`~argparse.Namespace`
:returns: :returns:
A copy of the default style guide with overridden values. A copy of the default style guide with overridden values.
:rtype:
:class:`~flake8.style_guide.StyleGuide`
""" """
per_file = utils.parse_files_to_codes_mapping(options.per_file_ignores) per_file = utils.parse_files_to_codes_mapping(options.per_file_ignores)
for filename, violations in per_file: for filename, violations in per_file:
yield self.default_style_guide.copy( yield self.default_style_guide.copy(
filename=filename, extend_ignore_with=violations, filename=filename, extend_ignore_with=violations
) )
def _style_guide_for(self, filename: str) -> StyleGuide: @functools.lru_cache(maxsize=None)
def style_guide_for(self, filename: str) -> "StyleGuide":
"""Find the StyleGuide for the filename in particular.""" """Find the StyleGuide for the filename in particular."""
return max( guides = sorted(
(g for g in self.style_guides if g.applies_to(filename)), (g for g in self.style_guides if g.applies_to(filename)),
key=lambda g: len(g.filename or ""), key=lambda g: len(g.filename or ""),
) )
if len(guides) > 1:
return guides[-1]
return guides[0]
@contextlib.contextmanager @contextlib.contextmanager
def processing_file(self, filename: str) -> Generator[StyleGuide]: def processing_file(
self, filename: str
) -> Generator["StyleGuide", None, None]:
"""Record the fact that we're processing the file's results.""" """Record the fact that we're processing the file's results."""
guide = self.style_guide_for(filename) guide = self.style_guide_for(filename)
with guide.processing_file(filename): with guide.processing_file(filename):
@ -262,35 +399,49 @@ class StyleGuideManager:
code: str, code: str,
filename: str, filename: str,
line_number: int, line_number: int,
column_number: int, column_number: Optional[int],
text: str, text: str,
physical_line: str | None = None, physical_line: Optional[str] = None,
) -> int: ) -> int:
"""Handle an error reported by a check. """Handle an error reported by a check.
:param code: :param str code:
The error code found, e.g., E123. The error code found, e.g., E123.
:param filename: :param str filename:
The file in which the error was found. The file in which the error was found.
:param line_number: :param int line_number:
The line number (where counting starts at 1) at which the error The line number (where counting starts at 1) at which the error
occurs. occurs.
:param column_number: :param int column_number:
The column number (where counting starts at 1) at which the error The column number (where counting starts at 1) at which the error
occurs. occurs.
:param text: :param str text:
The text of the error message. The text of the error message.
:param physical_line: :param str physical_line:
The actual physical line causing the error. The actual physical line causing the error.
:returns: :returns:
1 if the error was reported. 0 if it was ignored. This is to allow 1 if the error was reported. 0 if it was ignored. This is to allow
for counting of the number of errors found that were not ignored. for counting of the number of errors found that were not ignored.
:rtype:
int
""" """
guide = self.style_guide_for(filename) guide = self.style_guide_for(filename)
return guide.handle_error( return guide.handle_error(
code, filename, line_number, column_number, text, physical_line, code, filename, line_number, column_number, text, physical_line
) )
def add_diff_ranges(self, diffinfo: Dict[str, Set[int]]) -> None:
"""Update the StyleGuides to filter out information not in the diff.
This provides information to the underlying StyleGuides so that only
the errors in the line number ranges are reported.
:param dict diffinfo:
Dictionary mapping filenames to sets of line number ranges.
"""
for guide in self.style_guides:
guide.add_diff_ranges(diffinfo)
class StyleGuide: class StyleGuide:
"""Manage a Flake8 user's style guide.""" """Manage a Flake8 user's style guide."""
@ -300,8 +451,8 @@ class StyleGuide:
options: argparse.Namespace, options: argparse.Namespace,
formatter: base_formatter.BaseFormatter, formatter: base_formatter.BaseFormatter,
stats: statistics.Statistics, stats: statistics.Statistics,
filename: str | None = None, filename: Optional[str] = None,
decider: DecisionEngine | None = None, decider: Optional[DecisionEngine] = None,
): ):
"""Initialize our StyleGuide. """Initialize our StyleGuide.
@ -314,6 +465,7 @@ class StyleGuide:
self.filename = filename self.filename = filename
if self.filename: if self.filename:
self.filename = utils.normalize_path(self.filename) self.filename = utils.normalize_path(self.filename)
self._parsed_diff: Dict[str, Set[int]] = {}
def __repr__(self) -> str: def __repr__(self) -> str:
"""Make it easier to debug which StyleGuide we're using.""" """Make it easier to debug which StyleGuide we're using."""
@ -321,20 +473,21 @@ class StyleGuide:
def copy( def copy(
self, self,
filename: str | None = None, filename: Optional[str] = None,
extend_ignore_with: Sequence[str] | None = None, extend_ignore_with: Optional[Sequence[str]] = None,
) -> StyleGuide: ) -> "StyleGuide":
"""Create a copy of this style guide with different values.""" """Create a copy of this style guide with different values."""
filename = filename or self.filename filename = filename or self.filename
options = copy.deepcopy(self.options) options = copy.deepcopy(self.options)
options.extend_ignore = options.extend_ignore or [] options.ignore.extend(extend_ignore_with or [])
options.extend_ignore.extend(extend_ignore_with or [])
return StyleGuide( return StyleGuide(
options, self.formatter, self.stats, filename=filename, options, self.formatter, self.stats, filename=filename
) )
@contextlib.contextmanager @contextlib.contextmanager
def processing_file(self, filename: str) -> Generator[StyleGuide]: def processing_file(
self, filename: str
) -> Generator["StyleGuide", None, None]:
"""Record the fact that we're processing the file's results.""" """Record the fact that we're processing the file's results."""
self.formatter.beginning(filename) self.formatter.beginning(filename)
yield self yield self
@ -343,11 +496,13 @@ class StyleGuide:
def applies_to(self, filename: str) -> bool: def applies_to(self, filename: str) -> bool:
"""Check if this StyleGuide applies to the file. """Check if this StyleGuide applies to the file.
:param filename: :param str filename:
The name of the file with violations that we're potentially The name of the file with violations that we're potentially
applying this StyleGuide to. applying this StyleGuide to.
:returns: :returns:
True if this applies, False otherwise True if this applies, False otherwise
:rtype:
bool
""" """
if self.filename is None: if self.filename is None:
return True return True
@ -367,7 +522,7 @@ class StyleGuide:
This method does not look at whether the specific line is being This method does not look at whether the specific line is being
ignored in the file itself. ignored in the file itself.
:param code: :param str code:
The code for the check that has been run. The code for the check that has been run.
""" """
return self.decider.decision_for(code) return self.decider.decision_for(code)
@ -377,33 +532,36 @@ class StyleGuide:
code: str, code: str,
filename: str, filename: str,
line_number: int, line_number: int,
column_number: int, column_number: Optional[int],
text: str, text: str,
physical_line: str | None = None, physical_line: Optional[str] = None,
) -> int: ) -> int:
"""Handle an error reported by a check. """Handle an error reported by a check.
:param code: :param str code:
The error code found, e.g., E123. The error code found, e.g., E123.
:param filename: :param str filename:
The file in which the error was found. The file in which the error was found.
:param line_number: :param int line_number:
The line number (where counting starts at 1) at which the error The line number (where counting starts at 1) at which the error
occurs. occurs.
:param column_number: :param int column_number:
The column number (where counting starts at 1) at which the error The column number (where counting starts at 1) at which the error
occurs. occurs.
:param text: :param str text:
The text of the error message. The text of the error message.
:param physical_line: :param str physical_line:
The actual physical line causing the error. The actual physical line causing the error.
:returns: :returns:
1 if the error was reported. 0 if it was ignored. This is to allow 1 if the error was reported. 0 if it was ignored. This is to allow
for counting of the number of errors found that were not ignored. for counting of the number of errors found that were not ignored.
:rtype:
int
""" """
disable_noqa = self.options.disable_noqa disable_noqa = self.options.disable_noqa
# NOTE(sigmavirus24): Apparently we're provided with 0-indexed column # NOTE(sigmavirus24): Apparently we're provided with 0-indexed column
# numbers so we have to offset that here. # numbers so we have to offset that here. Also, if a SyntaxError is
# caught, column_number may be None.
if not column_number: if not column_number:
column_number = 0 column_number = 0
error = Violation( error = Violation(
@ -418,8 +576,38 @@ class StyleGuide:
self.should_report_error(error.code) is Decision.Selected self.should_report_error(error.code) is Decision.Selected
) )
is_not_inline_ignored = error.is_inline_ignored(disable_noqa) is False is_not_inline_ignored = error.is_inline_ignored(disable_noqa) is False
if error_is_selected and is_not_inline_ignored: is_included_in_diff = error.is_in(self._parsed_diff)
if error_is_selected and is_not_inline_ignored and is_included_in_diff:
self.formatter.handle(error) self.formatter.handle(error)
self.stats.record(error) self.stats.record(error)
return 1 return 1
return 0 return 0
def add_diff_ranges(self, diffinfo: Dict[str, Set[int]]) -> None:
"""Update the StyleGuide to filter out information not in the diff.
This provides information to the StyleGuide so that only the errors
in the line number ranges are reported.
:param dict diffinfo:
Dictionary mapping filenames to sets of line number ranges.
"""
self._parsed_diff = diffinfo
def find_more_specific(selected: str, ignored: str) -> Decision:
if selected.startswith(ignored) and selected != ignored:
return Decision.Selected
return Decision.Ignored
def find_first_match(
error_code: str, code_list: Tuple[str, ...]
) -> Optional[str]:
startswith = error_code.startswith
for code in code_list:
if startswith(code):
break
else:
return None
return code

View file

@ -1,8 +1,8 @@
"""Utility methods for flake8.""" """Utility methods for flake8."""
from __future__ import annotations import collections
import fnmatch as _fnmatch import fnmatch as _fnmatch
import functools import functools
import inspect
import io import io
import logging import logging
import os import os
@ -11,20 +11,31 @@ import re
import sys import sys
import textwrap import textwrap
import tokenize import tokenize
from collections.abc import Sequence from typing import Callable
from re import Pattern from typing import Dict
from typing import NamedTuple from typing import Generator
from typing import List
from typing import Optional
from typing import Pattern
from typing import Sequence
from typing import Set
from typing import Tuple
from typing import TYPE_CHECKING
from typing import Union
from flake8 import exceptions from flake8 import exceptions
if TYPE_CHECKING:
from flake8.plugins.manager import Plugin
DIFF_HUNK_REGEXP = re.compile(r"^@@ -\d+(?:,\d+)? \+(\d+)(?:,(\d+))? @@.*$")
COMMA_SEPARATED_LIST_RE = re.compile(r"[,\s]") COMMA_SEPARATED_LIST_RE = re.compile(r"[,\s]")
LOCAL_PLUGIN_LIST_RE = re.compile(r"[,\t\n\r\f\v]") LOCAL_PLUGIN_LIST_RE = re.compile(r"[,\t\n\r\f\v]")
NORMALIZE_PACKAGE_NAME_RE = re.compile(r"[-_.]+")
def parse_comma_separated_list( def parse_comma_separated_list(
value: str, regexp: Pattern[str] = COMMA_SEPARATED_LIST_RE, value: str, regexp: Pattern[str] = COMMA_SEPARATED_LIST_RE
) -> list[str]: ) -> List[str]:
"""Parse a comma-separated list. """Parse a comma-separated list.
:param value: :param value:
@ -32,8 +43,12 @@ def parse_comma_separated_list(
:param regexp: :param regexp:
Compiled regular expression used to split the value when it is a Compiled regular expression used to split the value when it is a
string. string.
:type regexp:
_sre.SRE_Pattern
:returns: :returns:
List of values with whitespace stripped. List of values with whitespace stripped.
:rtype:
list
""" """
assert isinstance(value, str), value assert isinstance(value, str), value
@ -42,11 +57,7 @@ def parse_comma_separated_list(
return [item for item in item_gen if item] return [item for item in item_gen if item]
class _Token(NamedTuple): _Token = collections.namedtuple("_Token", ("tp", "src"))
tp: str
src: str
_CODE, _FILE, _COLON, _COMMA, _WS = "code", "file", "colon", "comma", "ws" _CODE, _FILE, _COLON, _COMMA, _WS = "code", "file", "colon", "comma", "ws"
_EOF = "eof" _EOF = "eof"
_FILE_LIST_TOKEN_TYPES = [ _FILE_LIST_TOKEN_TYPES = [
@ -58,7 +69,7 @@ _FILE_LIST_TOKEN_TYPES = [
] ]
def _tokenize_files_to_codes_mapping(value: str) -> list[_Token]: def _tokenize_files_to_codes_mapping(value: str) -> List[_Token]:
tokens = [] tokens = []
i = 0 i = 0
while i < len(value): while i < len(value):
@ -76,8 +87,8 @@ def _tokenize_files_to_codes_mapping(value: str) -> list[_Token]:
def parse_files_to_codes_mapping( # noqa: C901 def parse_files_to_codes_mapping( # noqa: C901
value_: Sequence[str] | str, value_: Union[Sequence[str], str]
) -> list[tuple[str, list[str]]]: ) -> List[Tuple[str, List[str]]]:
"""Parse a files-to-codes mapping. """Parse a files-to-codes mapping.
A files-to-codes mapping a sequence of values specified as A files-to-codes mapping a sequence of values specified as
@ -85,21 +96,22 @@ def parse_files_to_codes_mapping( # noqa: C901
either comma or whitespace tokens. either comma or whitespace tokens.
:param value: String to be parsed and normalized. :param value: String to be parsed and normalized.
:type value: str
""" """
if not isinstance(value_, str): if not isinstance(value_, str):
value = "\n".join(value_) value = "\n".join(value_)
else: else:
value = value_ value = value_
ret: list[tuple[str, list[str]]] = [] ret: List[Tuple[str, List[str]]] = []
if not value.strip(): if not value.strip():
return ret return ret
class State: class State:
seen_sep = True seen_sep = True
seen_colon = False seen_colon = False
filenames: list[str] = [] filenames: List[str] = []
codes: list[str] = [] codes: List[str] = []
def _reset() -> None: def _reset() -> None:
if State.codes: if State.codes:
@ -115,7 +127,7 @@ def parse_files_to_codes_mapping( # noqa: C901
f"Expected `per-file-ignores` to be a mapping from file exclude " f"Expected `per-file-ignores` to be a mapping from file exclude "
f"patterns to ignore codes.\n\n" f"patterns to ignore codes.\n\n"
f"Configured `per-file-ignores` setting:\n\n" f"Configured `per-file-ignores` setting:\n\n"
f"{textwrap.indent(value.strip(), ' ')}", f"{textwrap.indent(value.strip(), ' ')}"
) )
for token in _tokenize_files_to_codes_mapping(value): for token in _tokenize_files_to_codes_mapping(value):
@ -150,12 +162,14 @@ def parse_files_to_codes_mapping( # noqa: C901
def normalize_paths( def normalize_paths(
paths: Sequence[str], parent: str = os.curdir, paths: Sequence[str], parent: str = os.curdir
) -> list[str]: ) -> List[str]:
"""Normalize a list of paths relative to a parent directory. """Normalize a list of paths relative to a parent directory.
:returns: :returns:
The normalized paths. The normalized paths.
:rtype:
[str]
""" """
assert isinstance(paths, list), paths assert isinstance(paths, list), paths
return [normalize_path(p, parent) for p in paths] return [normalize_path(p, parent) for p in paths]
@ -166,17 +180,17 @@ def normalize_path(path: str, parent: str = os.curdir) -> str:
:returns: :returns:
The normalized path. The normalized path.
:rtype:
str
""" """
# NOTE(sigmavirus24): Using os.path.sep and os.path.altsep allow for # NOTE(sigmavirus24): Using os.path.sep and os.path.altsep allow for
# Windows compatibility with both Windows-style paths (c:\foo\bar) and # Windows compatibility with both Windows-style paths (c:\\foo\bar) and
# Unix style paths (/foo/bar). # Unix style paths (/foo/bar).
separator = os.path.sep separator = os.path.sep
# NOTE(sigmavirus24): os.path.altsep may be None # NOTE(sigmavirus24): os.path.altsep may be None
alternate_separator = os.path.altsep or "" alternate_separator = os.path.altsep or ""
if ( if separator in path or (
path == "." alternate_separator and alternate_separator in path
or separator in path
or (alternate_separator and alternate_separator in path)
): ):
path = os.path.abspath(os.path.join(parent, path)) path = os.path.abspath(os.path.join(parent, path))
return path.rstrip(separator + alternate_separator) return path.rstrip(separator + alternate_separator)
@ -195,40 +209,200 @@ def stdin_get_value() -> str:
return stdin_value.decode("utf-8") return stdin_value.decode("utf-8")
def stdin_get_lines() -> list[str]: def stdin_get_lines() -> List[str]:
"""Return lines of stdin split according to file splitting.""" """Return lines of stdin split according to file splitting."""
return list(io.StringIO(stdin_get_value())) return list(io.StringIO(stdin_get_value()))
def is_using_stdin(paths: list[str]) -> bool: def parse_unified_diff(diff: Optional[str] = None) -> Dict[str, Set[int]]:
"""Parse the unified diff passed on stdin.
:returns:
dictionary mapping file names to sets of line numbers
:rtype:
dict
"""
# Allow us to not have to patch out stdin_get_value
if diff is None:
diff = stdin_get_value()
number_of_rows = None
current_path = None
parsed_paths: Dict[str, Set[int]] = collections.defaultdict(set)
for line in diff.splitlines():
if number_of_rows:
if not line or line[0] != "-":
number_of_rows -= 1
# We're in the part of the diff that has lines starting with +, -,
# and ' ' to show context and the changes made. We skip these
# because the information we care about is the filename and the
# range within it.
# When number_of_rows reaches 0, we will once again start
# searching for filenames and ranges.
continue
# NOTE(sigmavirus24): Diffs that we support look roughly like:
# diff a/file.py b/file.py
# ...
# --- a/file.py
# +++ b/file.py
# Below we're looking for that last line. Every diff tool that
# gives us this output may have additional information after
# ``b/file.py`` which it will separate with a \t, e.g.,
# +++ b/file.py\t100644
# Which is an example that has the new file permissions/mode.
# In this case we only care about the file name.
if line[:3] == "+++":
current_path = line[4:].split("\t", 1)[0]
# NOTE(sigmavirus24): This check is for diff output from git.
if current_path[:2] == "b/":
current_path = current_path[2:]
# We don't need to do anything else. We have set up our local
# ``current_path`` variable. We can skip the rest of this loop.
# The next line we will see will give us the hung information
# which is in the next section of logic.
continue
hunk_match = DIFF_HUNK_REGEXP.match(line)
# NOTE(sigmavirus24): pep8/pycodestyle check for:
# line[:3] == '@@ '
# But the DIFF_HUNK_REGEXP enforces that the line start with that
# So we can more simply check for a match instead of slicing and
# comparing.
if hunk_match:
(row, number_of_rows) = (
1 if not group else int(group) for group in hunk_match.groups()
)
assert current_path is not None
parsed_paths[current_path].update(range(row, row + number_of_rows))
# We have now parsed our diff into a dictionary that looks like:
# {'file.py': set(range(10, 16), range(18, 20)), ...}
return parsed_paths
def is_windows() -> bool:
"""Determine if we're running on Windows.
:returns:
True if running on Windows, otherwise False
:rtype:
bool
"""
return os.name == "nt"
def is_using_stdin(paths: List[str]) -> bool:
"""Determine if we're going to read from stdin. """Determine if we're going to read from stdin.
:param paths: :param list paths:
The paths that we're going to check. The paths that we're going to check.
:returns: :returns:
True if stdin (-) is in the path, otherwise False True if stdin (-) is in the path, otherwise False
:rtype:
bool
""" """
return "-" in paths return "-" in paths
def _default_predicate(*args: str) -> bool:
return False
def filenames_from(
arg: str, predicate: Optional[Callable[[str], bool]] = None
) -> Generator[str, None, None]:
"""Generate filenames from an argument.
:param str arg:
Parameter from the command-line.
:param callable predicate:
Predicate to use to filter out filenames. If the predicate
returns ``True`` we will exclude the filename, otherwise we
will yield it. By default, we include every filename
generated.
:returns:
Generator of paths
"""
if predicate is None:
predicate = _default_predicate
if predicate(arg):
return
if os.path.isdir(arg):
for root, sub_directories, files in os.walk(arg):
if predicate(root):
sub_directories[:] = []
continue
# NOTE(sigmavirus24): os.walk() will skip a directory if you
# remove it from the list of sub-directories.
for directory in sub_directories:
joined = os.path.join(root, directory)
if predicate(joined):
sub_directories.remove(directory)
for filename in files:
joined = os.path.join(root, filename)
if not predicate(joined):
yield joined
else:
yield arg
def fnmatch(filename: str, patterns: Sequence[str]) -> bool: def fnmatch(filename: str, patterns: Sequence[str]) -> bool:
"""Wrap :func:`fnmatch.fnmatch` to add some functionality. """Wrap :func:`fnmatch.fnmatch` to add some functionality.
:param filename: :param str filename:
Name of the file we're trying to match. Name of the file we're trying to match.
:param patterns: :param list patterns:
Patterns we're using to try to match the filename. Patterns we're using to try to match the filename.
:param default: :param bool default:
The default value if patterns is empty The default value if patterns is empty
:returns: :returns:
True if a pattern matches the filename, False if it doesn't. True if a pattern matches the filename, False if it doesn't.
``True`` if patterns is empty. ``default`` if patterns is empty.
""" """
if not patterns: if not patterns:
return True return True
return any(_fnmatch.fnmatch(filename, pattern) for pattern in patterns) return any(_fnmatch.fnmatch(filename, pattern) for pattern in patterns)
def parameters_for(plugin: "Plugin") -> Dict[str, bool]:
"""Return the parameters for the plugin.
This will inspect the plugin and return either the function parameters
if the plugin is a function or the parameters for ``__init__`` after
``self`` if the plugin is a class.
:param plugin:
The internal plugin object.
:type plugin:
flake8.plugins.manager.Plugin
:returns:
A dictionary mapping the parameter name to whether or not it is
required (a.k.a., is positional only/does not have a default).
:rtype:
dict([(str, bool)])
"""
func = plugin.plugin
is_class = not inspect.isfunction(func)
if is_class: # The plugin is a class
func = plugin.plugin.__init__
parameters = {
parameter.name: parameter.default is parameter.empty
for parameter in inspect.signature(func).parameters.values()
if parameter.kind == parameter.POSITIONAL_OR_KEYWORD
}
if is_class:
parameters.pop("self", None)
return parameters
def matches_filename( def matches_filename(
path: str, path: str,
patterns: Sequence[str], patterns: Sequence[str],
@ -237,14 +411,18 @@ def matches_filename(
) -> bool: ) -> bool:
"""Use fnmatch to discern if a path exists in patterns. """Use fnmatch to discern if a path exists in patterns.
:param path: :param str path:
The path to the file under question The path to the file under question
:param patterns: :param patterns:
The patterns to match the path against. The patterns to match the path against.
:param log_message: :type patterns:
list[str]
:param str log_message:
The message used for logging purposes. The message used for logging purposes.
:returns: :returns:
True if path matches patterns, False otherwise True if path matches patterns, False otherwise
:rtype:
bool
""" """
if not patterns: if not patterns:
return False return False
@ -267,14 +445,11 @@ def get_python_version() -> str:
:returns: :returns:
Implementation name, version, and platform as a string. Implementation name, version, and platform as a string.
:rtype:
str
""" """
return "{} {} on {}".format( return "{} {} on {}".format(
platform.python_implementation(), platform.python_implementation(),
platform.python_version(), platform.python_version(),
platform.system(), platform.system(),
) )
def normalize_pypi_name(s: str) -> str:
"""Normalize a distribution name according to PEP 503."""
return NORMALIZE_PACKAGE_NAME_RE.sub("-", s).lower()

View file

@ -1,69 +0,0 @@
"""Contains the Violation error class used internally."""
from __future__ import annotations
import functools
import linecache
import logging
from re import Match
from typing import NamedTuple
from flake8 import defaults
from flake8 import utils
LOG = logging.getLogger(__name__)
@functools.lru_cache(maxsize=512)
def _find_noqa(physical_line: str) -> Match[str] | None:
return defaults.NOQA_INLINE_REGEXP.search(physical_line)
class Violation(NamedTuple):
"""Class representing a violation reported by Flake8."""
code: str
filename: str
line_number: int
column_number: int
text: str
physical_line: str | None
def is_inline_ignored(self, disable_noqa: bool) -> bool:
"""Determine if a comment has been added to ignore this line.
:param disable_noqa:
Whether or not users have provided ``--disable-noqa``.
:returns:
True if error is ignored in-line, False otherwise.
"""
physical_line = self.physical_line
# TODO(sigmavirus24): Determine how to handle stdin with linecache
if disable_noqa:
return False
if physical_line is None:
physical_line = linecache.getline(self.filename, self.line_number)
noqa_match = _find_noqa(physical_line)
if noqa_match is None:
LOG.debug("%r is not inline ignored", self)
return False
codes_str = noqa_match.groupdict()["codes"]
if codes_str is None:
LOG.debug("%r is ignored by a blanket ``# noqa``", self)
return True
codes = set(utils.parse_comma_separated_list(codes_str))
if self.code in codes or self.code.startswith(tuple(codes)):
LOG.debug(
"%r is ignored specifically inline with ``# noqa: %s``",
self,
codes_str,
)
return True
LOG.debug(
"%r is not ignored inline with ``# noqa: %s``", self, codes_str,
)
return False

View file

@ -1,2 +1 @@
"""This is here because mypy doesn't understand PEP 420.""" """This is here because mypy doesn't understand PEP 420."""
from __future__ import annotations

View file

@ -1,6 +1,4 @@
"""Test configuration for py.test.""" """Test configuration for py.test."""
from __future__ import annotations
import sys import sys
import flake8 import flake8

42
tests/fixtures/config_files/README.rst vendored Normal file
View file

@ -0,0 +1,42 @@
About this directory
====================
The files in this directory are test fixtures for unit and integration tests.
Their purpose is described below. Please note the list of file names that can
not be created as they are already used by tests.
New fixtures are preferred over updating existing features unless existing
tests will fail.
Files that should not be created
--------------------------------
- ``tests/fixtures/config_files/missing.ini``
Purposes of existing fixtures
-----------------------------
``tests/fixtures/config_files/cli-specified.ini``
This should only be used when providing config file(s) specified by the
user on the command-line.
``tests/fixtures/config_files/local-config.ini``
This should be used when providing config files that would have been found
by looking for config files in the current working project directory.
``tests/fixtures/config_files/local-plugin.ini``
This is for testing configuring a plugin via flake8 config file instead of
setuptools entry-point.
``tests/fixtures/config_files/no-flake8-section.ini``
This should be used when parsing an ini file without a ``[flake8]``
section.
``tests/fixtures/config_files/user-config.ini``
This is an example configuration file that would be found in the user's
home directory (or XDG Configuration Directory).

View file

@ -0,0 +1,9 @@
[flake8]
exclude =
<<<<<<< 642f88cb1b6027e184d9a662b255f7fea4d9eacc
tests/fixtures/,
=======
tests/,
>>>>>>> HEAD
docs/
ignore = D203

View file

@ -0,0 +1,16 @@
[flake8]
# This is a flake8 config, there are many like it, but this is mine
ignore =
# Disable E123
E123,
# Disable W234
W234,
# Also disable E111
E111
exclude =
# Exclude foo/
foo/,
# Exclude bar/ while we're at it
bar/,
# Exclude bogus/
bogus/

View file

@ -0,0 +1,16 @@
[flake8]
# This is a flake8 config, there are many like it, but this is mine
# Disable E123
# Disable W234
# Also disable E111
ignore =
E123,
W234,
E111
# Exclude foo/
# Exclude bar/ while we're at it
# Exclude bogus/
exclude =
foo/,
bar/,
bogus/

Some files were not shown because too many files have changed in this diff Show more