mirror of
https://github.com/PyCQA/flake8.git
synced 2026-04-04 20:26:53 +00:00
225 lines
9.2 KiB
ReStructuredText
225 lines
9.2 KiB
ReStructuredText
Option and Configuration Handling
|
|
=================================
|
|
|
|
Option Management
|
|
-----------------
|
|
|
|
Command-line options are often also set in configuration files for Flake8.
|
|
While not all options are meant to be parsed from configuration files, many
|
|
default options are also parsed from configuration files as are most plugin
|
|
options.
|
|
|
|
In Flake8 2, plugins received a :class:`optparse.OptionParser` instance and
|
|
called :meth:`optparse.OptionParser.add_option` to register options. If the
|
|
plugin author also wanted to have that option parsed from config files they
|
|
also had to do something like:
|
|
|
|
.. code-block:: python
|
|
|
|
parser.config_options.append('my_config_option')
|
|
parser.config_options.extend(['config_opt1', 'config_opt2'])
|
|
|
|
This was previously undocumented and led to a lot of confusion as to why
|
|
registered options were not automatically parsed from configuration files.
|
|
|
|
Since Flake8 3 was rewritten from scratch, we decided to take a different
|
|
approach to configuration file parsing. Instead of needing to know about an
|
|
undocumented attribute that pep8 looks for, Flake8 3 now accepts a parameter
|
|
to ``add_option``, specifically ``parse_from_config`` which is a boolean
|
|
value.
|
|
|
|
Flake8 does this by creating its own abstractions on top of :mod:`optparse`.
|
|
The first abstraction is the :class:`flake8.options.manager.Option` class. The
|
|
second is the :class:`flake8.options.manager.OptionManager`. In fact, we add
|
|
three new parameters:
|
|
|
|
- ``parse_from_config``
|
|
|
|
- ``comma_separated_list``
|
|
|
|
- ``normalize_paths``
|
|
|
|
The last two are not specifically for configuration file handling, but they
|
|
do improve that dramatically. We found that there were options that when
|
|
specified in a configuration file, lended themselves to being split across
|
|
multiple lines and those options were almost always comma-separated. For
|
|
example, let's consider a user's list of ignored error codes for a project:
|
|
|
|
.. code-block:: ini
|
|
|
|
[flake8]
|
|
ignore =
|
|
E111, # Reasoning
|
|
E711, # Reasoning
|
|
E712, # Reasoning
|
|
E121, # Reasoning
|
|
E122, # Reasoning
|
|
E123, # Reasoning
|
|
E131, # Reasoning
|
|
E251 # Reasoning
|
|
|
|
It makes sense here to allow users to specify the value this way, but, the
|
|
standard libary's :class:`configparser.RawConfigParser` class does returns a
|
|
string that looks like
|
|
|
|
.. code-block:: python
|
|
|
|
"\nE111, \nE711, \nE712, \nE121, \nE122, \nE123, \nE131, \nE251 "
|
|
|
|
This means that a typical call to :meth:`str.split` with ``','`` will not be
|
|
sufficient here. Telling Flake8 that something is a comma-separated list
|
|
(e.g., ``comma_separated_list=True``) will handle this for you. Flake8 will
|
|
return:
|
|
|
|
.. code-block:: python
|
|
|
|
["E111", "E711", "E712", "E121", "E122", "E123", "E131", "E251"]
|
|
|
|
Next let's look at how users might like to specify their ``exclude`` list.
|
|
Presently OpenStack's Nova project has this line in their `tox.ini`_:
|
|
|
|
.. code-block:: ini
|
|
|
|
exclude = .venv,.git,.tox,dist,doc,*openstack/common/*,*lib/python*,*egg,build,tools/xenserver*,releasenotes
|
|
|
|
I think we can all agree that this would be easier to read like this:
|
|
|
|
.. code-block:: ini
|
|
|
|
exclude =
|
|
.venv,
|
|
.git,
|
|
.tox,
|
|
dist,
|
|
doc,
|
|
*openstack/common/*,
|
|
*lib/python*,
|
|
*egg,
|
|
build,
|
|
tools/xenserver*,
|
|
releasenotes
|
|
|
|
In this case, since these are actually intended to be paths, we would specify
|
|
both ``comma_separated_list=True`` and ``normalize_paths=True`` because we
|
|
want the paths to be provided to us with some consistency (either all absolute
|
|
paths or not).
|
|
|
|
Now let's look at how this would actually be utilized. Most plugin developers
|
|
will receive an instance of :class:`~flake8.options.manager.OptionManager` so
|
|
to ease the transition we kept the same API as the
|
|
:class:`optparse.OptionParser` object. The only difference is that
|
|
:meth:`~flake8.options.manager.OptionManager.add_option` accepts the three
|
|
extra arguments we highlighted above.
|
|
|
|
.. _tox.ini:
|
|
https://github.com/openstack/nova/blob/3eb190c4cfc0eefddac6c2cc1b94a699fb1687f8/tox.ini#L155
|
|
|
|
Configuration File Management
|
|
-----------------------------
|
|
|
|
In Flake8 2, configuration file discovery and management was handled by pep8.
|
|
In pep8's 1.6 release series, it drastically broke how discovery and merging
|
|
worked (as a result of trying to improve it). To avoid a dependency breaking
|
|
Flake8 again in the future, we have created our own discovery and management.
|
|
As part of managing this ourselves, we decided to change management/discovery
|
|
for 3.0.0. We have done the following:
|
|
|
|
- User files (files stored in a user's home directory or in the XDG directory
|
|
inside their home directory) are the first files read. For example, if the
|
|
user has a ``~/.flake8`` file, we will read that first.
|
|
|
|
- Project files (files stored in the current directory) are read next and
|
|
merged on top of the user file. In other words, configuration in project
|
|
files takes precedence over configuration in user files.
|
|
|
|
- **New in 3.0.0** The user can specify ``--append-config <path-to-file>``
|
|
repeatedly to include extra configuration files that should be read and
|
|
take precedence over user and project files.
|
|
|
|
- **New in 3.0.0** The user can specify ``--config <path-to-file>`` to so this
|
|
file is the only configuration file used. This is a change from Flake8 2
|
|
where pep8 would simply merge this configuration file into the configuration
|
|
generated by user and project files (where this takes precedence).
|
|
|
|
- **New in 3.0.0** The user can specify ``--isolated`` to disable
|
|
configuration via discovered configuration files.
|
|
|
|
To facilitate the configuration file management, we've taken a different
|
|
approach to discovery and management of files than pep8. In pep8 1.5, 1.6, and
|
|
1.7 configuration discovery and management was centralized in `66 lines of
|
|
very terse python`_ which was confusing and not very explicit. The terseness
|
|
of this function (Flake8's authors believe) caused the confusion and problems
|
|
with pep8's 1.6 series. As such, Flake8 has separated out discovery,
|
|
management, and merging into a module to make reasoning about each of these
|
|
pieces easier and more explicit (as well as easier to test).
|
|
|
|
Configuration file discovery is managed by the
|
|
:class:`~flake8.options.config.ConfigFileFinder` object. This object needs to
|
|
know information about the program's name, any extra arguments passed to it,
|
|
and any configuration files that should be appended to the list of discovered
|
|
files. It provides methods for finding the files and similiar methods for
|
|
parsing those fles. For example, it provides
|
|
:meth:`~flake8.options.config.ConfigFileFinder.local_config_files` to find
|
|
known local config files (and append the extra configuration files) and it
|
|
also provides :meth:`~flake8.options.config.ConfigFileFinder.local_configs`
|
|
to parse those configuration files.
|
|
|
|
.. note:: ``local_config_files`` also filters out non-existent files.
|
|
|
|
Configuration file merging and managemnt is controlled by the
|
|
:class:`~flake8.options.config.MergedConfigParser`. This requires the instance
|
|
of :class:`~flake8.options.manager.OptionManager` that the program is using,
|
|
the list of appended config files, and the list of extra arguments. This
|
|
object is currently the sole user of the
|
|
:class:`~flake8.options.config.ConfigFileFinder` object. It appropriately
|
|
initializes the object and uses it in each of
|
|
|
|
- :meth:`~flake8.options.config.MergedConfigParser.parse_cli_config`
|
|
- :meth:`~flake8.options.config.MergedConfigParser.parse_local_config`
|
|
- :meth:`~flake8.options.config.MergedConfigParser.parse_user_config`
|
|
|
|
Finally,
|
|
:meth:`~flake8.options.config.MergedConfigParser.merge_user_and_local_config`
|
|
takes the user and local configuration files that are parsed by
|
|
:meth:`~flake8.options.config.MergedConfigParser.parse_local_config` and
|
|
:meth:`~flake8.options.config.MergedConfigParser.parse_user_config`. The
|
|
main usage of the ``MergedConfigParser`` is in
|
|
:func:`~flake8.options.aggregator.aggregate_options`.
|
|
|
|
Aggregating Configuration File and Command Line Arguments
|
|
---------------------------------------------------------
|
|
|
|
:func:`~flake8.options.aggregator.aggregate_options` accepts an instance of
|
|
:class:`~flake8.options.maanger.OptionManager` and does the work to parse the
|
|
command-line arguments passed by the user necessary for creating an instance
|
|
of :class:`~flake8.options.config.MergedConfigParser`.
|
|
|
|
After parsing the configuration file, we determine the default ignore list. We
|
|
use the defaults from the OptionManager and update those with the parsed
|
|
configuration files. Finally we parse the user-provided options one last time
|
|
using the option defaults and configuration file values as defaults. The
|
|
parser merges on the command-line specified arguments for us so we have our
|
|
final, definitive, aggregated options.
|
|
|
|
.. _66 lines of very terse python:
|
|
https://github.com/PyCQA/pep8/blob/b8088a2b6bc5b76bece174efad877f764529bc74/pep8.py#L1981..L2047
|
|
|
|
API Documentation
|
|
-----------------
|
|
|
|
.. autofunction:: flake8.options.aggregator.aggregate_options
|
|
|
|
.. autoclass:: flake8.options.manager.Option
|
|
:members: __init__, normalize, to_optparse
|
|
|
|
.. autoclass:: flake8.options.manager.OptionManager
|
|
:members:
|
|
:special-members:
|
|
|
|
.. autoclass:: flake8.options.config.ConfigFileFinder
|
|
:members:
|
|
:special-members:
|
|
|
|
.. autoclass:: flake8.options.config.MergedConfigParser
|
|
:members:
|
|
:special-members:
|