Rename dev subdirectory to plugin-development

This should make the contents clearer

docs/source/plugin-development/cross-compatibility.rst

@@ -0,0 +1,150 @@
+====================================
+ Writing Plugins For Flake8 2 and 3
+====================================
+
+Plugins have existed for |Flake8| 2.x for a few years. There are a number of
+these on PyPI already. While it did not seem reasonable for |Flake8| to attempt
+to provide a backwards compatible shim for them, we did decide to try to
+document the easiest way to write a plugin that's compatible across both
+versions.
+
+.. note::
+
+    If your plugin does not register options, it *should* Just Work.
+
+The **only** breaking change in |Flake8| 3.0 is the fact that we no longer
+check the option parser for a list of strings to parse from a config file. On
+|Flake8| 2.x, to have an option parsed from the configuration files that
+|Flake8| finds and parses you would have to do something like:
+
+.. code-block:: python
+
+    parser.add_option('-X', '--example-flag', type='string',
+                      help='...')
+    parser.config_options.append('example-flag')
+
+For |Flake8| 3.0, we have added *three* arguments to the
+:meth:`~flake8.options.manager.OptionManager.add_option` method you will call
+on the parser you receive:
+
+- ``parse_from_config`` which expects ``True`` or ``False``
+
+  When ``True``, |Flake8| will parse the option from the config files |Flake8|
+  finds.
+
+- ``comma_separated_list`` which expects ``True`` or ``False``
+
+  When ``True``, |Flake8| will split the string intelligently and handle
+  extra whitespace. The parsed value will be a list.
+
+- ``normalize_paths`` which expects ``True`` or ``False``
+
+  When ``True``, |Flake8| will:
+
+  * remove trailing path separators (i.e., ``os.path.sep``)
+
+  * return the absolute path for values that have the separator in them
+
+All three of these options can be combined or used separately.
+
+
+Parsing Options from Configuration Files
+========================================
+
+The example from |Flake8| 2.x now looks like:
+
+.. code-block:: python
+
+    parser.add_option('-X', '--example-flag', type='string',
+                      parse_from_config=True,
+                      help='...')
+
+
+Parsing Comma-Separated Lists
+=============================
+
+Now let's imagine that the option we want to add is expecting a comma-separatd
+list of values from the user (e.g., ``--select E123,W503,F405``). |Flake8| 2.x
+often forced users to parse these lists themselves since pep8 special-cased
+certain flags and left others on their own. |Flake8| 3.0 adds
+``comma_separated_list`` so that the parsed option is already a list for
+plugin authors. When combined with ``parse_from_config`` this means that users
+can also do something like:
+
+.. code-block:: ini
+
+    example-flag =
+        first,
+        second,
+        third,
+        fourth,
+        fifth
+
+And |Flake8| will just return the list:
+
+.. code-block:: python
+
+    ["first", "second", "third", "fourth", "fifth"]
+
+
+Normalizing Values that Are Paths
+=================================
+
+Finally, let's imagine that our new option wants a path or list of paths. To
+ensure that these paths are semi-normalized (the way |Flake8| 2.x used to
+work) we need only pass ``normalize_paths=True``. If you have specified
+``comma_separated_list=True`` then this will parse the value as a list of
+paths that have been normalized. Otherwise, this will parse the value
+as a single path.
+
+
+Option Handling on Flake8 2 and 3
+=================================
+
+So, in conclusion, we can now write our plugin that relies on registering
+options with |Flake8| and have it work on |Flake8| 2.x and 3.x.
+
+.. code-block:: python
+
+    option_args = ('-X', '--example-flag')
+    option_kwargs = {
+        'type': 'string',
+        'parse_from_config': True,
+        'help': '...',
+    }
+    try:
+        # Flake8 3.x registration
+        parser.add_option(*option_args, **option_kwargs)
+    except TypeError:
+        # Flake8 2.x registration
+        parse_from_config = option_kwargs.pop('parse_from_config', False)
+        parser.add_option(*option_args, **option_kwargs)
+        if parse_from_config:
+            parser.config_options.append(option_args[-1].lstrip('-'))
+
+
+Or, you can write a tiny helper function:
+
+.. code-block:: python
+
+    def register_opt(parser, *args, **kwargs):
+        try:
+            # Flake8 3.x registration
+            parser.add_option(*args, **kwargs)
+        except TypeError:
+            # Flake8 2.x registration
+            parse_from_config = kwargs.pop('parse_from_config', False)
+            parser.add_option(*args, **kwargs)
+            if parse_from_config:
+                parser.config_options.append(args[-1].lstrip('-'))
+
+.. code-block:: python
+
+    @classmethod
+    def register_options(cls, parser):
+        register_opt(parser, '-X', '--example-flag', type='string',
+                     parse_from_config=True, help='...')
+
+The transition period is admittedly not fantastic, but we believe that this
+is a worthwhile change for plugin developers going forward. We also hope to
+help with the transition phase for as many plugins as we can manage.

docs/source/plugin-development/formatters.rst

@@ -0,0 +1,54 @@
+.. _formatting-plugins:
+
+===========================================
+ Developing a Formatting Plugin for Flake8
+===========================================
+
+|Flake8| allowed for custom formatting plugins in version
+3.0.0. Let's write a plugin together:
+
+.. code-block:: python
+
+    from flake8.formatting import base
+
+
+    class Example(base.BaseFormatter):
+        """Flake8's example formatter."""
+
+        pass
+
+We notice, as soon as we start, that we inherit from |Flake8|'s
+:class:`~flake8.formatting.base.BaseFormatter` class. If we follow the
+:ref:`instructions to register a plugin <register-a-plugin>` and try to use
+our example formatter, e.g., ``flake8 --format=example`` then
+|Flake8| will fail because we did not implement the ``format`` method.
+Let's do that next.
+
+.. code-block:: python
+
+    class Example(base.BaseFormatter):
+        """Flake8's example formatter."""
+
+        def format(self, error):
+            return 'Example formatter: {0!r}'.format(error)
+
+With that we're done. Obviously this isn't a very useful formatter, but it
+should highlight the simplicitly of creating a formatter with Flake8. If we
+wanted to instead create a formatter that aggregated the results and returned
+XML, JSON, or subunit we could also do that. |Flake8| interacts with the
+formatter in two ways:
+
+#. It creates the formatter and provides it the options parsed from the
+   configuration files and command-line
+
+#. It uses the instance of the formatter and calls ``handle`` with the error.
+
+By default :meth:`flake8.formatting.base.BaseFormatter.handle` simply calls
+the ``format`` method and then ``write``. Any extra handling you wish to do
+for formatting purposes should override the ``handle`` method.
+
+API Documentation
+=================
+
+.. autoclass:: flake8.formatting.base.BaseFormatter
+    :members:

docs/source/plugin-development/index.rst

@@ -0,0 +1,56 @@
+============================
+ Writing Plugins for Flake8
+============================
+
+Since |Flake8| 2.0, the |Flake8| tool has allowed for extensions and custom
+plugins. In |Flake8| 3.0, we're expanding that ability to customize and
+extend **and** we're attempting to thoroughly document it. Some of the
+documentation in this section may reference third-party documentation to
+reduce duplication and to point you, the developer, towards the authoritative
+documentation for those pieces.
+
+Getting Started
+===============
+
+To get started writing a |Flake8| :term:`plugin` you first need:
+
+- An idea for a plugin
+
+- An available package name on PyPI
+
+- One or more versions of Python installed
+
+- A text editor or IDE of some kind
+
+- An idea of what *kind* of plugin you want to build:
+
+  * Formatter
+
+  * Check
+
+Once you've gathered these things, you can get started.
+
+All plugins for |Flake8| must be registered via `entry points`_. In this
+section we cover:
+
+- How to register your plugin so |Flake8| can find it
+
+- How to make |Flake8| provide your check plugin with information (via
+  command-line flags, function/class parameters, etc.)
+
+- How to make a formatter plugin
+
+- How to write your check plugin so that it works with |Flake8| 2.x and 3.x
+
+.. toctree::
+    :caption: Plugin Developer Documentation
+    :maxdepth: 2
+
+    registering-plugins
+    plugin-parameters
+    formatters
+    cross-compatibility
+
+
+.. _entry points:
+    https://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points

docs/source/plugin-development/plugin-parameters.rst

@@ -0,0 +1,163 @@
+.. _plugin-parameters:
+
+==========================================
+ Receiving Information For A Check Plugin
+==========================================
+
+Plugins to |Flake8| have a great deal of information that they can request
+from a :class:`~flake8.processor.FileProcessor` instance. Historically,
+|Flake8| has supported two types of plugins:
+
+#. classes that accept parsed abstract syntax trees (ASTs)
+
+#. functions that accept a range of arguments
+
+|Flake8| now does not distinguish between the two types of plugins. Any plugin
+can accept either an AST or a range of arguments. Further, any plugin that has
+certain callable attributes can also register options and receive parsed
+options.
+
+
+Indicating Desired Data
+=======================
+
+|Flake8| inspects the plugin's signature to determine what parameters it
+expects using :func:`flake8.utils.parameters_for`.
+:attr:`flake8.plugins.manager.Plugin.parameters` caches the values so that
+each plugin makes that fairly expensive call once per plugin. When processing
+a file, a plugin can ask for any of the following:
+
+- :attr:`~flake8.processor.FileProcessor.blank_before`
+- :attr:`~flake8.processor.FileProcessor.blank_lines`
+- :attr:`~flake8.processor.FileProcessor.checker_state`
+- :attr:`~flake8.processor.FileProcessor.indect_char`
+- :attr:`~flake8.processor.FileProcessor.indent_level`
+- :attr:`~flake8.processor.FileProcessor.line_number`
+- :attr:`~flake8.processor.FileProcessor.logical_line`
+- :attr:`~flake8.processor.FileProcessor.max_line_length`
+- :attr:`~flake8.processor.FileProcessor.multiline`
+- :attr:`~flake8.processor.FileProcessor.noqa`
+- :attr:`~flake8.processor.FileProcessor.previous_indent_level`
+- :attr:`~flake8.processor.FileProcessor.previous_logical`
+- :attr:`~flake8.processor.FileProcessor.tokens`
+- :attr:`~flake8.processor.FileProcessor.total_lines`
+- :attr:`~flake8.processor.FileProcessor.verbose`
+
+Alternatively, a plugin can accept ``tree`` and ``filename``.
+``tree`` will be a parsed abstract syntax tree that will be used by plugins
+like PyFlakes and McCabe.
+
+
+Registering Options
+===================
+
+Any plugin that has callable attributes ``provide_options`` and
+``register_options`` can parse option information and register new options.
+
+Your ``register_options`` function should expect to receive an instance of
+|OptionManager|. An |OptionManager| instance behaves very similarly to
+:class:`optparse.OptionParser`. It, however, uses the layer that |Flake8| has
+developed on top of :mod:`optparse` to also handle configuration file parsing.
+:meth:`~flake8.options.manager.OptionManager.add_option` creates an |Option|
+which accepts the same parameters as :mod:`optparse` as well as three extra
+boolean parameters:
+
+- ``parse_from_config``
+
+  The command-line option should also be parsed from config files discovered
+  by |Flake8|.
+
+  .. note::
+
+      This takes the place of appending strings to a list on the
+      :class:`optparse.OptionParser`.
+
+- ``comma_separated_list``
+
+  The value provided to this option is a comma-separated list. After parsing
+  the value, it should be further broken up into a list. This also allows us
+  to handle values like:
+
+  .. code::
+
+      E123,E124,
+      E125,
+        E126
+
+- ``normalize_paths``
+
+  The value provided to this option is a path. It should be normalized to be
+  an absolute path. This can be combined with ``comma_separated_list`` to
+  allow a comma-separated list of paths.
+
+Each of these options works individually or can be combined. Let's look at a
+couple examples from |Flake8|. In each example, we will have
+``option_manager`` which is an instance of |OptionManager|.
+
+.. code-block:: python
+
+    option_manager.add_option(
+        '--max-line-length', type='int', metavar='n',
+        default=defaults.MAX_LINE_LENGTH, parse_from_config=True,
+        help='Maximum allowed line length for the entirety of this run. '
+             '(Default: %default)',
+    )
+
+Here we are adding the ``--max-line-length`` command-line option which is
+always an integer and will be parsed from the configuration file. Since we
+provide a default, we take advantage of :mod:`optparse`\ 's willingness to
+display that in the help text with ``%default``.
+
+.. code-block:: python
+
+    option_manager.add_option(
+        '--select', metavar='errors', default='',
+        parse_from_config=True, comma_separated_list=True,
+        help='Comma-separated list of errors and warnings to enable.'
+             ' For example, ``--select=E4,E51,W234``. (Default: %default)',
+    )
+
+In adding the ``--select`` command-line option, we're also indicating to the
+|OptionManager| that we want the value parsed from the config files and parsed
+as a comma-separated list.
+
+.. code-block:: python
+
+    option_manager.add_option(
+        '--exclude', metavar='patterns', default=defaults.EXCLUDE,
+        comma_separated_list=True, parse_from_config=True,
+        normalize_paths=True,
+        help='Comma-separated list of files or directories to exclude.'
+             '(Default: %default)',
+    )
+
+Finally, we show an option that uses all three extra flags. Values from
+``--exclude`` will be parsed from the config, converted from a comma-separated
+list, and then each item will be normalized.
+
+For information about other parameters to
+:meth:`~flake8.options.manager.OptionManager.add_option` refer to the
+documentation of :mod:`optparse`.
+
+
+Accessing Parsed Options
+========================
+
+When a plugin has a callable ``provide_options`` attribute, |Flake8| will call
+it and attempt to provide the |OptionManager| instance, the parsed options
+which will be an instance of :class:`optparse.Values`, and the extra arguments
+that were not parsed by the |OptionManager|. If that fails, we will just pass
+the :class:`optparse.Values`. In other words, your ``provide_options``
+callable will have one of the following signatures:
+
+.. code-block:: python
+
+    def provide_options(option_manager, options, args):
+        pass
+    # or
+    def provide_options(options):
+        pass
+
+.. substitutions
+.. |OptionManager| replace:: :class:`~flake8.options.manager.OptionManager`
+.. |Option| replace:: :class:`~flake8.options.manager.Option`

docs/source/plugin-development/registering-plugins.rst

@@ -0,0 +1,115 @@
+.. _register-a-plugin:
+
+==================================
+ Registering a Plugin with Flake8
+==================================
+
+To register any kind of plugin with |Flake8|, you need:
+
+#. A way to install the plugin (whether it is packaged on its own or
+   as part of something else). In this section, we will use a ``setup.py``
+   written for an example plugin.
+
+#. A name for your plugin that will (ideally) be unique.
+
+#. A somewhat recent version of setuptools (newer than 0.7.0 but preferably as
+   recent as you can attain).
+
+|Flake8| relies on functionality provided by setuptools called
+`Entry Points`_. These allow any package to register a plugin with |Flake8|
+via that package's ``setup.py`` file.
+
+Let's presume that we already have our plugin written and it's in a module
+called ``flake8_example``. We might have a ``setup.py`` that looks something
+like:
+
+.. code-block:: python
+
+    from __future__ import with_statement
+    import setuptools
+
+    requires = [
+        "flake8 > 3.0.0",
+    ]
+
+    flake8_entry_point = # ...
+
+    setuptools.setup(
+        name="flake8_example",
+        license="MIT",
+        version="0.1.0",
+        description="our extension to flake8",
+        author="Me",
+        author_email="example@example.com",
+        url="https://gitlab.com/me/flake8_example",
+        packages=[
+            "flake8_example",
+        ],
+        install_requires=requires,
+        entry_points={
+            flake8_entry_point: [
+                'X = flake8_example:ExamplePlugin',
+            ],
+        },
+        classifiers=[
+            "Environment :: Console",
+            "Intended Audience :: Developers",
+            "License :: OSI Approved :: MIT License",
+            "Programming Language :: Python",
+            "Programming Language :: Python :: 2",
+            "Programming Language :: Python :: 3",
+            "Topic :: Software Development :: Libraries :: Python Modules",
+            "Topic :: Software Development :: Quality Assurance",
+        ],
+    )
+
+Note specifically these lines:
+
+.. code-block:: python
+
+    flake8_entry_point = # ...
+
+    setuptools.setup(
+        # snip ...
+        entry_points={
+            flake8_entry_point: [
+                'X = flake8_example:ExamplePlugin',
+            ],
+        },
+        # snip ...
+    )
+
+We tell setuptools to register our entry point "X" inside the specific
+grouping of entry-points that flake8 should look in.
+
+|Flake8| presently looks at three groups:
+
+- ``flake8.extension``
+
+- ``flake8.listen``
+
+- ``flake8.report``
+
+If your plugin is one that adds checks to |Flake8|, you will use
+``flake8.extension``. If your plugin automatically fixes errors in code, you
+will use ``flake8.listen``.  Finally, if your plugin performs extra report
+handling (formatting, filtering, etc.) it will use ``flake8.report``.
+
+If our ``ExamplePlugin`` is something that adds checks, our code would look
+like:
+
+.. code-block:: python
+
+    setuptools.setup(
+        # snip ...
+        entry_points={
+            'flake8.extension': [
+                'X = flake8_example:ExamplePlugin',
+            ],
+        },
+        # snip ...
+    )
+
+
+.. _Entry Points:
+    https://pythonhosted.org/setuptools/pkg_resources.html#entry-points