Due to https://bugs.python.org/issue27649, we cannot continue to
expect multiprocessing to work as we expect and document it on Windows.
As such, we are going to revert back to our previous behaviour of
disabling it across all versions of Python on Windows to provide the
default expected behaviour of Flake8 on that Operating System.
Allow plugins that are on by default to be ignored
*Description of changes*
Previously, users were unable to ignore the error code of a plugin that was on by default. This adds a tiny bit of extra logic to re-enable that behaviour.
*Related to:* #195
See merge request !104
Previously, to ensure that plugins on by default were reported, we
added them to the select list. This means that ignoring them became
impossible. To accomodate our reporting logic and a user's ability
to ignore, we need to keep our select and extended select lists
separated.
This allows us to have a better understanding of who is selecting what,
where, and how and make our decision as to whether or not an error
should be reported more wisely.
Closes#195
Raise exception when entry_point not callable
Raise exception when entry_point not callable
E.g.:
```
$ .tox/flake8/bin/flake8 mobileweb
Traceback (most recent call last):
File ".tox/flake8/bin/flake8", line 9, in <module>
load_entry_point('flake8', 'console_scripts', 'flake8')()
File "/Users/marca/dev/git-repos/flake8/src/flake8/main/cli.py", line 16, in main
app.run(argv)
File "/Users/marca/dev/git-repos/flake8/src/flake8/main/application.py", line 316, in run
self._run(argv)
File "/Users/marca/dev/git-repos/flake8/src/flake8/main/application.py", line 299, in _run
self.initialize(argv)
File "/Users/marca/dev/git-repos/flake8/src/flake8/main/application.py", line 289, in initialize
self.find_plugins()
File "/Users/marca/dev/git-repos/flake8/src/flake8/main/application.py", line 143, in find_plugins
self.check_plugins.load_plugins()
File "/Users/marca/dev/git-repos/flake8/src/flake8/plugins/manager.py", line 375, in load_plugins
plugins = list(self.manager.map(load_plugin))
File "/Users/marca/dev/git-repos/flake8/src/flake8/plugins/manager.py", line 267, in map
yield func(self.plugins[name], *args, **kwargs)
File "/Users/marca/dev/git-repos/flake8/src/flake8/plugins/manager.py", line 373, in load_plugin
return plugin.load_plugin()
File "/Users/marca/dev/git-repos/flake8/src/flake8/plugins/manager.py", line 173, in load_plugin
raise failed_to_load
flake8.exceptions.FailedToLoadPlugin: Flake8 failed to load plugin "P999" due to Plugin <module 'teamcity.flake8_plugin' from '/Users/marca/dev/git-repos/teamcity-messages/teamcity/flake8_plugin.pyc'> is not a callable. It might be written for an older version of flake8 and might not work with this version.
```
This is nicer than the previous behavior of raising an obscure `PicklingError` and then hanging.
```
$ .tox/flake8/bin/flake8 mobileweb
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/queues.py", line 268, in _feed
send(obj)
PicklingError: Can't pickle <type 'module'>: attribute lookup __builtin__.module failed
```
*Related to:* 164
See merge request !101
E.g.:
```
$ .tox/flake8/bin/flake8 mobileweb
Traceback (most recent call last):
File ".tox/flake8/bin/flake8", line 9, in <module>
load_entry_point('flake8', 'console_scripts', 'flake8')()
File "/Users/marca/dev/git-repos/flake8/src/flake8/main/cli.py", line 16, in main
app.run(argv)
File "/Users/marca/dev/git-repos/flake8/src/flake8/main/application.py", line 316, in run
self._run(argv)
File "/Users/marca/dev/git-repos/flake8/src/flake8/main/application.py", line 299, in _run
self.initialize(argv)
File "/Users/marca/dev/git-repos/flake8/src/flake8/main/application.py", line 289, in initialize
self.find_plugins()
File "/Users/marca/dev/git-repos/flake8/src/flake8/main/application.py", line 143, in find_plugins
self.check_plugins.load_plugins()
File "/Users/marca/dev/git-repos/flake8/src/flake8/plugins/manager.py", line 375, in load_plugins
plugins = list(self.manager.map(load_plugin))
File "/Users/marca/dev/git-repos/flake8/src/flake8/plugins/manager.py", line 267, in map
yield func(self.plugins[name], *args, **kwargs)
File "/Users/marca/dev/git-repos/flake8/src/flake8/plugins/manager.py", line 373, in load_plugin
return plugin.load_plugin()
File "/Users/marca/dev/git-repos/flake8/src/flake8/plugins/manager.py", line 173, in load_plugin
raise failed_to_load
flake8.exceptions.FailedToLoadPlugin: Flake8 failed to load plugin "P999" due to Plugin <module 'teamcity.flake8_plugin' from '/Users/marca/dev/git-repos/teamcity-messages/teamcity/flake8_plugin.pyc'> is not a callable. It might be written for an older version of flake8 and might not work with this version.
```
This is nicer than the previous behavior of raising an obscure `PicklingError` and then hanging.
```
$ .tox/flake8/bin/flake8 mobileweb
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/queues.py", line 268, in _feed
send(obj)
PicklingError: Can't pickle <type 'module'>: attribute lookup __builtin__.module failed
```
See #164
Sort reports by line and column
*Description of changes*
The reports have been incorrectly sorted. This fixes the sorting order to use the line number and then column. It also adds a test to verify that the sorting algorithm works.
*Related to:* #196
See merge request !100
This makes the regular expression a bit more complex, and potentially
slower, but it will fix the issue where users had noqa comments with
colons followed by explanations.
Closes#178
Plugins do not have their parameters checked in advance, so when we
try to aggregate parameters for a plugin, there's a chance it may
request an attribute of the FileProcessor that simply does not exist.
We catch this and re-raise it but we should also capture it in the
checker manager and handle it appropriately there as well.
Fix multiprocessing on Windows
*Related to:* #184
I'm still running into #179 with this fix but all processes start now at least.
BTW the current design is pretty bad regarding memory footprint and bootstrap time as you have to pickle the manager object for *every* subprocess. So this does not only fix multiprocessing on Windows. It improves the multiprocessing design on other platforms, too.
See merge request !97
Fix read_lines_splits_lines test for CRLF endings
This change makes the test pass for when files are saved with CRLF Windows-style line endings since those are included in various `.readlines()` methods by default.
See merge request !96
Clarify what the X in the entry points does
*Description of changes*
Previously the `X` wasn't further explained in the documentation on how to
register a plugin. This plugs the hole at least for checking plugins.
*Related to:* #183
See merge request !93
When attempting to centralize all inline NoQA handling in the StyleGuide
we inadvertently broke plugins relying on it in combination with checker
state. For example, the check for E402 relies both on NoQA and the state
to determine if it has seen a non-import line. Placing NoQA on the sole
line that is not an import is more elegant than placing it on each of
the following import lines.
Closes#186
Update our flake8 testenv dependencies
*Description of changes*
Uncap Flake8 and add a lower bound to flake8-import-order
*Related to:* N/A
See merge request !94
Previously the `X` wasn't further explained in the documentation on how to
register a plugin. This plugs the hole at least for checking plugins.
See also: #183
Fix project config file discovery
*Description of changes*
Keep searching past the current directory to find project config files.
*Related to:* #181
See merge request !91
Flake8 3.0 was stopping once it found the current directory but the
historical behaviour (that we didn't intend to break) searched past
that (towards root) until it found one of the project/local config
file names that could be read.
Closes#181
The --tee option allows the linter report to be written to stdout, even
though it is being redirected to a file with the --output-file option.
This is useful if I want to store the report in a separate file for later
analysis but also be able to print the output on screen (e.g when running
in a CI environment).
Fundamentally on Flake8 2.x using -q altered the format of the errors
(and the behaviour a little) so it makes the most sense to implement
this logic with formatters rather than messy logic spread throughout
the project.
The FilenameOnly formatter will keep track of filenames already reported
and only print the name once while Nothing will print nothing.
Closes#180
Set-up and stop our formatter
*Description of changes*
Make `--output-file` work consistently (especially without verbose logging)
*Related to:* #180
See merge request !86
Update documentation to separate parameter types
This is the documentation update separated from !80 which should be pretty uncontroversial. I already applied the comment on the documentation here.
See merge request !83
Handle multiline strings with '# noqa'
*Description of changes*
I had overlooked a usecase of Flake8 where people use `# noqa` at the end of a multi-line string. This addresses that oversight
*Related to:* #177
See merge request !85
In Flake8 2.x we allowed people to use # noqa at the end of a multiline
string to ignore errors inside the string (e.g., E501). Being blissfully
ignorant of this, I never accounted for it in Flake8 3. This fixes the
oversight and allows multiline statements to have the # noqa at the end.
Closes#177
It updates the documentation to separate which parameters are static and
which are changed on each line. Using the latter parameters on plugins which
are only run once per file isn't very sensible.
Force flake8 test to below 3.x
The current version (0.8) of `flake8-import-order` still uses `parser.config_options` and so all builds fail because it installs Flake8 3.x.
Now the repository of that plugin already accounted for that but as long as it isn't released it'll cause all new builds to fail. Alternatively I can repurpose this merge request to actually enforce a newer version of `flake8-import-order` which supports Flake8 3.x.
See merge request !84
It is possible to write plugins which are only a function. At the moment they
are called on each line manually. This allows the function also to be called
on each file once. It works similar to creating the class and calling `run` on
it immediately. The plugin function needs to return a generator.
This is based on the original comment in the `FileChecker.run_ast_checks`
method, but slightly modified as the original comment would've called the
return of the function. But the function could return the reports directly.
