Platforms such as Termux on Android, and other exotic devices
do not provide a sem_open implementation on the OS level. This
is problematic, as the error resulting from this occurs when
calling multiprocessing.Pool, throwing an unhandled ImportError.
The issue itself is outlined in https://bugs.python.org/issue3770.
This change allows devices missing this system call to respond
to the missing feature by falling back to synchronous execution,
which appears to be the default behaviour if the multiprocessing
module is not found.
This change also adds a potential fix for developers working
on platforms where multiprocessing itself cannot be imported.
The existing code would set the name referencing the import to
None, but there are no clear checks to ensure this does not
result in an AttributeError later when multiprocessing.Pool
has accession attempts.
Existing users should see no difference in functionality, as they
will assumably already be able to use flake8, so will not be
missing this sem_open call.
Users on devices without the sem_open call will now be able
to use flake8 where they would be unable to before due to
unhandled ImportErrors.
Pytest 6.0.0rc1 attempts to log a repr of our mocked entry_point,
but it fails, becasue the repr metohod expects a valua attribute:
Reproducer:
$ tox -e py38 --force-dep 'pytest==6.0.0rc1'
...
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/__init__.py", line 1081, in emit
msg = self.format(record)
File "/usr/lib/python3.8/logging/__init__.py", line 925, in format
return fmt.format(record)
File "/usr/lib/python3.8/logging/__init__.py", line 664, in format
record.message = record.getMessage()
File "/usr/lib/python3.8/logging/__init__.py", line 369, in getMessage
msg = msg % self.args
File "/tmp/flake8/.tox/py38/lib/python3.8/site-packages/flake8/plugins/manager.py", line 44, in __repr__
self.name, self.entry_point.value
File "/tmp/flake8/.tox/py38/lib/python3.8/site-packages/mock/mock.py", line 632, in __getattr__
raise AttributeError("Mock object has no attribute %r" % name)
AttributeError: Mock object has no attribute 'value'
This fixes a regression introduced in daca2c8 and b14d47b.
The --output-file` option was consumed by the preliminary option parser.
However, the `BaseFormatter` class needs the option for setting the
output filename. This special cases this option to ensure it gets
re-parsed and respected when specified on the CLI.
Now that `ConfigFileFinder.config_file` attribute is used everywhere and
is constructed from the `--config` CLI option, the now unused
`cli_config` parameters can be safely removed.
Now that the `ConfigFileFinder` has the `.config_file` attribute, switch
the relevant code paths to utilize this public attribute.
Tests have been updated to either construct `ConfigFileFinder` or mock
the object appropriately.
Now that `ConfigFileFinder.ignore_config_files` attribute is used
everywhere and is constructed from the `--isolated` CLI option, the now
unused `isolated` parameters can be safely removed.
Now that the `ConfigFileFinder` has the `.ignore_config_files`
attribute, switch the relevant code paths to utilize this public
attribute.
Tests have been updated to either construct `ConfigFileFinder` or mock
the object appropriately.
This fixes a regression introduced in !346 to ensure that `--config` and
`--isolated` are recognized in `aggregate_options()`.
The regression manifested because `aggregate_options()` was relying on
re-parsing `argv` to obtain the option values. However, !346 changed
the preliminary parsing logic to only parse and "eat" what is necessary
and forward along the options needed before all the configuration was
loaded. This code path was overlooked because the tests in
`test_aggregator()` were passing but the call from the `Application`
object would never have these options in the remaining `argv` list to be
passed long.
We introduce a new `ArgumentParser` for registering the preliminary
options to be inherited by the `Application.option_manager`. The next
step will be to use the `Application.prelim_arg_parser` for parsing and
handling preliminary options and arguments.
Note that we prevent the preliminary parser from handling `-h/--help`
and defer to that to the primary parser.
Now that `args` parameters is not being used, it is safe to remove from
the constructor signature.
Further work is required to evaluate and clean-up tearing out the
threading-through of `args` from various callers and tests.
`flake8.main.cli.main()` is the primary entry point for the command-line
implementation of flake8 (invoked via `__main__` or `console_scripts`).
Therefore, it is reasonable for the entry point to be responsible for
obtaining command line arguments from `sys.argv` there.
Note that `sys.argv[1:]` is necessary in order to strip off the script
name. Formerly, this was not needed in
`Application.parse_preliminary_options_and_args()`, which was using
`sys.argv[:]` because the result of the argument parsing was just for
determining additional configuration to be loaded. Then, the *real* CLI
argument parsing was forwarding the original `None` value to
`argparse.ArgumentParser.parse_args()`, which internally was obtaining
arguments as `sys.argv[1:]`.
Additionally, the contract for various argument parsing methods to be
guaranteed a `List[str]`.
This adds two new methods to the BaseFormatter class:
- beginning
- finished
These will indicate when Flake8 begins and finishes processing a file.
Closes#251
It seems likely that the multiprocessing module on Windows is not
capable of serializing an object with the structure that we have and
preserving the attributes we dynamically set on plugins (like the
FlakesChecker). To avoid issues like this with all plugins (although
we have only found this on Windows with the FlakesChecker), let's try
serializing the Checkers PluginTypeManager to a dictionary so that the
only object that a Queue is really trying to serialize/deserialize is
the FlakesChecker itself.
Related to #179
This relies on two things:
1. Properly configuring flake8-import-order to use that style
2. Properly configuring flake8-import-order to know that flake8 is our
application name.
