When calling `ArgumentParser.parse_args()` with the `namespace`
argument, command-line options are just added to the namespace without
going through any of the argument parsing and type conversion logic
(e.g., the `type` keyword argument of `ArgumentParser.add_argument()`).
In other words, it is assumed that a namespace is well-formed from a
previous invocation of `ArgumentParser.parse_args()`.
The `values` parameter is intended to be values already-provided from
configuration files. To take advantage of the logic defined by
`ArgumentParser.add_argument()`, utilize
`ArgumentParser.set_defaults()` instead.
`flake8.main.cli.main()` is the primary entry point for the command-line
implementation of flake8 (invoked via `__main__` or `console_scripts`).
Therefore, it is reasonable for the entry point to be responsible for
obtaining command line arguments from `sys.argv` there.
Note that `sys.argv[1:]` is necessary in order to strip off the script
name. Formerly, this was not needed in
`Application.parse_preliminary_options_and_args()`, which was using
`sys.argv[:]` because the result of the argument parsing was just for
determining additional configuration to be loaded. Then, the *real* CLI
argument parsing was forwarding the original `None` value to
`argparse.ArgumentParser.parse_args()`, which internally was obtaining
arguments as `sys.argv[1:]`.
Additionally, the contract for various argument parsing methods to be
guaranteed a `List[str]`.
Now that callers are ensuring that `value` is not `None`, we can further
tighten the contract and remove the conditional to account when `None`
is passed-in for `value`.
Additionally, add a new test vector to account for when an empty string
is passed in, which would fail `if no value`.
Now that the contract has narrowed for `utils.normalize_paths()`
and `utils.parse_comma_separated_list()`, `Option.normalize()` must
handle when the option value is either a singular value or a sequence
(i.e., `list`) of values.
The paths where `Option.normalize()` is called are:
1. options/config.py: `MergedConfigParser.parse_*_config()`
There are 3 paths wehre eventually `_normalize_value()` is called.
2. options/manager.py: `OptionManager.parse_args()`
For (1), values are coming from the `configparser` module. For (2),
values are coming from `optparse.OptionParser`.
Rudimentary investigation seems to implicate that
`optparse.OptionParser.parse_args()` always returns values in a `list`
because it accumulates values. However, for `configparser`, the values
are a string due to the key-value nature of the INI format.
Given that `Option.normalize()` is the convergence point where
normalization of an option occurs, it is acceptable for the method to
also handle the parsing comma-separated values and path normalization by
the option value's type.
The `normalize_paths()` utility was doing too much — parsing
unstructured configuration path data and dispatching the scrubbed paths
to be normalized.
Towards moving the parsing of unstructured configuration path data
closer towards were configuration occurs, have the utility accept only
structured input for normalizing paths.
Move the path normalization for extra configuration file paths down into
the main `config` module where other path normalization occurs.
This also guarantees that the call to `utils.normalize_paths()` is given
a sequence, instead of a potential `None` value.
It's unnecessary to call the `normalize_paths()` function because it is
intended for dealing with multiple paths to normalize. Given that
`normalize_paths()` utilizes `normalize_path()`, just call
`normalize_path()` directly.
`Application.parse_preliminary_options_and_args` was previously, against
expectations, treating empty lists passed as the `argv` argument the
same way it treated `None`s.
This has been addressed and the correct behavior tested for in a unit
test of the `Application` class.
See issue #518 for details.
~40% improvement over status quo (perf measurements are best-of-5)
### before
```console
$ time flake8 /dev/null
real 0m0.337s
user 0m0.212s
sys 0m0.028s
```
### after
```console
$ time flake8 /dev/null
real 0m0.197s
user 0m0.182s
sys 0m0.012s
```
With a large number of errors, filenames, and per-file-ignores the style guide
selection would take a significant portion of execution time
(python3 70% / python2 99.9%).
Caching the styleguide lookup by filename eliminates this bottleneck.
