In a previous commit [0] there was an additional character
escaping added at the end that was not added to the list
in the comment for the function. This change adds the missing
character to the list of escaped characters from the previous
change.
[0] https://review.openstack.org/#/c/439897/
Change-Id: I6e83a10ee51f1f18176bf2d17a0092d5a3cc4dd4
Sphinx 1.6 will support the definition of multiple builders in a
setup.cfg file like so:
[build_sphinx]
builder = html man
Once we support this version of Sphinx, we should stop carrying the
custom versions of this tooling we use.
Upstreaming things FTW.
Change-Id: Ibf2a003229a4585df96b09da7ca547e201c5aef5
codesearch.o.o shows a single, long-dead project using this [1]. Let's
just remove it and push people to set 'builders' instead in they really
want LaTeX.
[1] http://codesearch.openstack.org/?q=build_sphinx_latex
Change-Id: I820d9c540ae81717d7b33bbb4d2a4031b529b52c
From pretty much the beginning [1], pbr has defaulted to building both
man page and html output, but has failed to document it anywhere. People
tend to copy-paste their 'setup.py' and 'conf.py', or rely on the
'cookiecutter' project, with very little understanding of what's going
on under the hood (and why would you care - it's docs :)). This means
that the vast majority of folks using 'pbr' (basically everyone in
OpenStack) have been unwittingly building "man pages" as part of their
doc builds for no good reason, which has also led to a lot of confusion
when this magic behavior is the cause of bugs [2][3].
There's no good reason that pbr should default to building both man
pages and html output. For folks that want this functionality, we should
document it so they can use it. For everyone else though, let's do the
sane thing and output html like the standard 'build_sphinx' plugin.
[1] https://github.com/openstack-dev/pbr/commit/5b8b7f1d
[2] https://bugs.launchpad.net/pbr/+bug/1681983
[3] https://bugs.launchpad.net/oslotest/+bug/1379998
Change-Id: I579134a2b7980669180c1666503b848835cc2957
Closes-Bug: #1681983
pbr currently hard-codes the list of warnings that are to be ignored.
Many OpenStack projects use remote images to add project "badges" based
on tags defined in the governance repository. Ignore the warning caused
by using remote images so we can unbreak documentation builds using
those badges.
Change-Id: If47e3ca6519cc9f70d62cd887707321fe9199f81
Addresses-Bug: #1682467
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
This change adds new handling when building a Changelog file
for specific characters that cause documentation building
warnings/errors to be emitted when sphinx tries to generate
a Changelog html page. The changes include:
- Escaping any '*' in a commit, which sphinx will interpret as the
start of a new line and throw a warning.
- Escaping any '_' in a commit, which in certain cases, sphinx will
interpret as an invalid link and create an error.
- Escaping any '`' in a commit, which in certain cases will
cause sphinx to interpet as a literal, and throw a warning.
After this change, any entries in the changelog that contain
the above "invalid" syntax no longer generate sphinx
warnings/errors and the offending entries now generate correctly.
Change-Id: I672ef4c56486e59a384849a4b182d11129726ae9
Currently sphinx config is initialized using sphinx.config,
however in recent versions of Sphinx, plugin specific parameters
as man_pages for man builder has been moved to the extension
and is not initialized from sphinx.config but using sphinx.application.
This is making man_pages to be empty when using sphinx 1.5 and man
builder is not properly called.
This patch initializes sphinx config using sphinx.application which
works fine with both old and new Sphinx versions.
Closes-Bug: #1674795
Depends-On: I7bde8fc1f2a7db5bd73635aa197377bf5ac614d2
Change-Id: Ib7c1a6fe8fbb5acfcfcfac61d0b53f080ff2b1e4
When using --coverage, also generate a machine-readable XML coverage
report. This is useful to build tooling around automated tracking of
coverage results.
Change-Id: Idd54ecc627896cc5eab4903658f10a344bdb1778
This legacy option provided the ability to fail on doc warnings.
However, this functionality is broken in recent releases and now exists
in Sphinx itself (since 1.5.0). Rather that fixing it and causing a
whole load of doc build errors introduced in the time since this option
was broken, remove it, preferring the new Sphinx option instead.
This allows us to remove a lot of test code which is essentially testing
Sphinx functionality only now, based on the assumption that Sphinx do
adequate testing themselves.
Change-Id: Ia4b6adefcd437cb1ceb4558b004c17359df2486d
A "testr" entry-point was added to PBR in
I2227adf3643ffe6bcece0c7d2127d23e20d52f7a in Jun 2013. However,
testrepository had already started providing this same
disutils.commands entry-point in Jan 2013 [1].
I am not sure if it is deterministic who wins when two packages
declare the same entry-point, but at least for me, running "python
setup.py testr ..." was not calling into any of the pbr hooks. I
discovered this because I wanted to use the --coverage-package-name
argument provided by the pbr testr wrapper because to s/-/_ in dib.
The original change (I8e4bc9bb78be37b4d13f8d6c2edfe2d67554ad78)
overrides the setuptools provided "test" [2] to call testrepository
when it sees a .testr.conf file (falling back to nose for
compatability). So it seems the intention was not to add a "testr"
command, but that "test" is overridden to run testr automatically when
available.
Therefore I think that testrepository owns the distutils.command
entry-point for "testr". To avoid confusion we should remove the
duplicate entry-point here and just leave the "test" command
overrides.
I've added documentation on the test command. The testr/test split is
quite confusing, being only one letter different, so that is called
out explicitly. Comments in testr_command are updated to reflect
what's happening better. This also converts the calls in tox.ini to
use the PBR wrapper "test".
[1] 2a36806c69 (diff-2eeaed663bd0d25b7e608891384b7298R96)
[2] https://github.com/pypa/setuptools/blob/master/setuptools/command/test.py
Change-Id: I5b68f03e782fef5c45424c04cc851be5988f9019
The revert in Ia6cfbfe5b10a5b714fbb9f21ca61380aaf231638 actually
broke Sphinx 1.3.x support again. Try to fix it for real this
time by avoiding an exception on missing man_pages.
NOTE(dmllr): don't change dict while iterating over it, hopefully
this fixes the gating failure with python 3.5.x
Change-Id: I52d45fa0a0d42de690d3a492068f7bb03483a224
Related-Bug: 1379998
This patch set modifies lines which are importing objects
instead of modules. As per openstack import guide lines, user should
import modules in a file not objects.
http://docs.openstack.org/developer/hacking/#imports
Change-Id: Iab667c04f4aa57d0c8c882144a550443dacc8f05
As per OpenStack Docstrings guide lines [1]:
[H401] Docstrings should not start with a space.
[H403] Multi line docstrings should end on a new line.
[H404] Multi line docstrings should start without a leading new line.
[H405] Multi line docstrings should start with a one line summary
followed by an empty line.
[1] http://docs.openstack.org/developer/hacking/#docstrings
Change-Id: Ic943e837b8b4f55c2957643ad29dfa004b3c2eb3
The --coverage-package-name allows to specify
multiple packages. This change updates the doc to
reflect that.
Change-Id: Id9cfedf699c3e15b7e1c7b20531488682775462a
tox -e py34 is reporting a deprecation warning for SafeConfigParser
/octavia/.tox/py34/lib/python3.4/site-packages/pbr/util.py:207:
DeprecationWarning: The SafeConfigParser class has been renamed to
ConfigParser in Python 3.2. This alias will be removed in future
versions. Use ConfigParser directly instead.
parser = configparser.SafeConfigParser()
Closes-Bug: #1618666
Change-Id: Ib280b778938b64717ee1cf94efae2f7b553c8f5e
When setup.cfg has a different value than pbr.version.VersionInfo(),
then the version call is unable to look up the right value in the
pkg_resources metadata. However, nothing in the current error message
communicates this to the user, which leads to them not being able to
look in all the right places for the possible error.
Change-Id: I31c4e0e2ed15986ba8274a0c5270a9d214ac4b48
We're currently ignoring data-files, and it looks like the problem
would sometimes be present starting with 2796f9, 0.5.7, and always be present
from 04984a, 0.5.15.
This normalises all config keys from - to _ as soon as we read the
config, which means future access and modification does not need to
concern itself with the possibility of the key being a '-' instead '_'.
This should make it more difficult for code accessing/modifying values
in the config to clobber user set values or be unaware of them, like
in the case of the files hook.
As well, support download-url, but properly expose it as download_url.
Co-Authored-By: Julien Danjou <julien@danjou.info>
Change-Id: I062774c706b8f7339dda46689a226b80ae6ac277
Some of tests use different method of assertTrue(isinstance(A, B))
or assertEqual(type(A), B).
The correct way is to use assertIsInstance(A, B) provided by testtools.
Change-Id: I6c7aa04a3a3aaaab7f428cb86549a4d043af802f
Older versions of git produce log output (using --format specifiers)
that doesn't prefix tags with a "tag:" prefix, though they do using
the --decorate log format. This was a bug in git that was fixed, but
the older git is still in the wild. This change combines both the new
--format specifier method with the --decorate method to find tags even
if used with the old, buggy git, while (hopefully) avoiding
identifying non-tags as tags.
This is based on I6b557687f2428a528a8aa7f76e84797162488887 but tries to
use a single format string that works across all versions of git.
Because the format string has changed, some of the test data has had to
change to reflect the new format.
Change-Id: I58a1b12e0f4054fd25bddc77fabb6c89239561ff
Closes-Bug: #1570458
Co-Authored-By: James Polley <jp@jamezpolley.com>
It seems like we only (currently) exposed to rpm
version translation command when the deb version
translation command is equally useful (for those
that package deb things).
Change-Id: I0df175e5206d9d3a806bf33c486765ad1aa8aa6b
Xenial PyPy adds multiarch to the compiled c library, so if PyPy reports
multiarch we add it to the soabi string.
If the installed PyPy is not configured for multiarch, the multiarch
config variable will not exist and `arch` will be None, in which case we
should only update the string if multiarch exists.
Change-Id: Idcd3af4878d5b66c6770f96e7235a7d2f848df81
assertEqual expects that the arguments provided to it should be
(expected, observed). If a particluar order is kept as a convention,
then it helps to provide a cleaner message to the developer if Unit
Tests fail. The following patch fixes this issue.
TrivialFix
Closes-Bug: #1259292
Change-Id: I03c414a763b2d51210aaec362405912f29850e36
I noticed the current API documentation is unreadable [1]. Remove the
inbuilt theme and move to oslosphinx for module/api documentation.
[1] http://docs.openstack.org/developer/pbr/api/pbr.version.html
Change-Id: I8d92d849b8a2ab4baca960ae8c271214d3c2b8f4
Pass the flag to tell Sphinx to treat warnings as errors.
Add compatability note for versions 1.4.0 and suppress
re-register warnings on subsequent runs.
Change-Id: I680b448471e687919d202e8f2abe57f8ba3b22ee
Closes-Bug: #1496882
Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
When we were reviewing all the tarballs in the release
team, we were comparing the generated tar.gz from sdist
and what was actually uploaded, we saw that the pbr.json
contents keeps changing but with the same key/value pairs.
So let's try to stick to a specific order by using
sort_keys when we are converting json into a string
Change-Id: I82186522240599a240737e2c87ad524d0ed8bd28
The script generated for wsgi-script entrypoint adds a argument for
listening on a specific port using argparse. Unfortunately it's not
compatible with oslo.config handling of config options, as used by
keystone for example. This fixes the situation by only parsing known
options.
Change-Id: I37f82e8d78a4288323854282da300c123561218a
Closes-Bug: #1501756
Some odd version strings can cause SemanticVersion.from_pip_string()
to raise IndexError. This change converts IndexError to ValueError.
Change-Id: Ic3046817b6c5808c61c4a3bc3d912501e6a52274
Closes-Bug: #1570145
This test was relying on the short sleep allowing the subprocess
time to write to an external file. Rather than write to an
external file, have the output go to a pipe that the test process
can read so that the test doesn't have to guess how long to sleep.
Change-Id: I128a9dc5c7525d941f6e00c0073d983eab1e44e6
email.MessageError exception does not exist, so if an error occur, the
code block will fail completely. Fix the path to the correct class.
Co-Authored-By: Ian Cordasco <graffatcolmingov@gmail.com>
Change-Id: Ic4e69405ca6346191da66a86f0b7aa5b3b75bed6
Currently, when the sem-ver processing is searching the git history,
it stores the canonical version and uses that as the tag, which is
fine if the project uses the canonical version as a tag. Instead,
return the original tag so that history can be correctly checked.
Change-Id: I1b83fc3dc3dd287ae3b23fd4cba07fdfdd673a7f
Closes-Bug: 1561254
Downstream consumers, such as OpenStack Ansible, generate wheels for all
packages (including services). More and more services are moving to use
the wsgi_scripts entry-points provided and handled by pbr. Unfortunately,
these scripts are not generated during wheel creation unless we force
them to be generated because Setuptools and Distutils will only generate
console_scripts entry-points.
This also fixes the C extension on Python 3 because it was previously
broken.
Change-Id: Icecc8474028436e8b2fb752d576204d9439fb0e7
Closes-bug: #1542383
We are silently dropping requirements that do not parse, do not do this
and bubble the error up.
Change-Id: I5bfce7c7d0be2ccd880e78509856074a09c57166
Closes-Bug: 1554331
The same variable name was being used for three argument group
objects. Replace some occurrences with more properly named variables.
Change-Id: Ibbec94d03f14fae6bed86c9479671d41b775c057
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Setuptools 0.6c11 is ancient, and doesn't install on Py3, so don't
attempt to do so. We don't test Py3 in the gate with integration, so
this is a noop there, however this means if we do so in the future
this won't bubble up as a failure. It also allows me to run this block
of integration tests with Py3 in addtion to Py27 locally.
Change-Id: I1dc6eb7591be847573231ca1056c2cd31b2f673a
In patch 789888 we began relying on EntryPoint.resolve(), which only
exists in setuptools >= 12, breaking support for setuptools < 12. This
includes the default virtualenv installations from may LTS distributions.
As ep.load with arguments is deprecated we will attempt to use
ep.resolve if it exists and fall back to ep.load(false) if it does not.
setuptools 11 is the odd release out, as this was when load with
arguments was deprecated, suggesting the use of ep._load. We will
continue to use ep.load(False) here.
Change-Id: I272f76fef7e447378697a3980372191f4ffe1a2f
The logic that splits the git log lines to find tags gets confused by
parentheses in the commit message and truncates the line at the first
one. To mitigate this, have git emit a null character and split on that.
As a consequence of doing and testing this, fix the bug where
non-semantic-version tags in the tag history would show up in the
ChangeLog.
Change-Id: I5e522d6d374534f2921aaded42780df97c3d1987
Old versions of pip do not properly evaluate blank envmarkers such as
':python_version>=2.7', so as long as we are not creating a bdist_wheel
we evaluate these immediately to support these old installations
Change-Id: I94a5f9bccd658a2529a727d99f5a2b79e6b00aa8
Closes-Bug: 1502692
Enabled H405 multi line docstring summary not separated with an empty
line in tox testing.
Change-Id: I1a4bfbd2bad6eb2c132843ca21451807737121df
Closes-Bug:#1524176
Distribution().get_command_list() is not hooked by setuptools like
print_commands() is, as such we do not get a complete command list.
Here we perform the hook ourselves to prevent the need to depend on
a super new version of setuptools.
Change-Id: Ie6088aedb92713be4b8b9af4a4ce3d300a114d7b
Although https://review.openstack.org/#/c/171309/
says Sphinx >=1.3, the actual implementation does not
support Sphinx >=1.3b1 and 1.3.1.
I have seen Sphinx release notes and source codes for
those versions, and found that up to 1.2.3 used
init_values(self) and Sphinx from 1.3b1 used
init_values(self, warn).
So to change the code from >=1.3.1 to >1.2.3, it can
more support Sphinx >=1.3b1 and <1.3.1.
Change-Id: Ia73be8111d3af7f72aaf640e4c7c519a5d0b08c5
Partial-Bug: #1379998
Related-Bug: #1496882
We have removed them from the d-g PROJECTS list so special casing in
PBR's tests is no longer necessary.
Change-Id: I6c4a587a21a81cb2dea08d2c13ecba033b1ebee0
Depends-On: I61c2f3f8a724dc789479278e7ba493e095c52305
This can arise if pbr has been installed indirectly (for instance,
because a package depends on mock which requires pbr) - pbr will be
running, but the cmd.distribution object has no pbr attribute. In this
case, pbr code should not run.
Change-Id: Ib7db0c8ab78e3cb700671f6a123ec603b4dbfdbe
Closes-bug: 1493735
Co-Authored-By: Sachi King <nakato@nakato.io>
We need to write some new targeted venv using tests. This fixture will
make that directly accessible without cruft. Also fixup a global-state
polluting hack in test_integration I noticed at the same time.
Change-Id: I247de9236919cb7a8b1a3257a357b4a4bac37dfd
When processing setup.cfg strip trailing comments from lines as this is
unsupported by setuptools.
Change-Id: I7712c07552b50830549644a6ff4167b90d897125
Closes-bug: 1494330
When configured as an Apache WSGI module a race condition
is possible during keystone cache initialization.
The operation raises exception region.RegionAlreadyConfigured.
This is a result of the race condition involving global
'application' variable being initialized several times
(1 per thread).
Apache modwsgi documentation suggests protecting global objects
with thread locks.
Change-Id: Ib9e2207b0d1d9cee656736e94865fb404b6a868d
Related-Bug: 1482271
PBR is currently creating a pbr.json for any install originating from a
git tree as pbr_json is an entry point for egg_info.writers. If pbr is
not set to True in setup() we now exit before doing any work.
Change-Id: Ia37e37c22c2396f6a10aa2075ea8c501e571fee3
Closes-Bug: 1483067
Since we don't strip comments from end of the requirements like:
requests-kerberos>=0.6;python_version=='2.7' # MIT
The currnet code ends up failing with a:
"Invalid environment marker"
We should strip the comments out before processing.
Closes-Bug: #1487835
Change-Id: I75d2e83a5f60f93e1c38d53fb4f0a29ba35120b1
Sphinx >= 1.3 changes:
- Prototypes for init_values() requires a handle to warnings.warn
- New warning for setting the html_theme in conf.py to 'default':
sphinx.errors.SphinxWarning: WARNING: 'default' html theme has been
renamed to 'classic'. Please change your html_theme setting either
to the new 'alabaster' default theme, or to 'classic' to keep using
the old default.
This change deals with the above sphinx updates by adding the handle to
init_values() and setting pbr to _not_ fail on warnings when building the
docs.
Change-Id: Iba92628263e20efca84aeada0e19000f4c9b1fac
When we try to find the git directory, if git is not present, treat
that the same as an error from git itself. Sadly tests for this are
nearly impossible to meaningfully write, since we always have git
installed. I can do a mock based test if folk want one.
Change-Id: If6160d1fb3def8133bdd0b66105e60ef93f80f82
Closes-Bug: #1481468
git._iter_log_online() returns None in some cases for instance
when SKIP_WRITE_GIT_CHANGELOG is set to true. This is fixed by
returning an empty list in _iter_log_online.
Also the check if a changelog based on git should be written
was moved from git._iter_log_online to git.write_git_changelog
which makes more sense since some functions were calling
git._iter_log_online to get the changelog for other reasons than
writing it.
Additionally a unittest was added to check that setting the
environment variable SKIP_WRITE_GIT_CHANGELOG to true does not
break anything when retrieving the git version.
Change-Id: Ib12df23ab25b290dd394f9cb1456b8d5da57306a
Closes-Bug: 1467440
readthedocs uses 'setup.py install' to prepare trees for doc creation,
but ChangeLog is not currently created there, and doing so would be
nice. This won't affect develop invocations AFAICT, and even if it
did, the overheads are ~10% of the time to run 0 tests in Nova today -
e.g. quite tolerable.
Change-Id: I7bc18fc9ca2dbe852598cc79b2ad6273fc53557d
As we attempt to get more of OpenStack API servers to be runnable as a
regular WSGI application, we should provide support in PBR for
building these base scripts just like the console scripts.
This adds a new 'wsgi_scripts' group which builds a base script that
will run under mod_wsgi as expected. It also has a CLI fallback mode,
so that the application can be brought up as a wsgiref simple_server
for quick local testing and development. All wsgiref servers default
to binding to port 8000, but that can be overridden.
To support this, and possible future expansion of the script types,
the group list now iterates over a dictionary of group_name =>
template mappings.
This includes basic testing.
It also includes tests which would run an actual wsgi
environment. These are currently skipped as they can't be reliably run
in the gate for timing reasons on stdout processing.
Change-Id: I334639d7ecaad2703d1ff675880a314cc28e2334
If there are boolean conditions in the env marker, the existing join
causes the extra tag to only be applied to the first of them, causing
carnage.
Change-Id: I7b735befae76b330f923276e1c48ef23816774d4
Intended usage, getting a rpm *compatible* version number from pbr
from a pbr using python package. Typically this will then get
put into a rpm spec file (what is used to build an rpm) and used to
build an rpm using `rpmbuild` (the thing used to translate rpm spec
files into rpms).
For example: this allows [1][2] (which I am the primary maintainer of
and godaddy, cray, and y! are users of) to get out of the whacky
and/or flakey rpm version number calculation/conversion process (which
previously just worked around by saying only use tags).
[1] https://github.com/stackforge/anvil
[2] https://anvil.readthedocs.org/en/latest/topics/summary.html
Change-Id: I26cd1d6e8aef69d52e8a4f36bd264c690e712ba3
Setting env variables with os.environ needs a string as value.
This fixes:
TypeError: must be string, not int
Closes-Bug: #1467880
Change-Id: Ib0c8bbf24f9d4f8926ad71773eb57c3ff41cbec2
Setuptools doesn't [yet] support markeres in tests_require parameters
to setup(). While we need to get that fixed, we also need to support
syncing marker constrained requirements from global-requirements.
Since version constraints are needed to successfully port to Python3
this won't be a long term viable solution: but few of our test-only
dependencies require version constraints, so we can do this in the
interim. The ones that do - such as python-mysql - we're either
moving away from, or centralising them into optional dependencies
on other packages where the tests_require limitation shouldn't apply.
Change-Id: I31adaf35c8d7b72fe3f8c9242cc356fe34d537e8
Jenkins
Gage Hugo
Stephen Finucane
Doug Hellmann
Clark Boylan
Alfredo Moralejo
Ian Wienand
Dirk Mueller
Cao Xuan Hoang
Thomas Morin
jiansong
Monty Taylor
Sachi King
zhangyanxian
Randall Nortman
Joshua Harlow
dineshbhor
weiweigu
Davanum Srinivas
Thomas Herve
Brant Knudson
Julien Danjou
Steve Kowalik
Ian Cordasco
Luo Gangyi
Ryan Bourgeois
Ian Y. Choi
Joshua Harlow
James Polley
Robert Collins
Alexander Makarov
Timothy Chavez
Florian Wilhelm
Sean Dague
Thomas Bechtold