Compare commits

...

138 Commits

Author SHA1 Message Date
Sean Mooney 5d1f322542 remove inspect.getargspec deprecation warning
In python 3 inspect.getargspec is deprecated and
replaced with inspect.getfullargspec which does not
exist on python 2.7. This change uses six to select
the correct version to use based on the python version
used.

Change-Id: I234a3509ff850d0c5616ebcfa240212b03db9e76
Closes-Bug: #1814288
2019-09-17 02:05:19 +01:00
Mike Bayer 971b9e62df Use engine.connect(); don't use private _run_visitor method
SQLAlchemy 1.3 has deprecated engine.contextual_connect() which
will be removed in 1.4.   Additionally, private methods like
Engine._run_visitor will also be removed.

Change-Id: I319785d7dd83ffe2c6e651a2494b073becc84684
2019-07-16 09:39:49 -04:00
Matt Riedemann 67d2bb2616 Claim support for python 3.6
This adds the voting and gating openstack-tox-py36 job
and adds python 3.6 to the classifiers list.

Change-Id: Ida2349de534399103e5d6f294a7e771910188ee4
2019-07-10 14:15:58 -04:00
Matt Riedemann 578f533cb8 Claim support for python 3.5
This makes the openstack-tox-py35 job voting and gating
and adds python 3.5 to the classifiers list.

Change-Id: I11c8cd0e33e0609d740f5742096438a2395b28ed
2019-07-10 14:15:49 -04:00
Matt Riedemann 1cc62d6ea2 Remove test-requirements-py*.txt files
We can control python version sensitive dependencies
in the single test-requirements.txt file so we can
drop the version-specific files. This should also make
the openstack-tox-py35 job start passing.

Change-Id: Ic3fa2c4417092e387ac5d177be75a22050a6f857
2019-07-10 14:15:39 -04:00
Matt Riedemann 3ee28fe67b Add bindep support
This adds a minimal bindep.txt to install mysql and postgresql
dependencies (copied from nova's bindep.txt). This is needed
now to get python jobs (pep8,py27,etc) working in the gate.

Change-Id: I4b7bccdb7cebd95753e8cd931846213f400aa652
Closes-Bug: #1836087
2019-07-10 12:14:20 -04:00
Matt Riedemann 078ffc62da Remove py26 tox targets
Support for python 2.6 was dropped in commit
8266c8dec0 but the
tox targets were missed, so they are removed here.

Change-Id: I2271717ce4a35183d2b773063564cbaf774fdfb0
2019-05-15 18:33:43 -04:00
Zuul 421e860222 Merge "Import zuul jobs" 2019-05-15 22:27:31 +00:00
Zuul 3239c42b69 Merge "Use mysqlclient" 2019-05-15 20:11:24 +00:00
Andreas Jaeger e922c240c5 Import zuul jobs
Add the project-specific tox py27sa07 job to this repo, convert it
from legacy-sqlalchemy-migrate-tox-py27sa07 to Zuul v3.

Also, import other jobs from project-config so that the jobs
are managed by the team in-tree.

Change-Id: I9296b3e40246ab89655411b660c5ba45bc9cf2ff
2019-05-15 21:30:05 +02:00
Zuul e85f8ba65a Merge "Fix docs build" 2019-05-15 19:08:48 +00:00
Andreas Jaeger abb234e1f2 Fix docs build
Fixes for docs build:
* Use doc/requirements to follow changes in OpenStack CI for the docs
  building, add a tox environment for this.
* Remove reference to "project's download page", the link was removed in
  change I2db594d279e1229e5b1600cecad86fe0c3612115 and now we have a
  dead link that errors building if warnings are treated as errors.
* Change I2db594d279e1229e5b1600cecad86fe0c3612115 added a column for
  DB2 header but not for any entries, enhance table with extra column.
  This fixes building of the table if warnings are treated as errors.
* Update to sphinx 1.6.x since that is what is used in OpenStack CI job,
  remove issuetracker, it does not work with sphinx 1.6 anymore.
* Disable html_static_path - the content does not exist
* Use python3 for docs building

Change-Id: I76ee4d6dc45f9b04115f093951ae8b745f3ac026
2019-05-15 10:49:02 +02:00
OpenDev Sysadmins 8c90555e9c OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:50:22 +00:00
Matt Riedemann 8acab2cd75 Change title in README.rst
This is needed to be able to release to pypi:

$ python3 setup.py check --restructuredtext --strict
running check
warning: check: Duplicate implicit target name: "sqlalchemy-migrate".

Change-Id: I26dd765a5273199d6666dd9ed618dc8189a2e8ed
2019-01-22 09:02:49 -05:00
Zuul f0664995a1 Merge "Import MutableMapping from the correct Python module" 2019-01-21 22:17:29 +00:00
Zuul 6803334bb1 Merge "Don't use deprecated / non-functional "force" parameter" 2019-01-21 22:13:03 +00:00
Mike Bayer fe64667106 Don't use deprecated / non-functional "force" parameter
The "force" parameter in SQLAlchemy IdentifierPreparer.quote()
has been a no-op since 0.9 in
031ef08078,
which was six years ago.   In SQLAlchemy 1.3 this parameter
will be removed entirely.   Bump requirements to 0.9 series
here and remove usage of the "force" flag.

Change-Id: I4492df2e7d2075fefbf13d6782de11f7d402f6b8
2019-01-18 13:11:40 -05:00
Mike Bayer 851bd217a0 Use mysqlclient
The test requirement for MySQL-Python refers to a package that
has had no releases since 2014 and no longer builds on some MySQL /
MariaDB variants.  MySQL-Python has for years been superseded by
the mysqlclient fork which is actively maintained.   mysqlclient
may at some point be allowed to take over the MySQL-Python name
as per PEP-541 however this hasn't happened yet.

Change-Id: I67dd2338dc74e59c2deb2f1bd1b848e3c63e10e8
2019-01-16 16:08:25 -05:00
Corey Bryant 231a4d2ae9 Use legacy_alter_table ON in sqlite recreate_table
Use "PRAGMA legacy_alter_table = ON" with sqlite >= 3.26 when
using "ALTER TABLE RENAME TO migration_tmp" to maintain legacy
behavior.

As of sqlite version 3.26, when a table is renamed using
"ALTER TABLE RENAME TO", REFERENCES clauses that refer to the
table will be updated. To maintain legacy (3.24 and earlier)
behavior, "PRAGMA legacy_alter_table" can be set to true and
"PRAGMA foreign_keys" can be set to false. [1]

[1] https://www.sqlite.org/src/info/ae9638e9c0ad0c36

Thanks to "László Böszörményi (GCS)" <gcs@debian.org> for
providing the code for this patch, which has since been
slightly modified.

Change-Id: I539988ab2ad6df6c8f423ecec15364ad8fcc7267
Closes-Bug: 1807262
2019-01-15 22:01:26 +00:00
Zuul 8fd7226f18 Merge "Add .eggs in .gitignore" 2019-01-15 20:14:08 +00:00
Zuul fb138f6de7 Merge "Remove py26 support" 2019-01-15 20:14:08 +00:00
Anusree 8266c8dec0 Remove py26 support
As of mitaka, the infra team won't have the resources available to
reasonably test py26, also the oslo team is dropping py26 support
from their libraries. sine we rely on oslo for a lot of our work,
and depend on infra for our CI, we should drop py26 support too.

Change-Id: I6af3716f5daf73febdabcd79853a09512428c289
Closes-Bug: 1519510
2019-01-15 18:25:36 +00:00
Nicola Soranzo d45ea279b9 Add .eggs in .gitignore
Change-Id: Ia7738e03aff843b1b120c0d5d13a7b64f2f33201
2019-01-15 18:24:57 +00:00
Chih-Hsuan Yen a00dab7bcf Import MutableMapping from the correct Python module
Change-Id: Ifb66fe22bc607b13f5c4756d3b93f5e8206c33e3
2019-01-15 18:16:06 +00:00
Jonathan Herlin 9f0bda970d Update mailinglist from dev to discuss
openstack-dev was decomissioned this night in https://review.openstack.org/621258
Update openstack-dev to openstack-discuss

Change-Id: Ic84b2422d37aa1ebb443fff4152c7ebe5a098566
2019-01-15 17:38:52 +00:00
Chih-Hsuan Yen f44d07956a Get rid of psycopg2 warnings by disabling wheels
The psycopg2 wheel package warnings are causing some tests
to fail which expect there to be no stderr output. This fixes
the problem by not using the wheel binary for that package.

Closes-Bug: #1811876

Change-Id: Id43e74d8d343ab4e80d1d246543bada1ed4d06ad
2019-01-15 17:35:30 +00:00
Haikel Guemar 02c26a2ced Enforce that pbr used is >= 1.8
It otherwise fails if used against older pbr (e.g distro packaging build)

Change-Id: I19dbd5d14a9135408ad21a34834f0bd1fb3ea55d
2017-04-07 15:59:23 +02:00
Jenkins 9a7a37fba4 Merge "Fix spelling mistake" 2017-03-24 21:23:38 +00:00
Tony Breeds ca6f3a93b5 Use a modern PBR package
The 2.0.0 is breaking in that it removes the use of warnerrors in
build_sphinux.

sqlalchemy-migrate isn't using that feature, so it shoudln't break.

The cap on pbr is preventing other OpenStack projects that would like to
use pbr 2.0.0 (and sphinx 1.5.1) from doing so as it breaks
co-installability with sqlalchemy-migrate

Change-Id: I1c907201c717fe42caca24831985a119f2a1738b
Related-Bug: 1668848
2017-03-01 16:29:33 +11:00
Jenkins b45033de73 Merge "Prepare for using standard python tests" 2017-02-09 18:06:50 +00:00
Andreas Jaeger 7fc88b35e8 Prepare for using standard python tests
Add simple script to setup mysql and postgresql databases, this script
can be run by users during testing and will be run by CI systems for
specific setup before running unit tests.

This allows to change in project-config the python-db jobs to
python-jobs since python-jobs will call this script initially.

See also
http://lists.openstack.org/pipermail/openstack-dev/2016-November/107784.html

Needed-By: I46d883ce1b25338d01a2b0e2b071b15adab00520
Change-Id: I8c5015616a5a12501edd16932d4148930c01a06b
2017-02-09 06:49:29 +01:00
Jenkins e1a8ec708a Merge "Update .gitreview for new namespace" 2017-02-01 21:00:20 +00:00
dineshbhor 09ec8f7816 Fix spelling mistake
TrivialFix

Change-Id: I089d9e8b57895d9878bf82d2fac397722fccc083
2016-10-26 19:57:24 +05:30
Mike Bayer e9175a37ce Set autoincrement to False when modifying to non-Integer datatype
Starting in SQLAlchemy 1.1, the rules for when "autoincrement=True"
may be set on a column are more strict.  The migrate tests are
testing the alteration of a column from Integer to String
and then regenerating; this means we need to set autoincrement
to False as well.   A related issue in SQLAlchemy 1.1 is
also being fixed (see https://bitbucket.org/zzzeek/sqlalchemy/issues/3835/),
however this fix is not needed in order for the tests to pass here.

Change-Id: Ibd3a75fff13312411df87e17b6e5764865d69728
2016-10-20 17:47:19 -04:00
Jenkins d58469a6ae Merge "Raise VersionNotFoundError instead of KeyError" 2016-06-03 13:45:36 +00:00
dineshbhor 2a32681036 Raise VersionNotFoundError instead of KeyError
Currently migrate.versioning.api.upgrade() raises KeyError
instead of sqlalchemy-migrate specific exception if migration
script file is not present in migration repository.

Raised migrate.exception.VersionNotFoundError if the specified
migration script does not exist in the repository. Made
VersionNotFoundError exception class as a subclass of KeyError
in order to avoid breaking existing users looking for KeyError.

Related-Bug: #1546441
Change-Id: I0210d56a6e85f03c44cea027f50863faaf050c1d
2016-05-27 11:57:24 +05:30
dineshbhor 9356e5e28a Fix DeprecationWarning on setuptools >= 11.3
Python 3.4 unit test is failing because of
DeprecationWarning: Parameters to load are deprecated.
Called .resolve and .require separately on setuptools >= 11.3.

Made provision to call .resolve() method if setuptools >= 10.2
and less than 11.3 else call .load() method.

Change-Id: I5ba80edfbf6b7c8399c66f01d57c91bd02eab274
Closes-Bug: #1586060
2016-05-26 20:12:25 +05:30
Jeremy Stanley 00eb42318b Update .gitreview for new namespace
Change-Id: Ie13030d01b439115f0e8f0e8b8ea1844d99644ee
2015-10-17 22:38:53 +00:00
Jenkins fe3e08ae0b Merge "Update URLs in documentation" 2015-08-29 14:27:46 +00:00
Victor Stinner 5cf4071e37 Update URLs in documentation
* online doc moved from http://packages.python.org/sqlalchemy-migrate/
  to https://sqlalchemy-migrate.readthedocs.org/
* source code moved from http://code.google.com/p/sqlalchemy-migrate/
  to https://github.com/stackforge/sqlalchemy-migrate
* bug tracker moved from
  http://code.google.com/p/sqlalchemy-migrate/issues/list to
  https://bugs.launchpad.net/sqlalchemy-migrate

Change-Id: I2db594d279e1229e5b1600cecad86fe0c3612115
2015-08-29 06:33:47 -07:00
Jenkins 3c7ac7559c Merge "Add VerNum.__index__() for Python 3 support" 2015-07-31 02:11:27 +00:00
Victor Stinner 1384e901b0 Add VerNum.__index__() for Python 3 support
On Python 3, some functions like range() don't try to call the __int__()
method to cast an object to integer, but try instead the __index__()
method.

Add an __index__() method to mimick correctly the int type on Python 3.

Change-Id: I8df116d80e201778714a59367600eaef644266ed
2015-07-28 12:56:52 +02:00
Thomas Goirand fb55b01a9a Fixes usage function for Py3
The usage function of migrate_repository.py isn't Python 3 compatible,
and this hasn't be caught by unit tests. This patch fixes the function,
so at least the file can be compiled in Py3.

Change-Id: Ib9333e46e7526e82acde573d4b2046b2bf9a7ae0
2015-07-22 18:03:34 +00:00
Matt Riedemann 8252703f56 Unblock migrate (py26 and py3* testing issues)
There are two changes which have to go together to pass the gate
tests:

1. Update pbr and mock requirements from global-requirements

mock 1.2 supports py26 again so make that the minimum version. The
same change is being made in g-r with:

Ic6b9e18eaec9c81bbbbc57129e024904be928e09

Sync up with latest pbr in global-requirements while we're at it.

Closes-Bug: #1474925

2. Fix the importpath module to work with python >= 3.3 where the
__import__ built-in is raising an ImportError on a temporary file
that is added to the system path.

Closes-Bug: #1475339

Change-Id: Ie98938ba75f3983094dd540b7d26a7ec46be4f6e
2015-07-18 10:39:35 -07:00
Matt Riedemann 050b646e86 Revert "Revert "uncap pbr and sqla requirements""
This reverts commit e4d0e5be8d

Now that flake8-related dependencies have been updated in the
dependent change the pep8 job is fixed.

Change-Id: Idfa6a18836d7ce02dfaa5d9da1a51c98ad987f51
2015-07-05 06:47:03 -07:00
Matt Riedemann dc07f8de8f Update flake8 related dependencies
In order to raise the cap on pbr we need to update
the dependent versions of the flake8 related packages
for the pep8 job since they have capped pbr.

A couple of simple hacking issues are fixed, the rest
are ignored.

Change-Id: Icddb5bf284da7b6463ebcfc7512726149ffe6085
2015-07-05 06:44:03 -07:00
Jenkins d4e6d892ae Merge "Add Python 3 classifiers" 2015-07-04 16:46:48 +00:00
Matt Riedemann e4d0e5be8d Revert "uncap pbr and sqla requirements"
This reverts commit 35832555c5

The non-voting pep8 job was failing due a VersionConflict
with pbr, so this shouldn't have been merged.

Change-Id: I4917b92121cac524fd89575f30d72d7319cbe20c
2015-07-04 14:12:55 +00:00
Sean Dague 35832555c5 uncap pbr and sqla requirements
The cap of pbr causes issues now that pbr 1.2 is basically needed for
a lot of projects to do the right thing with requirements ranges. The
sqla capping is preventing new versions of sqla to be used in the
OpenStack gates, and shouldn't be capped by a library.

Change-Id: I5fc142eb8c9d616db2ed9b2f3e4e4d1147e131ff
2015-07-02 14:07:37 -07:00
Mike Bayer a94dae7a01 Update tests and reqs for SQLA 1.0
Lift the requirements to support SQLAlchemy 1.0.  Two tests
were calling upon revised APIs and required adjustment.

Change-Id: Ic91a91bb3c915027b522eace302f2ed074233294
2015-07-01 20:01:52 +00:00
Mike Bayer b8def7cbfb Ignore stderr output when invoking migrate script in tests
Under Python 2.6 a setuptools warning is produced when
the migrate runner runs.  Since migrate invokes its own
commandline client from tests in a separate shell, the
fixture we're using to do that must be told not to complain
about this stderr.

Change-Id: Ib5823754d6ffabe954665f2a7529ed0e56591ebf
2015-07-01 11:27:15 -04:00
Victor Stinner d651f123b9 Add Python 3 classifiers
sqlachemy-migrate works on Python 3.3 and Python 3.4: announce it in
classifiers in setup.cfg.

Change-Id: Ibb2a803941dcc7b05efd407692b3627203ba2afe
2015-04-17 09:48:17 +02:00
Qin Zhao e57ee4c3a4 Fix ibmdb2 index name handling
The ibmdb2 code calls _index_identifier() when it handles index name. This
method only exists from sqlalchemy 0.6.5 to 0.7.*. Nova code change
https://review.openstack.org/#/c/153123/ attempts to drop a db constraint and
it fails to sync nova db with sqlalchemy 0.9.8 running against db2. Need to let
ibmdb2 code identify sqlalchemy version and call the correct method to handle
index name.

Closes-Bug: 1428477

Change-Id: Ie6333f9cea0209c1ea290356873a1a1bcf409bed
2015-03-16 15:21:03 +00:00
Jenkins 8a638ce9d6 Merge "Don't run the test if _setup() fails" 2015-02-24 23:07:21 +00:00
Jenkins 24521798bc Merge "script: strip comments in SQL statements" 2015-02-24 23:05:33 +00:00
Jenkins bf40e79a6a Merge "Fix .gitignore for .tox and .testrepository" 2015-02-24 19:30:30 +00:00
Roman Podoliaka 5feeaba69f Don't run the test if _setup() fails
Change-Id: I2c89c98961044f0e0a1d9b4c2eeea190c5830eed
2015-02-23 17:03:25 +02:00
Yuval Langer c8c5c4bdd8 Correcting minor typo
Change-Id: Ib8b897414039224971a1535c268c8a21fc3534fb
2015-02-22 14:03:55 +02:00
Matt Riedemann 9d212e6fb9 Fix .gitignore for .tox and .testrepository
The entries for tox and testrepository weren't
working with `git clean -df` so this updates them
to match what's in nova's .gitignore. Testing
manually shows this doesn't blast those directories
when running git clean.

Change-Id: Iba569afabea748369f76351bf5521ca1c0fdce0c
Closes-Bug: #1423983
2015-02-20 10:43:00 -08:00
Matt Riedemann ae64d828df allow dropping fkeys with sqlite
This implements the ability to drop foreign keys
with sqlite. It's basically the same implementation
used for dropping unique constraints so the common
code is refactored.

The existing FKey test that was skipping sqlite is
no longer skipped to show this works.

Change-Id: Idaaf4229e34af4c21c3bcead4b4e22491d24238e
Closes-Bug: #1423955
2015-02-20 10:04:35 -08:00
Mike Bayer 997855db3b Add pretty_tox setup
This changeset adds the pretty_tox script runner to SQLAlchemy-migrate,
so that current test runs can be viewed clearly.

Change-Id: I3884703e24cb636983a0202c46899c772419d401
2015-01-29 15:19:02 -05:00
Jenkins 75034abe51 Merge "Update requirements file matching global requ" 2015-01-27 15:20:40 +00:00
Jenkins 6aca7a17fc Merge "Work toward Python 3.4 support and testing" 2015-01-15 23:04:03 +00:00
Jenkins 0a618361ba Merge "Fixes the auto-generated manage.py" 2015-01-15 22:53:59 +00:00
Jenkins a15a7cec42 Merge "Replace assertNotEquals with assertNotEqual." 2015-01-15 22:18:05 +00:00
Ihar Hrachyshka b9caaae4fc script: strip comments in SQL statements
Regular expression does not match correctly against statements that contain
comments at their start. So strip those comments first (and whitespaces, while
we are at it).

Change-Id: Iad9b544bf995374d76cab1e125658aae2f8511f4
Closes-Bug: #1410494
2015-01-14 01:45:25 +01:00
Monty Taylor b011e6cb39 Remove svn version tag setting
This is confusing our poor friend setuptools into appending a post0 on
to release versions.

Change-Id: I59a763f00f47d9a2fb0d26428a41fee82deff49d
2015-01-12 16:21:02 -08:00
Jenkins 25c7d750f5 Merge "pep8: mark all pep8 checks that currently fail as ignored" 2015-01-11 10:38:02 +00:00
Jenkins 397682b9c8 Merge "Use native sqlalchemy 0.9 quote attribute with ibmdb2" 2015-01-10 00:03:23 +00:00
Ihar Hrachyshka 938757e7aa Ignore transaction management statements in SQL scripts
Now that we don't run SQL script with a single .execute() call,
transaction management statements (BEGIN, COMMIT, END) fail with
operational error. Ignore them if occurred in a script.

All in all, transactions are managed by SQLAlchemy-migrate itself, so
there is no reason for the calling code to pass those statements in a
SQL script. Warn user about the statements that are ignored.

The ideal response to such a warning from library users is removing
those redundant transaction management statements from their scripts.

Note: ROLLBACK is not ignored even though it's one of transaction
management statements because its usage in migration scripts is insane
anyway, and we're better fail on it and not (almost) silently ignore it.

Change-Id: Ie4179c0e9341d42656d66821aaac23f8dcd33927
Closes-bug: 1368391
2015-01-09 14:32:27 -08:00
Rahul Priyadarshi 74553f426c Use native sqlalchemy 0.9 quote attribute with ibmdb2
Commit 8d6ce64cd0 started using the native
quote attribute built into sqlalchemy 0.9 but missed the changes to the
ibmdb2 changeset, so alter table statements fail for DB2 on sqlalchemy
>= 0.9 (tested against 0.9.8).

This fixes the same issue for the ibmdb2 changeset.

Change-Id: Ia3fa6c3090b5eab29ed7746f4795d502990b8a2f
2015-01-09 14:31:28 -08:00
Brant Knudson 244c6c55d7 Don't add warnings filter on import
The changeset module was adding a warnings filter on import. This
affects all applications that wind up importing it. A library
shouldn't modify the warnings filters unless asked.

Closes-Bug: #1407736
Change-Id: I893f8be48efd3d3642e977ab587c9e6dc867258b
2015-01-08 16:22:59 -06:00
Cyril Roelandt 677f374a68 Replace assertNotEquals with assertNotEqual.
The former is deprecated and the latter should be used.

Change-Id: I9d6dca41cb737062e6d4467c24dbc88901ab9a14
2014-10-30 08:56:50 +01:00
Jenkins 1e83840c98 Merge "Fix ibmdb2 unique constraint handling for sqlalchemy 0.9" 2014-10-10 12:32:25 +00:00
Longgeek cee9136256 Update requirements file matching global requ
* update the pbr version
* update the sqlalchemy version

Change-Id: I21364e2b00d0d5cb9428f8ede1ca0e14aad390fc
2014-09-12 19:52:02 +08:00
Jeremy Stanley c14a311a48 Work toward Python 3.4 support and testing
Change-Id: Idcf09fcbab416bc7af201e7244ba375b5ba1d430
2014-09-03 20:06:21 +00:00
Ihar Hrachyshka 30f6aeab91 pep8: mark all pep8 checks that currently fail as ignored
Change-Id: Ic1dfb51598b920611acb1a0e437a32c5af90beda
2014-08-23 21:34:13 +02:00
Ihar Hrachyshka 93ae21007d SqlScript: execute multiple statements one by one
Some drivers [f.e. MySQL Connector] do not like multiple statements
being passed to .execute(). They require either passing multi=True
parameter to .execute() that is not DB-API 2.0 defined, or executing
those statements one by one.

For that patch, I've chosen the second option to stay away from driver
specific hacks.

Also removed SQLite hack that seems to be related to the same multiple
statements issue.

blueprint enable-mysql-connector

Change-Id: Ic6d53ed1fef8aee9471f3540f06b39cd5ee4ef82
2014-08-23 21:34:13 +02:00
Ihar Hrachyshka be1dd6730a Make sure we don't throw away exception on SQL script failure
If SQL script failed, we don't currently log the failure anywhere, so
users have hard time debugging an issue, if anything arises.

Let's log the failure before proceeding with rollback.

Change-Id: Ic92b1403c00bb238a68265a15150a4be6f6b2346
2014-08-23 21:34:13 +02:00
Ihar Hrachyshka feb0c15aee Pin testtools to < 0.9.36
New testtools require that setUp() and tearDown() are not called twice,
while test_schema is built around multiple calls. So pinning the version
for now until a proper fix is implemented.

Closes-Bug: 1360252
Change-Id: I184db35242d036dacc7933c1d762cccc7f5c40bb
2014-08-23 21:34:07 +02:00
Matt Riedemann 7bb74f70e9 Fix ibmdb2 unique constraint handling for sqlalchemy 0.9
The ibmdb2 unique constraint code was accessing the private _all_cols
member var in iterating over columns which breaks in sqlalchemy 0.9 so
fix up the code to not use internals of sqlalchemy.

UniqueConstraint in sqlalchemy extends ColumnCollectionConstraint
which implements __iter__ to generate a tuple of the columns in
the constraint, so we just iterate over the constraint as the fix.

This is based on a patch from Rahul Priyadarshi in ibm-db-sa issue
158:

https://code.google.com/p/ibm-db/issues/detail?id=158

Co-Authored-By: Rahul Priyadarshi <rahul.priyadarshi@in.ibm.com>

Change-Id: I0f06f6314c382e83573d762abe5981db0a02a83a
2014-08-04 07:34:08 -07:00
Peter Conerly 5542e1c59a Fixes the auto-generated manage.py
Removes `six` from the locals().copy() so that it won't be templated into
manage.py.  Currently manage.py is mis-templated and is failing.

Change-Id: Ib3b7c7caac998fbaa45c3370547c9b8bf13abe41
Closes-Bug: 171
2014-08-02 11:41:39 -07:00
Jenkins 942e03b2b1 Merge "Fix 3 files with Windows line endings to Unix line endings." 2014-07-22 21:48:39 +00:00
Jenkins 089663761c Merge "Move patch from oslo to drop unique constraints with sqlite" 2014-05-05 15:15:03 +00:00
Matt Riedemann 93efb62fd1 Move patch from oslo to drop unique constraints with sqlite
oslo-incubator commit 3f503faac for making sqlite work with dropping
unique constraints in database migrations. This was made in
oslo-incubator since at the time sqlalchemy-migrate was not in
stackforge. Now that we can update sqlalchemy-migrate, move the patch
over from oslo.

This change also adds the support for the case that a unique constraint
is dropped because the column it's on is dropped.

Note that there are already unit tests that cover dropping a unique
constraint directly and implicitly via dropping a column that is in
the unique constraint.

Related-Bug: #1307266

Change-Id: I5ee8082a83aebf66f6e1dacb093ed79e13f73f5e
2014-04-15 19:22:03 -07:00
Cyril Roelandt a03b141a95 Port to Python3
Brief summary of the modifications:

* Use six for compatibility with both Python 2 and 3;
* Replace UserDict.DictMixin with collections.MutableMapping;
* Fix relative imports;
* Use test-requirements.txt for requirements that are common to both Python 2
  and 3, and test-requirements-py{2,3}.txt for version-specific requirements;
* Miscellaneous fixes.
* Use a specific test_db_py3.cfg file for Python 3, that only runs tests on
  sqlite.

Thanks to Victor Stinner who co-wrote this patch.

Change-Id: Ia6dc536c39d274924c21fd5bb619e8e5721e04c4
Co-Authored-By: Victor Stinner <victor.stinner@enovance.com>
2014-04-09 17:32:52 +02:00
Cyril Roelandt 07909159ae tests: Replace "self.assert_" by "self.assertTrue"
The assert_() method is deprecated and can be safely replaced by assertTrue().
This patch makes sure that running the tests does not fill the screen with
warnings.

Change-Id: I8966b7f7a44f1573a4d2c398717bfc68ae40b197
2014-03-31 15:08:29 +02:00
Jenkins fad07f1c8e Merge "Convert tabs to spaces in a couple of rst files" 2014-03-29 18:27:34 +00:00
Jenkins e20068490b Merge "Eradicate trailing whitespace" 2014-03-29 18:27:33 +00:00
Sean Dague 0b08e91df2 turn on testing for sqla 0.9
this enables tox testing for the sqla 0.9 changes, and provides
new targets for sqla 0.9 and 0.8.

Change-Id: I297dce0267bd10cd7db0fe270945c8e5a3431167
2014-03-05 08:20:54 -05:00
Thomas Goirand 7161cf2c9e Replace AbstractType by TypeEngine
AbstractType not longer exists in the class  hierarchy for types.
TypeEngine was its direct descendant, so use that instead.

Change-Id: Idbfaee4b0d3acbc4795913ddf2ab4e1c9b6d065c
2014-03-05 08:20:54 -05:00
Thomas Goirand b58b4a353c fix scripttest compat
There's no script_path param in the current version of
scripttest. This patch fixes that by removing the param,
which by the way isn't useful.

Change-Id: Ic78cea25bb472702473e98b48a8ff74c01545aa3
2014-03-05 08:20:54 -05:00
Thomas Goirand 8d6ce64cd0 Use native quote attribute introduced in sqla 0.9
In SQLA 0.9 there is now a native .quote attribute on many objects.
Conditionally use this instead of the old method if the attribute
exists, to remove deprecation messages (and prepare for when the
other way will be fully removed).

Change-Id: I3c5fada13e044c1c4102acc0455226ce1524f2e2
2014-03-05 08:20:49 -05:00
Thomas Goirand bcb6991615 Fix genmodel for SQLA 0.9
Problem:
* Some python code was auto generated and exec'ed in that
package.
* The python code that was problematic had a 'Table' definition
* The generated code imports '*' from sqlalchemy
* One among the 'Column' was defined as an 'INTEGER' type, which
points to sqlalchemy.sql.sqltypes.INTEGER
* The INTEGER class was initialised with a parameter display_width
which contradicts with sqlalchemy.sql.sqltypes.INTEGER.__init__,
which does not accept any parameters
* The 'INTEGER' class should have been imported from mysql dialects'
type module
Solution:
* While generating, in the header part, I am now checking if any of
the column.type.__class__.__name__ has 'dialects' in it.
* If I find any, I am adding the import rule such that the type is
imported from the dialects' type.

This patch has been tested with SQLA 0.9.3, for which it fixes the
unit tests, and with SQLA 0.8.2, which doesn't have (new) problems
with this patch.

Change-Id: Ie0e09b45388462629100017bea3ea8a314d148d8
2014-03-05 06:55:25 -05:00
Jenkins 0ffaaae935 Merge "UniqueConstraint named and escaped twice" 2014-03-05 08:46:17 +00:00
Jenkins 716124f2ad Merge "Conditionally import ibmdb2/ibm_db_sa" 2014-03-04 20:38:53 +00:00
Matt Riedemann 12a6bcfa8c Conditionally import ibmdb2/ibm_db_sa
Since ibm_db_sa is not part of sqlalchemy, we need to handle the
conditional import of the module in visitor.py so we don't get an
ImportError if ibm_db_sa is not available.

Closes-Bug: #1287229

Change-Id: Ida070b629ce3b9be727ae49973bb6a71543c1dcf
2014-03-04 10:12:40 -08:00
Thomas Goirand f944414bd4 migrate needs subunit >= 0.0.18
The unit tests of subunit are failing with an older version of
subunit, so we need to fix the test-requirements.txt to reflect
the reality.

Change-Id: I786c70a02196b00fb0d96feb5fd1a0b5281f6672
2014-03-02 15:18:31 +08:00
Thomas Goirand 98ae3ac832 UniqueConstraint named and escaped twice
This patch fixes get_constraint_name in the
ANSIConstraintCommon class. It's part of the
fixes needed for SQLA 0.9.x compat.

Change-Id: I1f1648af48f459bd18f99bb42fa9a272186fb37d
2014-03-02 13:12:45 +08:00
David Ripton 6502d4017a Fix 3 files with Windows line endings to Unix line endings.
Change-Id: Iadc8e5d195bf998a117da4b7102a8955e238dd4e
2014-02-27 10:32:17 -05:00
David Ripton d6fbf12989 Eradicate trailing whitespace
Remove all trailing spaces and tabs in every file in the project.
People have editors configured to do this, which causes them to
accidentally make little whitespace changes in unrelated commits,
which makes those commits harder to review.  Better to fix them all
at once.

Change-Id: I17d89f55f41d8599e0ab1a31f646cd161289703e
2014-02-26 15:04:54 -05:00
David Ripton 82f739b96f Convert tabs to spaces in a couple of rst files
These restructured text files mostly use spaces, but a few stray
tabs crept in.  Change them for consistency.

Change-Id: I89f390d02c737798007423a5126d81ad3d9e032e
2014-02-26 15:00:50 -05:00
Jenkins 21fcdad0f4 Merge "Add DB2 10.5 Support" 2014-02-23 19:16:52 +00:00
Sean Dague fe148d87b4 uncap SQLA in requirements.txt
migrate can't just take a global requirements sync because it
needs to be tested against multiple versions of SQLA to assure
compatibility. A recent change had the effect of only testing
migrate against SQLA 0.7, which is definitely *not* what we
want to be doing.

this reverts that change, and leaves very specific comments to
hopefully prevent this from happening in the future.

Change-Id: Icb4e136f0de6caa224019bb955341c4b67c5e1a1
2014-02-23 11:38:53 -05:00
Matt Riedemann 85317aead6 Add DB2 10.5 Support
This patch adds the initial support for DB2 10.5 to migrate. It
includes:

1. The dialect implementation for DB2.
2. The dialect registration with the visitor.
3. Code to parse the engine name in version.py.
4. A new dependency on ibm_db_sa in test-requirements.txt.
5. A connection string in test_db.cfg for ibm_db_sa.

Part of blueprint add-db2-support

Co-authored-by: Sheng Bo Hou <sbhou@cn.ibm.com>
Co-authored-by: Thuy Christenson <thuy@us.ibm.com>
Co-authored-by: Rahul Priyadarshi <rahul.priyadarshi@in.ibm.com>
Change-Id: I745ec615487b1b06c5d1a09ea316f376d66ee4c0
2014-02-17 07:17:31 -08:00
Sascha Peilicke d58e47f85c Sync with global requirements
Change-Id: If05a5d65558ca848b80c3f57c19cb5029adbe58a
2014-02-12 16:46:49 +01:00
Eric Harney ce351c06be Fix broken development version link in README
Change-Id: Id9035c167d397cdad32d418c37108586344d8edc
2013-11-15 13:41:47 -05:00
Monty Taylor 928a846619 Un-break the version in migrate/__init__.py
Change-Id: Ie642315c20866435a709d576c99d6e293e7a7e63
2013-11-15 13:04:56 -05:00
Dan Prince 73df211afe Fix the version number to match the last release.
** NOTE: our release process really should do this
ahead of time.

Change-Id: Ic0cce0d57b4f05092417c4cf1a4ca5a74812ec3c
2013-11-15 12:09:52 -05:00
Jenkins 82f94dd116 Merge "Remove the tag_build line from setup.cfg" 2013-11-15 07:04:25 +00:00
David Ripton fbc7ef03d1 Remove the tag_build line from setup.cfg
It was causing the release tarball to have an extra ".dev." in its filename,
which then broke the script that uploads the release to PyPI.

Change-Id: I995fb3e0393468568601e8614c0532b93fbe8ceb
2013-11-14 14:55:10 -05:00
Sascha Peilicke 40624e7ddc Drop setuptools_git test requirement
We don't need this since we're using pbr.

Change-Id: I7c5985fac66d5e7a4fca1b3945a18a364a94e971
2013-11-14 13:40:12 +01:00
Matt Riedemann c49b8beb83 Fix int overflow exception in unittest
Fixes:

  File ".../versioning/version.py", line 30, in __init__
      if self < 0:
  OverflowError: Python int too large to convert to C long

Don't use __cmp__ which is deprecated and restricted to C
long ints, rather than python's arbitrary precision ints.

Copied from Pádraig Brady's Fedora patch:

http://pkgs.fedoraproject.org/cgit/python-migrate.git/commit/?id=a01bf449

Co-authored-by: Pádraig Brady <pbrady@redhat.com>
Change-Id: I71f349f97507525b2f2edaf034005d67b6cc3987
2013-11-06 07:19:42 -08:00
Roman Podolyaka 2485118c24 Fix dropping of indexed columns in sqlite/sa08
Version 0.8 of SQLAlchemy added support of indexes
on expressions in addition to plain table columns,
which changed the way indexes are created.

This broke support of dropping columns of composite
indexes for SQLite: due to limitations of ALTER in
SQLite every time a column is dropped, we recreate
the whole table without the given column; if a
column is a part of a composite index, we change the
index definition to omit that column and then indexes
are recreated too.

SQLAlchemy versions starting from 0.8 no more pay
attention to 'columns' attribute of Index instances
when generating DDL for indexes, so when one of columns
of a composite index is dropped, we try to create a
new index on the column that doesn't exist anymore,
which of course fails.

Closes-Bug: #1241038

Change-Id: I777b8ce36e36f49bfb0889908811a063cf1a527b
2013-10-18 16:09:16 +03:00
Roman Podolyaka a91766a1ac Run tests on PostgreSQL and MySQL too
In addition to running tests with different Python and SQLAlchemy
versions, they should also be run on different DB backends, which
are used in production (PostgreSQL and MySQL).

This patch:
  - modifies test_db.cfg to run tests on PostgreSQL and MySQL
    (Jenkins Slave credentials are used here, to ensure these
    tests are always run by Jenkins gate); if a backend is not
    available, test cases will be skipped for it
  - concurrency is set to 1 (sharing of the one MySQL or PostgreSQL
    DB among different test runner processes would lead to
    race conditions)
  - fixes tests dropping FK columns in MySQL: in earlier MySQL
    versions dropping a column that is a part of a FK constraint
    would lead to dropping of the FK too. As of MySQL 5.5 that's
    not the case anymore: if one tries to drop such column, he/she
    will get a very obscure error (something like "Error on rename
    of './openstack_citest/#sql-4af_aa2' to './openstack_citest/tmp_adddropcol'
    (errno: 150)") '\nALTER TABLE tmp_adddropcol DROP COLUMN r2').
    So the solution if to drop FK constraints first, and only then
    the columns it is made up of

Change-Id: I8c5d2874c83e7df46da69969ed54d85437c849e7
2013-10-12 07:04:33 +00:00
Monty Taylor 838c1cbb14 Update tox requirements
Update to needing 1.6, which gives us the ability to alter how the
software is installed into the virtualenv. It also brings in pip 1.4,
which lets us avoid getting pre-releases of things we weren't expecting.

Change-Id: I3189f06610d776a032b5f8bf0910f59e4ed45719
2013-10-07 12:03:28 -04:00
Monty Taylor 2ac02e0ef5 Stop using the d2to1-based pbr
pbr updated itself to remove the d2to1 dependency. One of the main
reasons for this is a direct upstream dependency on distribute, which
causes the distribute/setuptools merge bug to get triggered, which
breaks people's system if they install with pip < 1.4.

Remove d2to1 references, and also bump the required version requested of
pbr to a version late enough to no longer depend on d2to1.

Remove the version specifier from setup.py, because setup_requires has
no way to trigger a version requirement upgrade. The flat version should
be sufficient for the setup.py interface in perpetuity.

Remove the setup-hook from setup.cfg, as it is no longer needed.

Change-Id: I40d3507c77802d13fd894aa16714b2ed6aa7d59f
2013-10-07 12:02:24 -04:00
Pádraig Brady 5c646eb558 decouple index name generation from sqlalchemy version
In commit 0.7.2-16-gc670d1d the _index_identifier() implementation
was copied from sqlalchemy, as that function was renamed
in sqlalchemy 0.8.  Instead handle call the renamed function
when appropriate, to decouple ourselves from the sqlalchemy
implementation.

Change-Id: I97b22c20d96758fc5b6bd55318218edb26c5b5d0
2013-09-23 15:18:59 +01:00
Roman Podolyaka ddea3a8bc8 Run tests with different SQLAlchemy versions
The test suite should be run for all possible combinations
of Python and SQLAlchemy versions we claim to support, which
are Python 2.6 and 2.7 and SQLAlchemy 0.7.x (old stable) and
0.8.x (mainline stable).

py26/py27 tox envs install the latest stable version of SQLAlchemy
(currently, 0.8.x branch), so two additional tox environments added
to run the test suite with SQLAlchemy 0.7.x (for both Python 2.6 and 2.7).

Change-Id: I50c7009d8b808ce3bbb1f0a27c50f5cb5116cdb3
2013-08-05 10:40:55 +03:00
Roman Podolyaka 427149eef2 Add a workaround for pytz and pip>=1.4
pip versions >= 1.4 will not install pytz unless a minimum
version containing a letter is specified (this workaround is
taken from Babel Python library).

Change-Id: Ic013fca82816cd89e446282d3936e38ea2c39274
2013-08-05 10:38:46 +03:00
Monty Taylor b5d64cdbe4 Add a reqs files for RTFD
RTFD only has a single requirements field, so make one that installs the
total set of needed requirements to allow docs to build properly.

Change-Id: I79b8202a6a7944cec052a2431dff670258bf7171
2013-07-12 10:53:37 -04:00
Pádraig Brady 74ccf7a397 Fix exceptions for SQLAlchemy 0.8
Change-Id: Ib541ac9d6b397300e34ca8b65aad459b612858c3
2013-07-11 22:01:32 -04:00
Josip Delic c670d1d36a added bugfixes for 0.8 2013-07-11 17:07:04 -04:00
Monty Taylor a71799ea2c Updated to OpenStack Build stuff. 2013-07-11 16:54:46 -04:00
Monty Taylor 85fe71617f Removed hg and google code references 2013-07-11 15:53:46 -04:00
Monty Taylor e0edf4d4a1 Initial changes to import into StackForge. 2013-07-11 15:50:07 -04:00
Jan Dittberner 08cacdac59 update changelog 2013-02-09 13:08:53 +01:00
Jan Dittberner d9f322cf6e fix error, Text columns have no width 2013-02-09 13:07:39 +01:00
Jan Dittberner 195f95550b fix deprecation warning by using MetaData.reflect 2013-02-09 13:02:59 +01:00
Jan Dittberner 2e26236baa update credits and changelog 2013-02-09 11:18:27 +01:00
Alex Favaro 2ff11c176a Import correct exceptions module (Fixes issue 154). 2013-02-09 11:15:26 +01:00
Jan Dittberner 84f4faf113 update changelog and credits 2012-01-23 21:36:13 +01:00
asuffield@gmail.com fe1f1deca8 apply patch for issue #72 by asuffield@gmail.com
fixes #72
2012-01-23 21:34:12 +01:00
Jan Dittberner ddad9c7f8c update changelog and credits 2011-12-17 20:27:22 +01:00
Jason Michalski 7a5148a30b Fix excludeTablesgetDiffOfModelAgainstModel is not passing
excludeTables correctly

Fixes issue 140
2011-12-17 20:25:04 +01:00
Jan Dittberner 08310eec9d start next development iteration 2011-11-01 22:10:37 +01:00
Jan Dittberner aadd728366 Added signature for changeset ad06c76fc174 2011-11-01 21:35:16 +01:00
Jan Dittberner da930f3b01 tag for release 0.7.2 2011-11-01 21:34:26 +01:00
108 changed files with 2417 additions and 1152 deletions

View File

@ -1,6 +1,8 @@
syntax: glob
.eggs/
AUTHORS
ChangeLog
.tox
.testrepository/
*.pyc
*data/*
*build/*

4
.gitreview Normal file
View File

@ -0,0 +1,4 @@
[gerrit]
host=review.opendev.org
port=29418
project=x/sqlalchemy-migrate.git

View File

@ -1,3 +0,0 @@
65742e996d94a99c6a7e01e26acfbdaa6aba4efc 0 iQIcBAABCAAGBQJMOhCuAAoJEKc+AFVVj7jdDxwP/jhb9kMSLtmxzG3k1vboKE0It2b4CM/qIj554IQolqo3p5HvHq6O9fYhMV492VR36JbwEUei0kbcl4UzNy3BXnpZjHUSlpgoYkDcIJXtMhUBsoOiQ4IZmnBf4TEMtDbO6QrYbrCO75lGsjIc90D85bOK0mj7ILjSQa9o0Ic7qbMW02CRMIar0Y+0J79sWUu2gRhesHdVm3z5LpkgCkntlTf6u0zwZXV3o4NRu80ZWi0uFvCfPga5itFqgdZmV50mMWoJb3eF2H/yL811ShZergrI+q91V/uKQfyQrOUm7M6hSdP58Z5S+z5HgGKk7rlk0ha9JQbEnd8/E6cH4XFaytIZC7QsO2NoJrJo8ycw3NApkO8+n5gRKSxK+xWgTO677xNR76rMUKqdj10SsZFa9sIhGk77okX0oxdg8DHB+f0Ay2fvdmgPLx8aSHth/NRhzw9/97z4KkOMG4ZcvZK3TRxyscISKsTPJIv1iAwE6avdjJfh/2rp8zqGQEUREU9EJUBYkPkjBVX+Rflg7aY4Sa5SxRxkPxgGpDJHu02VoCeINIOBDX6lhg7FyeZMYeMXA8+BNa/xwaeh7PES+svW9KGHwMo9zVQBwvzG1WBQ4S3DeL1um7eJ1k88OyhTSzcN8CiOMFk3hYd8i4qLOuTAjccbcv8JKuQPOKpoNirPO843
35038c66152bc1b0d37131da3049843a9ee4fbe7 0 iQIcBAABCAAGBQJN4A2yAAoJEKc+AFVVj7jdCiEQALxC8o6dzcyZaqlSvzNEzTXE14rdH+SNefg1KDQB5p5/X63mWq8TDQxblV/ZHDbgs6ffRJ/nS3ftPYUPoZbz1kutnZWK6+3PF4xZhRIrDakfP7IOOFGIZBM+crWlGdpvCOJVXLLDmsPvvDDHfrR2RLNGtv+gGGySluVF4KA2mzHfihpZnbGTzmWJOttKDPbYiPwft0h94SRNr5gb306ML5nNwSbs9z1NlZP2adHhTtGhLhZkDyiV4Q9HDFx2RYylxfBJkYYOItgOosBFe23MxKITIdwVH2g6httUSqbAdV2O2wfpTuc+geWWE3GboEY8GxNGACMy5kKmi74mfummSIscsC0SrtYqumiR5vOdQ6azKjYn5eEi9+Uiv/zWsVwGf6ylo+88upOkD+rz4Oa4WffdeLCOgDIjXyuSLD+lagep2wSm/L9kVXAlsQpAK+qGU4+ieDgT9xe/946IqcPplE+K43ivvDrpeLZt0xSS6y5ad8TqlwpreFWIDTL4hHZr+Hp+c6pX2aaARYHqdMWj7NmZ5YTY7RruotgS/XtzS9BFRg/FVmvxqI2jApD5oEtYa6PYWJn5B1nipwN4szYzVoxbJANrwa/yc99JuykBRsL6li6MECziCrCejz1kPQqyxBZjkcBqSC+GcSh2n3YTcH6lbtHtLQP5Cdc/IMqEhdNG
fbb2817a1e3f556b9b3ec1a5569f3f94353da0cc 0 iQIcBAABCAAGBQJN4A3MAAoJEKc+AFVVj7jddNcQAKJLNEmYUVU9l1YUYmtzVezLiDlEnViaUjfMGhnodk9nWTQfRaFp5wEEOhz8Ag8usly1Icy5UwcJ87PF+yTq13yxZn4LpqSFk5Ci6s9q8w3j+JMfL8nsKXV/DqM7Q1i0eveElSUCS3ejoK+EYU29erSQJ0PYzhHz2+h67QcaHWp1BB+9wghWaQNtV4RDhDm5phYeGnbcZTa1ONJRZc3z/NjQK/LAX1qmuC6Mx0o32l3BqlhPeX3mNL9c2nBTXPTQQ26Wqu3oDxtu0v442aqWOZ/SEbqKcUUL0h/OTuiRq18iCljlItmteAOLz3DH+1gVRVv3KsNOz/i1Cr4bvTVl7Ks7W2jYGowaufurMYjUeGx7I3b+2p7++CqSpyXdbYae4dLfijgTiX7niF8L1jcCjHGGl9bjQnfe+hyqpfCUlDUKoNLWYLJu6iyz/3saJvAB1IKJQohZMgkiJYyZtHTEgQULeK2iYnwjxiUEAuNRs0hPS0fcZOrlKjSf7GUXRygDJqXcCXvkGO4h7amyt3Kqgciy5ll9IToEV+6VgvFvAUaFoOPyh3aT4ci4cCbWpcNLD/w9GVoO135niCqNmiXjD4A+VgAdAwKG0s4ZDS5r3lFz8gRAw5aTy6nALjKMQdd35lA7cOcVeRnqlAl6W51fQBt3PEeFmF4tmwizFUKc8rKF

View File

@ -1,4 +0,0 @@
cb01bf174b05b1590258d6c996b89f60ebd88e5a v0.6
c2526dce0768f11e6bf88afb641a6a9058fa685c v0.6.1
35038c66152bc1b0d37131da3049843a9ee4fbe7 v0.7
fbb2817a1e3f556b9b3ec1a5569f3f94353da0cc v0.7.1

8
.testr.conf Normal file
View File

@ -0,0 +1,8 @@
[DEFAULT]
test_command=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-60} \
${PYTHON:-python} -m subunit.run discover -t ./ . $LISTOPT $IDOPTION
test_id_option=--load-list $IDFILE
test_list_option=--list

37
.zuul.yaml Normal file
View File

@ -0,0 +1,37 @@
- project:
templates:
- docs-on-readthedocs
- openstack-python-jobs
- openstack-python35-jobs
- openstack-python36-jobs
vars:
rtd_webhook_id: '61274'
check:
jobs:
- sqlalchemy-migrate-tox-py27sa07
- sqlalchemy-migrate-devstack:
voting: false
gate:
jobs:
- sqlalchemy-migrate-tox-py27sa07
- job:
name: sqlalchemy-migrate-tox-py27sa07
parent: tox
description: |
Run tests for sqlalchemy-migrate project.
Uses tox with the ``py27sa07`` environment.
vars:
tox_envlist: py27sa07
- job:
name: sqlalchemy-migrate-devstack
parent: legacy-dsvm-base
run: playbooks/sqlalchemy-migrate-devstack-dsvm/run.yaml
post-run: playbooks/sqlalchemy-migrate-devstack-dsvm/post.yaml
timeout: 10800
required-projects:
- openstack/devstack
- openstack/devstack-gate
- x/sqlalchemy-migrate

View File

@ -1,7 +1,8 @@
include AUTHORS
include ChangeLog
include README
recursive-include docs *
recursive-include migrate *
recursive-include tests *
global-exclude *pyc
exclude .hgtags
recursive-exclude docs/_build *

44
README
View File

@ -1,44 +0,0 @@
Inspired by Ruby on Rails' migrations, Migrate provides a way to deal with
database schema changes in `SQLAlchemy <http://sqlalchemy.org>`_ projects.
Migrate extends SQLAlchemy to have database changeset handling. It provides a
database change repository mechanism which can be used from the command line as
well as from inside python code.
Help
------
Sphinx documentation is available at the project page `packages.python.org
<http://packages.python.org/sqlalchemy-migrate/>`_.
Users and developers can be found at #sqlalchemy-migrate on Freenode IRC
network and at the public users mailing list `migrate-users
<http://groups.google.com/group/migrate-users>`_.
New releases and major changes are announced at the public announce mailing
list `migrate-announce <http://groups.google.com/group/migrate-announce>`_
and at the Python package index `sqlalchemy-migrate
<http://pypi.python.org/pypi/sqlalchemy-migrate>`_.
Homepage is located at `code.google.com
<http://code.google.com/p/sqlalchemy-migrate/>`_
You can also clone a current `development version
<http://code.google.com/p/sqlalchemy-migrate/source/checkout>`_ from the
project's `mercurial <http://mercurial.selenic.com/>`_ trunk.
Tests and Bugs
------------------
To run automated tests:
* Copy test_db.cfg.tmpl to test_db.cfg
* Edit test_db.cfg with database connection strings suitable for running tests.
(Use empty databases.)
* $ pip install -r test-req.pip
* $ python setup.py develop
* $ nosetests
Please report any issues with sqlalchemy-migrate to the issue tracker at
`code.google.com issues
<http://code.google.com/p/sqlalchemy-migrate/issues/list>`_

47
README.rst Normal file
View File

@ -0,0 +1,47 @@
SQLAlchemy Migrate
==================
Fork from http://code.google.com/p/sqlalchemy-migrate/ to get it working with
SQLAlchemy 0.8.
Inspired by Ruby on Rails' migrations, Migrate provides a way to deal with
database schema changes in `SQLAlchemy <http://sqlalchemy.org>`_ projects.
Migrate extends SQLAlchemy to have database changeset handling. It provides a
database change repository mechanism which can be used from the command line as
well as from inside python code.
Help
----
Sphinx documentation is available at the project page `readthedocs.org
<https://sqlalchemy-migrate.readthedocs.org/>`_.
Users and developers can be found at #openstack-dev on Freenode IRC
network and at the public users mailing list `migrate-users
<http://groups.google.com/group/migrate-users>`_.
New releases and major changes are announced at the public announce mailing
list `openstack-dev
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>`_
and at the Python package index `sqlalchemy-migrate
<http://pypi.python.org/pypi/sqlalchemy-migrate>`_.
Homepage is located at `stackforge
<http://github.com/stackforge/sqlalchemy-migrate/>`_
You can also clone a current `development version
<http://github.com/stackforge/sqlalchemy-migrate>`_
Tests and Bugs
--------------
To run automated tests:
* install tox: ``pip install -U tox``
* run tox: ``tox``
* to test only a specific Python version: ``tox -e py27`` (Python 2.7)
Please report any issues with sqlalchemy-migrate to the issue tracker at
`Launchpad issues
<https://bugs.launchpad.net/sqlalchemy-migrate>`_

1
TODO
View File

@ -26,7 +26,6 @@ Unknown milestone
- verbose output on migration failures
- interactive migration script resolution?
- backend for versioning management
- port to unittest2
Documentation updates in 0.6.1
- glossary

16
bindep.txt Normal file
View File

@ -0,0 +1,16 @@
# This is a cross-platform list tracking distribution packages needed for install and tests;
# see https://docs.openstack.org/infra/bindep/ for additional information.
# NOTE(mriedem): This list is woefully incomplete but is just listing mysql
# and postgresql binary dependencies to make tools/test-setup.sh work.
libmysqlclient-dev [platform:dpkg]
libpq-dev [platform:dpkg test]
mysql [platform:rpm]
mysql-client [platform:dpkg]
mysql-devel [platform:rpm test]
mysql-server
postgresql
postgresql-client [platform:dpkg]
postgresql-devel [platform:rpm test]
postgresql-server [platform:rpm]

View File

@ -1,6 +0,0 @@
SQLAlchemy >= 0.6
decorator
Tempita >= 0.4
setuptools
Sphinx
sphinxcontrib_issuetracker

2
doc-requirements.txt Normal file
View File

@ -0,0 +1,2 @@
-r requirements.txt
-r test-requirements.txt

1
doc/requirements.txt Normal file
View File

@ -0,0 +1 @@
sphinx>=1.6.2,!=1.6.6 # BSD

View File

@ -1,3 +1,34 @@
0.7.3 (201x-xx-xx)
---------------------------
Changes
******************
-
Documentation
******************
-
Features
******************
-
Fixed Bugs
******************
- #140: excludeTablesgetDiffOfModelAgainstModel is not passing excludeTables
correctly (patch by Jason Michalski)
- #72: Regression against issue #38, migrate drops engine reference (patch by
asuffield@gmail.com)
- #154: versioning/schema.py imports deprecated sqlalchemy.exceptions (patch by
Alex Favaro)
- fix deprecation warning using MetaData.reflect instead of reflect=True
constructor argument
- fix test failure by removing unsupported length argument for Text column
0.7.2 (2011-11-01)
---------------------------
@ -99,9 +130,9 @@ Fixed bugs
- fixed case sensitivity in setup.py dependencies
- moved :py:mod:`migrate.changeset.exceptions` and
:py:mod:`migrate.versioning.exceptions` to :py:mod:`migrate.exceptions`
- cleared up test output and improved testing of deprecation warnings.
- cleared up test output and improved testing of deprecation warnings.
- some documentation fixes
- #107: fixed syntax error in genmodel.py
- #107: fixed syntax error in genmodel.py
- #96: fixed bug with column dropping in sqlite
- #94: fixed bug that prevented non-unique indexes being created
- fixed bug with column dropping involving foreign keys
@ -109,7 +140,7 @@ Fixed bugs
- rewrite of the schema diff internals, now supporting column
differences in additon to missing columns and tables.
- fixed bug when passing empty list in
:py:func:`migrate.versioning.shell.main` failed
:py:func:`migrate.versioning.shell.main` failed
- #108: Fixed issues with firebird support.
0.6 (11.07.2010)

View File

@ -80,10 +80,10 @@ You can create a column with :meth:`~ChangesetColumn.create`:
You can pass `primary_key_name`, `index_name` and `unique_name` to the
:meth:`~ChangesetColumn.create` method to issue ``ALTER TABLE ADD
CONSTRAINT`` after changing the column.
CONSTRAINT`` after changing the column.
For multi columns constraints and other advanced configuration, check the
:ref:`constraint tutorial <constraint-tutorial>`.
:ref:`constraint tutorial <constraint-tutorial>`.
.. versionadded:: 0.6.0
@ -189,30 +189,30 @@ The following rundowns are true for all constraints classes:
cons = PrimaryKeyConstraint('id', 'num', table=self.table)
# Create the constraint
cons.create()
# Create the constraint
cons.create()
# Drop the constraint
cons.drop()
# Drop the constraint
cons.drop()
You can also pass in :class:`~sqlalchemy.schema.Column` objects (and table
argument can be left out):
.. code-block:: python
cons = PrimaryKeyConstraint(col1, col2)
cons = PrimaryKeyConstraint(col1, col2)
#. Some dialects support ``CASCADE`` option when dropping constraints:
.. code-block:: python
cons = PrimaryKeyConstraint(col1, col2)
cons = PrimaryKeyConstraint(col1, col2)
# Create the constraint
cons.create()
# Create the constraint
cons.create()
# Drop the constraint
cons.drop(cascade=True)
# Drop the constraint
cons.drop(cascade=True)
.. note::
SQLAlchemy Migrate will try to guess the name of the constraints for
@ -244,12 +244,12 @@ Foreign key constraints:
.. code-block:: python
from migrate.changeset.constraint import ForeignKeyConstraint
cons = ForeignKeyConstraint([table.c.fkey], [othertable.c.id])
# Create the constraint
cons.create()
# Drop the constraint
cons.drop()
@ -258,9 +258,9 @@ Check constraints:
.. code-block:: python
from migrate.changeset.constraint import CheckConstraint
cons = CheckConstraint('id > 3', columns=[table.c.id])
# Create the constraint
cons.create()
@ -272,7 +272,7 @@ Unique constraints:
.. code-block:: python
from migrate.changeset.constraint import UniqueConstraint
cons = UniqueConstraint('id', 'age', table=self.table)
# Create the constraint

View File

@ -28,7 +28,8 @@ sys.path.append(os.path.dirname(os.path.abspath('.')))
# Add any Sphinx extension module names here, as strings. They can be extensions
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = ['sphinx.ext.autodoc', 'sphinx.ext.intersphinx', 'sphinxcontrib.issuetracker']
extensions = ['sphinx.ext.autodoc',
'sphinx.ext.intersphinx']
# link to sqlalchemy docs
intersphinx_mapping = {
@ -56,9 +57,9 @@ copyright = u'2011, Evan Rosson, Jan Dittberner, Domen Kožar, Chris Withers'
# built documents.
#
# The short X.Y version.
version = '0.7.2'
version = '0.7.3'
# The full version, including alpha/beta/rc tags.
release = '0.7.2'
release = '0.7.3.dev'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
@ -128,7 +129,7 @@ html_style = 'default.css'
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# html_static_path = ['_static']
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.

View File

@ -23,6 +23,7 @@ improve sqlalchemy-migrate:
- Adam Lowry
- Adomas Paltanavicius
- Alexander Artemenko
- Alex Favaro
- Andrew Bialecki
- Andrew Grossman
- Andrew Lenards
@ -54,10 +55,11 @@ improve sqlalchemy-migrate:
- Ilya Shabalin
- James Mills
- Jarrod Chesney
- Jason Michalski
- Jason R. Coombs
- Jayson Vantuyl
- Jason Yamada-Hanff
- Jay Pipes
- Jayson Vantuyl
- Jeremy Cantrell
- Jeremy Slade
- Jeroen Ruigrok van der Werven
@ -92,6 +94,7 @@ improve sqlalchemy-migrate:
- Nicholas Retallack
- Nick Barendt
- Patrick Shields
- Paul Bonser
- Paul Johnston
- Pawel Bylina
- Pedro Algarvio
@ -115,5 +118,7 @@ improve sqlalchemy-migrate:
- Yeeland Chen
- Yuen Ho Wong
- asuffield (at) gmail (dot) com
If you helped us in the past and miss your name please tell us about your
contribution and we will add you to the list.

52
doc/source/download.rst Normal file
View File

@ -0,0 +1,52 @@
Download
--------
You can get the latest version of SQLAlchemy Migrate from the
the `cheese shop`_, pip_ or via easy_install_::
$ easy_install sqlalchemy-migrate
or::
$ pip install sqlalchemy-migrate
You should now be able to use the :command:`migrate` command from the command
line::
$ migrate
This should list all available commands. To get more information regarding a
command use::
$ migrate help COMMAND
If you'd like to be notified when new versions of SQLAlchemy Migrate
are released, subscribe to `openstack-dev`_.
.. _pip: http://pip.openplans.org/
.. _easy_install: http://peak.telecommunity.com/DevCenter/EasyInstall#installing-easy-install
.. _sqlalchemy: http://www.sqlalchemy.org/download.html
.. _`cheese shop`: http://pypi.python.org/pypi/sqlalchemy-migrate
.. _`openstack-dev`: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
.. _development:
Development
-----------
If you would like to contribute to the development of OpenStack,
you must follow the steps in this page:
http://docs.openstack.org/infra/manual/developers.html
Once those steps have been completed, changes to OpenStack
should be submitted for review via the Gerrit tool, following
the workflow documented at:
http://docs.openstack.org/infra/manual/developers.html#development-workflow
Pull requests submitted through GitHub will be ignored.
Bugs should be filed on Launchpad, not GitHub:
https://bugs.launchpad.net/sqlalchemy-migrate

View File

@ -1,7 +1,7 @@
FAQ
===
Q: Adding a **nullable=False** column
Q: Adding a **nullable=False** column
**************************************
A: Your table probably already contains data. That means if you add column, it's contents will be NULL.

View File

@ -13,7 +13,7 @@ Automatically generating the code for this sort of task seems like a good soluti
* Maintence, usually a problem with auto-generated code, is not an issue: old database migration scripts are not the subject of maintenance; the correct solution is usually a new migration script.
Implementation is a problem: finding the 'diff' of two databases to determine what columns to add is not trivial. Fortunately, there exist tools that claim to do this for us: [http://sqlfairy.sourceforge.net/ SQL::Translator] and [http://xml2ddl.berlios.de/ XML to DDL] both claim to have this capability.
Implementation is a problem: finding the 'diff' of two databases to determine what columns to add is not trivial. Fortunately, there exist tools that claim to do this for us: [http://sqlfairy.sourceforge.net/ SQL::Translator] and [http://xml2ddl.berlios.de/ XML to DDL] both claim to have this capability.
...
@ -23,4 +23,4 @@ All that said, this is ''not'' something I'm going to attempt during the Summer
* The project has a deadline and I have plenty else to do already
* Lots of people with more experience than me say this would take more time than it's worth
It's something that might be considered for future work if this project is successful, though.
It's something that might be considered for future work if this project is successful, though.

View File

@ -1,9 +1,9 @@
Important to our system is the API used for making database changes.
Important to our system is the API used for making database changes.
=== Raw SQL; .sql script ===
Require users to write raw SQL. Migration scripts are .sql scripts (with database version information in a header comment).
Require users to write raw SQL. Migration scripts are .sql scripts (with database version information in a header comment).
+ Familiar interface for experienced DBAs.
+ Familiar interface for experienced DBAs.
+ No new API to learn[[br]]
SQL is used elsewhere; many people know SQL already. Those who are still learning SQL will gain expertise not in the API of a specific tool, but in a language which will help them elsewhere. (On the other hand, those who are familiar with Python with no desire to learn SQL might find a Python API more intuitive.)
@ -15,12 +15,12 @@ SQL is used elsewhere; many people know SQL already. Those who are still learnin
Some things are possible in Python that aren't in SQL - for example, suppose we want to use some functions from our application in a migration script. (The user might also simply prefer Python.)
- Loss of database independence.[[br]]
There isn't much we can do to specify different actions for a particular DBMS besides copying the .sql file, which is obviously bad form.
There isn't much we can do to specify different actions for a particular DBMS besides copying the .sql file, which is obviously bad form.
=== Raw SQL; Python script ===
Require users to write raw SQL. Migration scripts are python scripts whose API does little beyond specifying what DBMS(es) a particular statement should apply to.
Require users to write raw SQL. Migration scripts are python scripts whose API does little beyond specifying what DBMS(es) a particular statement should apply to.
For example,
For example,
{{{
run("CREATE TABLE test[...]") # runs for all databases
run("ALTER TABLE test ADD COLUMN varchar2[...]",oracle) # runs for Oracle only
@ -42,12 +42,12 @@ run("ALTER TABLE test ADD COLUMN"+sql("varchar",postgres|mysql,"varchar2",oracle
The user can write scripts to deal with conflicts between databases, but they're not really database-independent: the user has to deal with conflicts between databases; our system doesn't help them.
+ Minimal new API to learn. [[br]]
There is a new API to learn, but it is extremely small, depending mostly on SQL DDL. This has the advantages of "no new API" in our first solution.
There is a new API to learn, but it is extremely small, depending mostly on SQL DDL. This has the advantages of "no new API" in our first solution.
- More verbose than .sql scripts.
- More verbose than .sql scripts.
=== Raw SQL; automatic translation between each dialect ===
Same as the above suggestion, but allow the user to specify a 'default' dialect of SQL that we'll interpret and whose quirks we'll deal with.
Same as the above suggestion, but allow the user to specify a 'default' dialect of SQL that we'll interpret and whose quirks we'll deal with.
That is, write everything in SQL and try to automatically resolve the conflicts of different DBMSes.
For example, take the following script:
@ -74,7 +74,7 @@ CREATE TABLE test (
)
}}}
+ Database-independence issues of the above SQL solutions are resolved.[[br]]
+ Database-independence issues of the above SQL solutions are resolved.[[br]]
Ideally, this solution would be as database-independent as a Python API for database changes (discussed next), but with all the advantages of writing SQL (no new API).
- Difficult implementation[[br]]
@ -83,7 +83,7 @@ Obviously, this is not easy to implement - there is a great deal of parsing logi
It seems tools for this already exist; an effective tool would trivialize this implementation. I experimented a bit with [http://sqlfairy.sourceforge.net/ SQL::Translator] and [http://xml2ddl.berlios.de/ XML to DDL]; however, I had difficulties with both.
- Database-specific features ensure that this cannot possibly be "complete". [[br]]
For example, Postgres has an 'interval' type to represent times and (AFAIK) MySQL does not.
For example, Postgres has an 'interval' type to represent times and (AFAIK) MySQL does not.
=== Database-independent Python API ===
Create a Python API through which we may manage database changes. Scripts would be based on the existing SQLAlchemy API when possible.
@ -104,13 +104,13 @@ add_column('test','id',Integer,notNull=True)
}}}
This would use engines, similar to SQLAlchemy's, to deal with database-independence issues.
We would, of course, allow users to write raw SQL if they wish. This would be done in the manner outlined in the second solution above; this allows us to write our entire script in SQL and ignore the Python API if we wish, or write parts of our solution in SQL to deal with specific databases.
We would, of course, allow users to write raw SQL if they wish. This would be done in the manner outlined in the second solution above; this allows us to write our entire script in SQL and ignore the Python API if we wish, or write parts of our solution in SQL to deal with specific databases.
+ Deals with database-independence thoroughly and with minimal user effort.[[br]]
SQLAlchemy-style engines would be used for this; issues of different DBMS syntax are resolved with minimal user effort. (Database-specific features would still need handwritten SQL.)
+ Familiar interface for SQLAlchemy users.[[br]]
In addition, we can often cut-and-paste column definitions from SQLAlchemy tables, easing one particular task.
In addition, we can often cut-and-paste column definitions from SQLAlchemy tables, easing one particular task.
- Requires that the user learn a new API. [[br]]
SQL already exists; people know it. SQL newbies might be more comfortable with a Python interface, but folks who already know SQL must learn a whole new API. (On the other hand, the user *can* write things in SQL if they wish, learning only the most minimal of APIs, if they are willing to resolve issues of database-independence themself.)
@ -119,29 +119,29 @@ SQL already exists; people know it. SQL newbies might be more comfortable with a
SQL already exists/has been tested. A new Python API does not/has not, and much of the work seems to consist of little more than reinventing the wheel.
- Script behavior might change under different versions of the project.[[br]]
...where .sql scripts behave the same regardless of the project's version.
...where .sql scripts behave the same regardless of the project's version.
=== Generate .sql scripts from a Python API ===
Attempts to take the best of the first and last solutions. An API similar to the previous solution would be used, but rather than immediately being applied to the database, .sql scripts are generated for each type of database we're interested in. These .sql scripts are what's actually applied to the database.
This would essentially allow users to skip the Python script step entirely if they wished, and write migration scripts in SQL instead, as in solution 1.
This would essentially allow users to skip the Python script step entirely if they wished, and write migration scripts in SQL instead, as in solution 1.
+ Database-independence is an option, when needed.
+ Database-independence is an option, when needed.
+ A familiar interface/an interface that can interact with other tools is an option, when needed.
+ A familiar interface/an interface that can interact with other tools is an option, when needed.
+ Easy to inspect the SQL generated by a script, to ensure it's what we're expecting.
+ Easy to inspect the SQL generated by a script, to ensure it's what we're expecting.
+ Migration scripts won't change behavior across different versions of the project. [[br]]
Once a Python script is translated to a .sql script, its behavior is consistent across different versions of the project, unlike a pure Python solution.
Once a Python script is translated to a .sql script, its behavior is consistent across different versions of the project, unlike a pure Python solution.
- Multiple ways to do a single task: not Pythonic.[[br]]
I never really liked that word - "Pythonic" - but it does apply here. Multiple ways to do a single task has the potential to cause confusion, especially in a large project if many people do the same task different ways. We have to support both ways of doing things, as well.
I never really liked that word - "Pythonic" - but it does apply here. Multiple ways to do a single task has the potential to cause confusion, especially in a large project if many people do the same task different ways. We have to support both ways of doing things, as well.
----
'''Conclusion''': The last solution, generating .sql scripts from a Python API, seems to be best.
'''Conclusion''': The last solution, generating .sql scripts from a Python API, seems to be best.
The first solution (.sql scripts) suffers from a lack of database-independence, but is familiar to experienced database developers, useful with other tools, and shows exactly what will be done to the database. The Python API solution has no trouble with database-independence, but suffers from other problems that the .sql solution doesn't. The last solution resolves both reasonably well. Multiple ways to do a single task might be called "not Pythonic", but IMO, the trade-off is worth this cost.
The first solution (.sql scripts) suffers from a lack of database-independence, but is familiar to experienced database developers, useful with other tools, and shows exactly what will be done to the database. The Python API solution has no trouble with database-independence, but suffers from other problems that the .sql solution doesn't. The last solution resolves both reasonably well. Multiple ways to do a single task might be called "not Pythonic", but IMO, the trade-off is worth this cost.
Automatic translation between different dialects of SQL might have potential for use in a solution, but existing tools for this aren't reliable enough, as far as I can tell.
Automatic translation between different dialects of SQL might have potential for use in a solution, but existing tools for this aren't reliable enough, as far as I can tell.

View File

@ -7,7 +7,7 @@ An option not discussed below is "no versioning"; that is, simply apply any scri
A single integer version number would specify the version of each database. This is stored in the database in a table, let's call it "schema"; each migration script is associated with a certain database version number.
+ Simple implementation[[br]]
Of the 3 solutions presented here, this one is by far the simplest.
Of the 3 solutions presented here, this one is by far the simplest.
+ Past success[[br]]
Used in [http://www.rubyonrails.org/ Ruby on Rails' migrations].
@ -32,13 +32,13 @@ Similar to the database-wide version number; the contents of the new table are m
- Determining the version of database-specific objects (ie. stored procedures, functions) is difficult.
- Ultimately gains nothing over the previous solution.[[br]]
The intent here was to allow multiple people to write scripts for a single database, but if database-wide version numbers aren't assigned until the script is placed in the repository, we could already do this.
The intent here was to allow multiple people to write scripts for a single database, but if database-wide version numbers aren't assigned until the script is placed in the repository, we could already do this.
=== Version determined via introspection ===
Each script has a schema associated with it, rather than a version number. The database schema is loaded, analyzed, and compared to the schema expected by the script.
Each script has a schema associated with it, rather than a version number. The database schema is loaded, analyzed, and compared to the schema expected by the script.
+ No modifications to the database are necessary for this versioning system.[[br]]
The primary advantage here is that no changes to the database are required.
The primary advantage here is that no changes to the database are required.
- Most difficult solution to implement, by far.[[br]]
Comparing the state of every schema object in the database is much more complex than simply comparing a version number, especially since we need to do it in a database-independent way (ie. we can't just diff the dump of each schema). SQLAlchemy's reflection would certainly be very helpful, but this remains the most complex solution.
@ -53,4 +53,4 @@ When version numbers are stored in the database, you have some idea of where an
----
'''Conclusion''': database-wide version numbers are the best way to go.
'''Conclusion''': database-wide version numbers are the best way to go.

View File

@ -1,4 +1,4 @@
This is very much a draft/brainstorm right now. It should be made prettier and thought about in more detail later, but it at least gives some idea of the direction we're headed right now.
This is very much a draft/brainstorm right now. It should be made prettier and thought about in more detail later, but it at least gives some idea of the direction we're headed right now.
----
* Two distinct tools; should not be coupled (can work independently):
* Versioning tool

View File

@ -1,32 +1,32 @@
== Goals ==
=== DBMS-independent schema changes ===
Many projects need to run on more than one DBMS. Similar changes need to be applied to both types of databases upon a schema change. The usual solution to database changes - .sql scripts with ALTER statements - runs into problems since different DBMSes have different dialects of SQL; we end up having to create a different script for each DBMS. This project will simplify this by providing an API, similar to the table definition API that already exists in SQLAlchemy, to alter a table independent of the DBMS being used, where possible.
Many projects need to run on more than one DBMS. Similar changes need to be applied to both types of databases upon a schema change. The usual solution to database changes - .sql scripts with ALTER statements - runs into problems since different DBMSes have different dialects of SQL; we end up having to create a different script for each DBMS. This project will simplify this by providing an API, similar to the table definition API that already exists in SQLAlchemy, to alter a table independent of the DBMS being used, where possible.
This project will support all DBMSes currently supported by SQLAlchemy: SQLite, Postgres, MySQL, Oracle, and MS SQL. Adding support for more should be as possible as it is in SQLAlchemy.
Many are already used to writing .sql scripts for database changes, aren't interested in learning a new API, and have projects where DBMS-independence isn't an issue. Writing SQL statements as part of a (Python) change script must be an option, of course. Writing change scripts as .sql scripts, eliminating Python scripts from the picture entirely, would be nice too, although this is a lower-priority goal.
Many are already used to writing .sql scripts for database changes, aren't interested in learning a new API, and have projects where DBMS-independence isn't an issue. Writing SQL statements as part of a (Python) change script must be an option, of course. Writing change scripts as .sql scripts, eliminating Python scripts from the picture entirely, would be nice too, although this is a lower-priority goal.
=== Database versioning and change script organization ===
Once we've accumulated a set of change scripts, it's important to know which ones have been applied/need to be applied to a particular database: suppose we need to upgrade a database that's extremenly out-of-date; figuring out the scripts to run by hand is tedious. Applying changes in the wrong order, or applying changes when they shouldn't be applied, is bad; attempting to manage all of this by hand inevitably leads to an accident. This project will be able to detect the version of a particular database and apply the scripts required to bring it up to the latest version, or up to any specified version number (given all change scripts required to reach that version number).
Once we've accumulated a set of change scripts, it's important to know which ones have been applied/need to be applied to a particular database: suppose we need to upgrade a database that's extremenly out-of-date; figuring out the scripts to run by hand is tedious. Applying changes in the wrong order, or applying changes when they shouldn't be applied, is bad; attempting to manage all of this by hand inevitably leads to an accident. This project will be able to detect the version of a particular database and apply the scripts required to bring it up to the latest version, or up to any specified version number (given all change scripts required to reach that version number).
Sometimes we need to be able to revert a schema to an older version. There's no automatic way to do this without rebuilding the database from scratch, so our project will allow one to write scripts to downgrade the database as well as upgrade it. If such scripts have been written, we should be able to apply them in the correct order, just like upgrading.
Sometimes we need to be able to revert a schema to an older version. There's no automatic way to do this without rebuilding the database from scratch, so our project will allow one to write scripts to downgrade the database as well as upgrade it. If such scripts have been written, we should be able to apply them in the correct order, just like upgrading.
Large projects inevitably accumulate a large number of database change scripts; it's important that we have a place to keep them. Once a script has been written, this project will deal with organizing it among existing change scripts, and the user will never have to look at it again.
Large projects inevitably accumulate a large number of database change scripts; it's important that we have a place to keep them. Once a script has been written, this project will deal with organizing it among existing change scripts, and the user will never have to look at it again.
=== Change testing ===
It's important to test one's database changes before applying them to a production database (unless you happen to like disasters). Much testing is up to the user and can't be automated, but there's a few places we can help ensure at least a minimal level of schema integrity. A few examples are below; we could add more later.
It's important to test one's database changes before applying them to a production database (unless you happen to like disasters). Much testing is up to the user and can't be automated, but there's a few places we can help ensure at least a minimal level of schema integrity. A few examples are below; we could add more later.
Given an obsolete schema, a database change script, and an up-to-date schema known to be correct, this project will be able to ensure that applying the
change script to the obsolete schema will result in an up-to-date schema - all without actually changing the obsolete database. Folks who have SQLAlchemy create their database using table.create() might find this useful; this is also useful for ensuring database downgrade scripts are correct.
change script to the obsolete schema will result in an up-to-date schema - all without actually changing the obsolete database. Folks who have SQLAlchemy create their database using table.create() might find this useful; this is also useful for ensuring database downgrade scripts are correct.
Given a schema of a known version and a complete set of change scripts up to that version, this project will be able to detect if the schema matches its version. If a schema has gone through changes not present in migration scripts, this test will fail; if applying all scripts in sequence up to the specified version creates an identical schema, this test will succeed. Identifying that a schema is corrupt is sufficient; it would be nice if we could give a clue as to what's wrong, but this is lower priority. (Implementation: we'll probably show a diff of two schema dumps; this should be enough to tell the user what's gone wrong.)
Given a schema of a known version and a complete set of change scripts up to that version, this project will be able to detect if the schema matches its version. If a schema has gone through changes not present in migration scripts, this test will fail; if applying all scripts in sequence up to the specified version creates an identical schema, this test will succeed. Identifying that a schema is corrupt is sufficient; it would be nice if we could give a clue as to what's wrong, but this is lower priority. (Implementation: we'll probably show a diff of two schema dumps; this should be enough to tell the user what's gone wrong.)
== Non-Goals ==
ie. things we will '''not''' try to do (at least, during the Summer of Code)
=== Automatic generation of schema changes ===
For example, one might define a table:
For example, one might define a table:
{{{
CREATE TABLE person (
id integer,

View File

@ -69,5 +69,5 @@ One recurring problem I've had with this project is dealing with changes to the
I'm now working on another project that will be making use of SQLAlchemy: it fits many of my project's requirements, but lacks a migration tool that will be much needed. This presents an opportunity for me to make my first contribution to open source - I've long been interested in open source software and use it regularly, but haven't contributed to any until now. I'm particularly interested in the application of this tool with the TurboGears framework, as this project was inspired by a suggestion the TurboGears mailing list and I'm working on a project using TurboGears - but there is no reason to couple an SQLAlchemy enhancement with TurboGears; this project may be used by anyone who uses SQLAlchemy.
Further information:
Further information:
http://evan.zealgame.com/soc

View File

@ -2,11 +2,11 @@ This plan has several problems and has been modified; new plan is discussed in w
----
One problem with [http://www.rubyonrails.org/ Ruby on Rails'] (very good) schema migration system is the behavior of scripts that depend on outside sources; ie. the application. If those change, there's no guarantee that such scripts will behave as they did before, and you'll get strange results.
One problem with [http://www.rubyonrails.org/ Ruby on Rails'] (very good) schema migration system is the behavior of scripts that depend on outside sources; ie. the application. If those change, there's no guarantee that such scripts will behave as they did before, and you'll get strange results.
For example, suppose one defines a SQLAlchemy table:
{{{
users = Table('users', metadata,
users = Table('users', metadata,
Column('user_id', Integer, primary_key = True),
Column('user_name', String(16), nullable = False),
Column('password', String(20), nullable = False)
@ -30,7 +30,7 @@ def upgrade():
}}}
...and change our application's table definition:
{{{
users = Table('users', metadata,
users = Table('users', metadata,
Column('user_id', Integer, primary_key = True),
Column('user_name', String(16), nullable = False),
Column('password', String(20), nullable = False),

View File

@ -1,6 +1,6 @@
My original plan for Migrate's RepositoryFormat had several problems:
* Bind parameters: We needed to bind parameters into statements to get something suitable for an .sql file. For some types of parameters, there's no clean way to do this without writing an entire parser - too great a cost for this project. There's a reason why SQLAlchemy's logs display the statement and its parameters separately: the binding is done at a lower level than we have access to.
* Bind parameters: We needed to bind parameters into statements to get something suitable for an .sql file. For some types of parameters, there's no clean way to do this without writing an entire parser - too great a cost for this project. There's a reason why SQLAlchemy's logs display the statement and its parameters separately: the binding is done at a lower level than we have access to.
* Failure: Discussed in #17, the old format had no easy way to find the Python statements associated with an SQL error. This makes it difficult to debug scripts.
A new format will be used to solve this problem instead.
@ -15,9 +15,9 @@ These files will contain the following information:
* Parameters to be bound to the statement
* A Python stack trace at the point the statement was logged - this allows us to tell what Python statements are associated with an SQL statement when there's an error
These files will be created by pickling a Python object with the above information.
These files will be created by pickling a Python object with the above information.
Such files may be executed by loading the log and having SQLAlchemy execute them as it might have before.
Such files may be executed by loading the log and having SQLAlchemy execute them as it might have before.
Good:
* Since the statements and bind parameters are stored separately and executed as SQLAlchemy would normally execute them, one problem discussed above is eliminated.

View File

@ -7,9 +7,9 @@
:Author: Evan Rosson
:Maintainer: Domen Kožar <domenNO@SPAMdev.si>
:Maintainer: Jan Dittberner <jan.dittbernerNO@SPAMgooglemail.com>
:Issues: http://code.google.com/p/sqlalchemy-migrate/issues/list
:Source Code: http://code.google.com/p/sqlalchemy-migrate/
:CI Tool: http://jenkins.gnuviech-server.de/job/sqlalchemy-migrate-all/
:Source Code: https://github.com/stackforge/sqlalchemy-migrate
:Documentation: https://sqlalchemy-migrate.readthedocs.org/
:Issues: https://bugs.launchpad.net/sqlalchemy-migrate
:Generated: |today|
:License: MIT
:Version: |release|
@ -24,7 +24,7 @@
mentored by Jonathan LaCour.
The project was taken over by a small group of volunteers when Evan had no
free time for the project. It is now hosted as a `Google Code project`_.
free time for the project. It is now hosted as a `Github project`_.
During the hosting change the project was renamed to SQLAlchemy Migrate.
Currently, sqlalchemy-migrate supports Python versions from 2.6 to 2.7.
@ -55,7 +55,7 @@ Dialect support
.. list-table::
:header-rows: 1
:widths: 25 10 10 10 10 10 11
:widths: 25 10 10 10 10 10 11 10
* - Operation / Dialect
- :ref:`sqlite <sqlite-d>`
@ -64,6 +64,7 @@ Dialect support
- :ref:`oracle <oracle-d>`
- :ref:`firebird <firebird-d>`
- mssql
- DB2
* - :ref:`ALTER TABLE RENAME TABLE <table-rename>`
- yes
- yes
@ -71,6 +72,7 @@ Dialect support
- yes
- no
- not supported
- unknown
* - :ref:`ALTER TABLE RENAME COLUMN <column-alter>`
- yes (workaround) [#1]_
- yes
@ -78,6 +80,7 @@ Dialect support
- yes
- yes
- not supported
- unknown
* - :ref:`ALTER TABLE ADD COLUMN <column-create>`
- yes (workaround) [#2]_
- yes
@ -85,6 +88,7 @@ Dialect support
- yes
- yes
- not supported
- unknown
* - :ref:`ALTER TABLE DROP COLUMN <column-drop>`
- yes (workaround) [#1]_
- yes
@ -92,6 +96,7 @@ Dialect support
- yes
- yes
- not supported
- unknown
* - :ref:`ALTER TABLE ALTER COLUMN <column-alter>`
- yes (workaround) [#1]_
- yes
@ -99,6 +104,7 @@ Dialect support
- yes (with limitations) [#3]_
- yes [#4]_
- not supported
- unknown
* - :ref:`ALTER TABLE ADD CONSTRAINT <constraint-tutorial>`
- partial (workaround) [#1]_
- yes
@ -106,6 +112,7 @@ Dialect support
- yes
- yes
- not supported
- unknown
* - :ref:`ALTER TABLE DROP CONSTRAINT <constraint-tutorial>`
- partial (workaround) [#1]_
- yes
@ -113,6 +120,7 @@ Dialect support
- yes
- yes
- not supported
- unknown
* - :ref:`RENAME INDEX <index-rename>`
- no
- yes
@ -120,12 +128,13 @@ Dialect support
- yes
- yes
- not supported
- unknown
.. [#1] Table is renamed to temporary table, new table is created followed by
INSERT statements.
.. [#2] See http://www.sqlite.org/lang_altertable.html for more information.
In cases not supported my sqlite, table is renamed to temporary table,
In cases not supported by sqlite, table is renamed to temporary table,
new table is created followed by INSERT statements.
.. [#3] You can not change datatype or rename column if table has NOT NULL
data, see http://blogs.x2line.com/al/archive/2005/08/30/1231.aspx for
@ -160,12 +169,12 @@ SQLAlchemy Migrate is split into two parts, database schema versioning
glossary
.. _`google's summer of code`: http://code.google.com/soc
.. _`Google Code project`: http://code.google.com/p/sqlalchemy-migrate
.. _`Github project`: https://github.com/stackforge/sqlalchemy-migrate
.. _sqlalchemy: http://www.sqlalchemy.org
API Documentation
------------------
------------------
.. toctree::

View File

<
@ -33,7 +33,7 @@ body {
#content p,
#content ul,
#content blockquote {
#content blockquote {
line-height: 1.6em;
}
@ -55,7 +55,7 @@ h1, h2, h3 {
margin-bottom: .7em;
}
h1 {
h1 {
font-size: 2.5em;
}
h2 {
@ -73,9 +73,9 @@ h1 a, h2 a, h3 a {