From 6c0d7d1f6406973a105aa25a8ed201b953eeed99 Mon Sep 17 00:00:00 2001 From: Gillian Gunson Date: Wed, 19 Oct 2016 05:16:41 -0700 Subject: [PATCH 1/5] added requirement about identical table definitions --- doc/requirements-and-limitations.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/requirements-and-limitations.md b/doc/requirements-and-limitations.md index ae577f2..f8272a0 100644 --- a/doc/requirements-and-limitations.md +++ b/doc/requirements-and-limitations.md @@ -4,6 +4,8 @@ - You will need to have one server serving Row Based Replication (RBR) format binary logs. Right now `FULL` row image is supported. `MINIMAL` to be supported in the near future. `gh-ost` prefers to work with replicas. You may [still have your master configured with Statement Based Replication](migrating-with-sbr.md) (SBR). +- If you are using a replica, the table must have an identical schema between the master and replica. + - `gh-ost` requires an account with these privileges: - `ALTER, CREATE, DELETE, DROP, INDEX, INSERT, LOCK TABLES, SELECT, TRIGGER, UPDATE` on the database (schema) where your migrated table is, or of course on `*.*` From 7d313467f10d9c9346bfcf39dfffd92bc2d1884f Mon Sep 17 00:00:00 2001 From: Gillian Gunson Date: Wed, 19 Oct 2016 05:18:21 -0700 Subject: [PATCH 2/5] unified spacing of notes through page --- doc/requirements-and-limitations.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/doc/requirements-and-limitations.md b/doc/requirements-and-limitations.md index f8272a0..8da1cb0 100644 --- a/doc/requirements-and-limitations.md +++ b/doc/requirements-and-limitations.md @@ -17,20 +17,29 @@ The `SUPER` privilege is required for `STOP SLAVE`, `START SLAVE` operations. Th - Switching your `binlog_format` to `ROW`, in the case where it is _not_ `ROW` and you explicitly specified `--switch-to-rbr` - If your replication is already in RBR (`binlog_format=ROW`) you can specify `--assume-rbr` to avoid the `STOP SLAVE/START SLAVE` operations, hence no need for `SUPER`. + - Running `--test-on-replica`: before the cut-over phase, `gh-ost` stops replication so that you can compare the two tables and satisfy that the migration is sound. ### Limitations - Foreign keys not supported. They may be supported in the future, to some extent. + - Triggers are not supported. They may be supported in the future. + - MySQL 5.7 generated columns are not supported. They may be supported in the future. + - The two _before_ & _after_ tables must share some `UNIQUE KEY`. Such key would be used by `gh-ost` to iterate the table. - As an example, if your table has a single `UNIQUE KEY` and no `PRIMARY KEY`, and you wish to replace it with a `PRIMARY KEY`, you will need two migrations: one to add the `PRIMARY KEY` (this migration will use the existing `UNIQUE KEY`), another to drop the now redundant `UNIQUE KEY` (this migration will use the `PRIMARY KEY`). + - The chosen migration key must not include columns with `NULL` values. - `gh-ost` will do its best to pick a migration key with non-nullable columns. It will by default refuse a migration where the only possible `UNIQUE KEY` includes nullable-columns. You may override this refusal via `--allow-nullable-unique-key` but **you must** be sure there are no actual `NULL` values in those columns. Such `NULL` values would cause a data integrity problem and potentially a corrupted migration. + - It is not allowed to migrate a table where another table exists with same name and different upper/lower case. - For example, you may not migrate `MyTable` if another table called `MYtable` exists in the same schema. + - Amazon RDS and Google Cloud SQL are currently not supported - We began working towards removing this limitation. See tracking issue: https://github.com/github/gh-ost/issues/163 + - Multisource is not supported when migrating via replica. It _should_ work (but never tested) when connecting directly to master (`--allow-on-master`) + - Master-master setup is only supported in active-passive setup. Active-active (where table is being written to on both masters concurrently) is unsupported. It may be supported in the future. From 48c2b6dc14e84f35928ee7331858e39afe8b4e85 Mon Sep 17 00:00:00 2001 From: Gillian Gunson Date: Wed, 19 Oct 2016 05:29:34 -0700 Subject: [PATCH 3/5] Added note about table schema needing to be identical --- doc/the-fine-print.md | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/doc/the-fine-print.md b/doc/the-fine-print.md index 3f1fe40..d00d531 100644 --- a/doc/the-fine-print.md +++ b/doc/the-fine-print.md @@ -4,24 +4,27 @@ Here are technical considerations you may be interested in. We write here things # Connecting to replica -`gh-ost` prefers connecting to replica. If your master uses Statement Based Replication, this is a _requirement_. +`gh-ost` prefers connecting to a replica. If your master uses Statement Based Replication, this is a _requirement_. What does "connect to replica" mean? - `gh-ost` connects to the replica as a normal client - It additionally connects as a replica to the replica (pretends to be a MySQL replica itself) -- It auto-detects master +- It auto-detects the master -`gh-ost` reads the RBR binary logs from the replica, and applies events onto the master as tables are being migrated. +`gh-ost` reads the RBR binary logs from the replica, and applies events onto the master as part of the table migration. THE FINE PRINT: - You trust the replica's binary logs to represent events applied on master. - If you don't trust the replica, if you suspect there's data drift between replica & master, take notice. If your master is RBR, do instead connect `gh-ost` to master, via `--allow-on-master` (see [cheatsheet](cheatsheet.md)). - Our take: we trust replica data; if master dies in production, we promote a replica. Our read serving is based on replica(s). - + - If you don't trust the replica, or if you suspect there's data drift between replica & master, take notice. + - If the table on the replica has a different schema than the master, `gh-ost` likely won't work correctly. + - Our take: we trust replica data; if master dies in production, we promote a replica. Our read serving is based on replica(s). + +- If your master is RBR, do instead connect `gh-ost` to master, via `--allow-on-master` (see [cheatsheet](cheatsheet.md)). + - Replication needs to run. - This is an obvious, but worth stating. You cannot perform a migration with "connect to replica" if your replica lags. `gh-ost` will actually do all it can so that replication does not lag, and avoid critical operations at such time when replication does lag. + - This is an obvious, but worth stating. You cannot perform a migration with "connect to replica" if your replica lags. `gh-ost` will actually do all it can so that replication does not lag, and avoid critical operations if replication is lagging. # Network usage @@ -30,12 +33,12 @@ THE FINE PRINT: THE FINE PRINT: - `gh-ost` delivers more network traffic than other online-schema-change tools, that let MySQL handle all data transfer internally. This is part of the [triggerless design](triggerless-design.md). - Our take: we deal with cross-DC migration traffic and this is working well for us. + - Our take: we deal with cross-DC migration traffic and this is working well for us. # Impersonating as a replica -`gh-ost` impersonates as a replica: connects to a MySQL server, says "oh hey, I'm a replica, please send me binary logs kthx". +`gh-ost` impersonates as a replica: it connects to a MySQL server, says "oh hey, I'm a replica, please send me binary logs kthx". THE FINE PRINT: -- `SHOW SLAVE HOSTS` or `SHOW PROCESSLIST` will list down this strange "replica" that you can't really connect to. +- `SHOW SLAVE HOSTS` or `SHOW PROCESSLIST` will list this strange "replica" that you can't really connect to. From 559c7895ca186551f7da6d5f5ed1fe057129f9f0 Mon Sep 17 00:00:00 2001 From: Akshay Chhajed Date: Wed, 19 Oct 2016 23:31:16 +0530 Subject: [PATCH 4/5] Modified slave update check to test tungsten flag --- go/logic/inspect.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/go/logic/inspect.go b/go/logic/inspect.go index 1b6f7f0..ceb5c16 100644 --- a/go/logic/inspect.go +++ b/go/logic/inspect.go @@ -327,7 +327,7 @@ func (this *Inspector) validateLogSlaveUpdates() error { if err := this.db.QueryRow(query).Scan(&logSlaveUpdates); err != nil { return err } - if !logSlaveUpdates && !this.migrationContext.InspectorIsAlsoApplier() { + if !logSlaveUpdates && !this.migrationContext.InspectorIsAlsoApplier() && !this.migrationContext.IsTungsten { return fmt.Errorf("%s:%d must have log_slave_updates enabled", this.connectionConfig.Key.Hostname, this.connectionConfig.Key.Port) } From 3c767b3611d95ecfa116f530021e3e13075f04eb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Fernando=20=C3=81lvarez?= Date: Wed, 19 Oct 2016 23:41:00 +0200 Subject: [PATCH 5/5] doc: fix link to cheatsheet on What If page --- doc/what-if.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/what-if.md b/doc/what-if.md index 62db3ee..a097b34 100644 --- a/doc/what-if.md +++ b/doc/what-if.md @@ -33,7 +33,7 @@ You're always able to actively begin [throttling](throttle.md). Just touch the ` ### What if my replicas don't use binary logs? -If the master is running Row Based Replication (RBR) - point `gh-ost` to the master, and specify `--allow-on-master`. See [cheatsheets](cheatsheets.md) +If the master is running Row Based Replication (RBR) - point `gh-ost` to the master, and specify `--allow-on-master`. See [cheatsheets](cheatsheet.md) If the master is running Statement Based Replication (SBR) - you have no alternative but to reconfigure a replica with: