Merge branch 'master' into enum-sort-test

This commit is contained in:
Shlomi Noach 2016-10-20 10:16:02 +02:00
commit 82a76ae879
4 changed files with 26 additions and 12 deletions

View File

@ -4,6 +4,8 @@
- You will need to have one server serving Row Based Replication (RBR) format binary logs. Right now `FULL` row image is supported. `MINIMAL` to be supported in the near future. `gh-ost` prefers to work with replicas. You may [still have your master configured with Statement Based Replication](migrating-with-sbr.md) (SBR).
- If you are using a replica, the table must have an identical schema between the master and replica.
- `gh-ost` requires an account with these privileges:
- `ALTER, CREATE, DELETE, DROP, INDEX, INSERT, LOCK TABLES, SELECT, TRIGGER, UPDATE` on the database (schema) where your migrated table is, or of course on `*.*`
@ -15,21 +17,30 @@ The `SUPER` privilege is required for `STOP SLAVE`, `START SLAVE` operations. Th
- Switching your `binlog_format` to `ROW`, in the case where it is _not_ `ROW` and you explicitly specified `--switch-to-rbr`
- If your replication is already in RBR (`binlog_format=ROW`) you can specify `--assume-rbr` to avoid the `STOP SLAVE/START SLAVE` operations, hence no need for `SUPER`.
- Running `--test-on-replica`: before the cut-over phase, `gh-ost` stops replication so that you can compare the two tables and satisfy that the migration is sound.
### Limitations
- Foreign keys not supported. They may be supported in the future, to some extent.
- Triggers are not supported. They may be supported in the future.
- MySQL 5.7 generated columns are not supported. They may be supported in the future.
- The two _before_ & _after_ tables must share some `UNIQUE KEY`. Such key would be used by `gh-ost` to iterate the table.
- As an example, if your table has a single `UNIQUE KEY` and no `PRIMARY KEY`, and you wish to replace it with a `PRIMARY KEY`, you will need two migrations: one to add the `PRIMARY KEY` (this migration will use the existing `UNIQUE KEY`), another to drop the now redundant `UNIQUE KEY` (this migration will use the `PRIMARY KEY`).
- The chosen migration key must not include columns with `NULL` values.
- `gh-ost` will do its best to pick a migration key with non-nullable columns. It will by default refuse a migration where the only possible `UNIQUE KEY` includes nullable-columns. You may override this refusal via `--allow-nullable-unique-key` but **you must** be sure there are no actual `NULL` values in those columns. Such `NULL` values would cause a data integrity problem and potentially a corrupted migration.
- It is not allowed to migrate a table where another table exists with same name and different upper/lower case.
- For example, you may not migrate `MyTable` if another table called `MYtable` exists in the same schema.
- Amazon RDS and Google Cloud SQL are currently not supported
- We began working towards removing this limitation. See tracking issue: https://github.com/github/gh-ost/issues/163
- Multisource is not supported when migrating via replica. It _should_ work (but never tested) when connecting directly to master (`--allow-on-master`)
- Master-master setup is only supported in active-passive setup. Active-active (where table is being written to on both masters concurrently) is unsupported. It may be supported in the future.
- If you have en `enum` field as part of your migration key (typically the `PRIMARY KEY`), migration performance will be degraded and potentially bad. [Read more](https://github.com/github/gh-ost/pull/277#issuecomment-254811520)

View File

@ -4,24 +4,27 @@ Here are technical considerations you may be interested in. We write here things
# Connecting to replica
`gh-ost` prefers connecting to replica. If your master uses Statement Based Replication, this is a _requirement_.
`gh-ost` prefers connecting to a replica. If your master uses Statement Based Replication, this is a _requirement_.
What does "connect to replica" mean?
- `gh-ost` connects to the replica as a normal client
- It additionally connects as a replica to the replica (pretends to be a MySQL replica itself)
- It auto-detects master
- It auto-detects the master
`gh-ost` reads the RBR binary logs from the replica, and applies events onto the master as tables are being migrated.
`gh-ost` reads the RBR binary logs from the replica, and applies events onto the master as part of the table migration.
THE FINE PRINT:
- You trust the replica's binary logs to represent events applied on master.
If you don't trust the replica, if you suspect there's data drift between replica & master, take notice. If your master is RBR, do instead connect `gh-ost` to master, via `--allow-on-master` (see [cheatsheet](cheatsheet.md)).
Our take: we trust replica data; if master dies in production, we promote a replica. Our read serving is based on replica(s).
- If you don't trust the replica, or if you suspect there's data drift between replica & master, take notice.
- If the table on the replica has a different schema than the master, `gh-ost` likely won't work correctly.
- Our take: we trust replica data; if master dies in production, we promote a replica. Our read serving is based on replica(s).
- If your master is RBR, do instead connect `gh-ost` to master, via `--allow-on-master` (see [cheatsheet](cheatsheet.md)).
- Replication needs to run.
This is an obvious, but worth stating. You cannot perform a migration with "connect to replica" if your replica lags. `gh-ost` will actually do all it can so that replication does not lag, and avoid critical operations at such time when replication does lag.
- This is an obvious, but worth stating. You cannot perform a migration with "connect to replica" if your replica lags. `gh-ost` will actually do all it can so that replication does not lag, and avoid critical operations if replication is lagging.
# Network usage
@ -30,12 +33,12 @@ THE FINE PRINT:
THE FINE PRINT:
- `gh-ost` delivers more network traffic than other online-schema-change tools, that let MySQL handle all data transfer internally. This is part of the [triggerless design](triggerless-design.md).
Our take: we deal with cross-DC migration traffic and this is working well for us.
- Our take: we deal with cross-DC migration traffic and this is working well for us.
# Impersonating as a replica
`gh-ost` impersonates as a replica: connects to a MySQL server, says "oh hey, I'm a replica, please send me binary logs kthx".
`gh-ost` impersonates as a replica: it connects to a MySQL server, says "oh hey, I'm a replica, please send me binary logs kthx".
THE FINE PRINT:
- `SHOW SLAVE HOSTS` or `SHOW PROCESSLIST` will list down this strange "replica" that you can't really connect to.
- `SHOW SLAVE HOSTS` or `SHOW PROCESSLIST` will list this strange "replica" that you can't really connect to.

View File

@ -33,7 +33,7 @@ You're always able to actively begin [throttling](throttle.md). Just touch the `
### What if my replicas don't use binary logs?
If the master is running Row Based Replication (RBR) - point `gh-ost` to the master, and specify `--allow-on-master`. See [cheatsheets](cheatsheets.md)
If the master is running Row Based Replication (RBR) - point `gh-ost` to the master, and specify `--allow-on-master`. See [cheatsheets](cheatsheet.md)
If the master is running Statement Based Replication (SBR) - you have no alternative but to reconfigure a replica with:

View File

@ -327,7 +327,7 @@ func (this *Inspector) validateLogSlaveUpdates() error {
if err := this.db.QueryRow(query).Scan(&logSlaveUpdates); err != nil {
return err
}
if !logSlaveUpdates && !this.migrationContext.InspectorIsAlsoApplier() {
if !logSlaveUpdates && !this.migrationContext.InspectorIsAlsoApplier() && !this.migrationContext.IsTungsten {
return fmt.Errorf("%s:%d must have log_slave_updates enabled", this.connectionConfig.Key.Hostname, this.connectionConfig.Key.Port)
}