more documentation

- what if?
- requirements and limitations
- more abotu variables
- logo

doc/command-line-flags.md

@@ -2,7 +2,21 @@
 
 A more in-depth discussion of various `gh-ost` command line flags: implementation, implication, use cases.
 
-##### conf
+#### allow-on-master
+
+By default, `gh-ost` would like you to connect to a replica, from where it figures out the master by itself. This wiring is required should your master execute using `binlog_format=STATEMENT`.
+
+If, for some reason, you do not wish `gh-ost` to connect to a replica, you may connect it directly to the master and approve this via `--allow-on-master`.
+
+#### approve-renamed-columns
+
+When your migration issues a column rename (`change column old_name new_name ...`) `gh-ost` analyzes the statement to try an associate the old column name with new column name. Otherwise the new structure may also look like some column was dropped and another was added.
+
+`gh-ost` will print out what it thinks the _rename_ implied, but will not issue the migration unless you provide with `--approve-renamed-columns`.
+
+If you think `gh-ost` is mistaken and that there's actually no _rename_ involved, you may pass `--skip-renamed-columns` instead. This will cause `gh-ost` to disassociate the column values; data will not be copied between those columns.
+
+#### conf
 
 `--conf=/path/to/my.cnf`: file where credentials are specified. Should be in (or contain) the following format:
 
@@ -12,11 +26,11 @@ user=gromit
 password=123456
   ```
 
-##### cut-over
+#### cut-over
 
 Optional. Default is `safe`. See more discussion in [cut-over](cut-over.md)
 
-##### exact-rowcount
+#### exact-rowcount
 
 A `gh-ost` execution need to copy whatever rows you have in your existing table onto the ghost table. This can, and often be, a large number. Exactly what that number is?
 `gh-ost` initially estimates the number of rows in your table by issuing an `explain select * from your_table`. This will use statistics on your table and return with a rough estimate. How rough? It might go as low as half or as high as double the actual number of rows in your table. This is the same method as used in [`pt-online-schema-change`](https://www.percona.com/doc/percona-toolkit/2.2/pt-online-schema-change.html).
@@ -29,20 +43,28 @@ A `gh-ost` execution need to copy whatever rows you have in your existing table
 
 While the ongoing estimated number of rows is still heuristic, it's almost exact, such that the reported  [ETA](understanding-output.md) or percentage progress is typically accurate to the second throughout a multiple-hour operation.
 
-##### execute
+#### execute
 
 Without this parameter, migration is a _noop_: testing table creation and validity of migration, but not touching data.
 
-##### initially-drop-ghost-table
+#### initially-drop-ghost-table
 
 `gh-ost` maintains two tables while migrating: the _ghost_ table (which is synced from your original table and finally replaces it) and a changelog table, which is used internally for bookkeeping. By default, it panics and aborts if it sees those tables upon startup. Provide `--initially-drop-ghost-table` and `--initially-drop-old-table` to let `gh-ost` know it's OK to drop them beforehand.
 
 We think `gh-ost` should not take chances or make assumptions about the user's tables. Dropping tables can be a dangerous, locking operation. We let the user explicitly approve such operations.
 
-##### initially-drop-old-table
+#### initially-drop-old-table
 
 See #initially-drop-ghost-table
 
-##### test-on-replica
+#### migrate-on-replica
+
+Typically `gh-ost` is used to migrate tables on a master. If you wish to only perform the migration in full on a replica, connect `gh-ost` to said replica and pass `--migrate-on-replica`. `gh-ost` will briefly connect to the master but other issue no changes on the master. Migration will be fully executed on the replica, while making sure to maintain a small replication lag.
+
+#### skip-renamed-columns
+
+See `approve-renamed-columns`
+
+#### test-on-replica
 
 Issue the migration on a replica; do not modify data on master. Useful for validating, testing and benchmarking. See [test-on-replica](test-on-replica.md)

doc/requirements-and-limitations.md

@@ -0,0 +1,23 @@
+# Requirements and limitations
+
+### Requirements
+
+- You will need to have one server serving Row Based Replication (RBR) format binary logs. `gh-ost` prefers to work with replicas. You may [still have your master configured with Statement Based Replication](migrating-with-sbr) (SBR).
+
+- `gh-ost` requires an account with these privileges:
+
+  - `ALTER, CREATE, DELETE, DROP, INDEX, INSERT, LOCK TABLES, SELECT, TRIGGER, UPDATE` on the database (schema) where your migrated table is, or of course on `*.*`
+  - `SUPER, REPLICATION SLAVE` on `*.*`
+
+### Limitations
+
+- Foreign keys not supported. They may be supported in the future, to some extent.
+- Triggers are not supported. they may be supported in the future.
+- The two _before_ & _after_ tables must share some `UNIQUE KEY`. Such key would be used by `gh-ost` to iterate the table.
+  - As an example, if your table has a single `UNIQUE KEY` and no `PRIMARY KEY`, and you wish to replace it with a `PRIMARY KEY`, you will need two migrations: one to add the `PRIMARY KEY` (this migration will use the existing `UNIQUE KEY`), another to drop the now redundant `UNIQUE KEY` (this migration will use the `PRIMARY KEY`).
+- The chosen migration key must not include columns with `NULL` values.
+  - `gh-ost` will do its best to pick a migration key with non-nullable columns. It will by default refuse a migration where the only possible `UNIQUE KEY` includes nullable-columns. You may override this refusal via `--allow-nullable-unique-key` but **you must** be sure there are no actual `NULL` values in those columns. Such `NULL` values would cause a data integrity problem and potentially a corrupted migration.
+- It is not allowed to migrate a table where another table exists with same name and different upper/lower case.
+  - For example, you may not migrate `MyTable` if another table called `MYtable` exists in the same schema.
+- Amazon `RDS` is probably not supported (due to `SUPER` requirement)
+- Multisource is not supported when migrating via replica. It _should_ work (but never tested) when connecting directly to master (`--allow-on-master`)

doc/what-if.md

@@ -0,0 +1,32 @@
+# What if?
+
+Technical questions and answers. This document will be updated as we go
+
+### What if I'm using Statement Based Replication?
+
+You can still migrate tables with `gh-ost`. We do that. What you will need is a replica configured with:
+
+- `log_bin`
+- `log_slave_updates`
+- `binlog_format=ROW`
+
+Thus, the replica will transform the master's SBR binlogs into RBR binlogs. `gh-ost` is happy to read the binary logs from the replica. [Read more](migrating-with-sbr.md)
+
+### What if gh-ost crashes halfway through, or I kill it?
+
+Unlike trigger-based solutions, there's nothing urgent to clean up in the event `gh-ost` bails out or gets killed. There are the two tables creates by `gh-ost`:
+
+- The _ghost_ table: `_yourtablename_gho`
+- The _changelog_ table: `_yourtablename_ghc`
+
+You may instruct `gh-ost` to drop these tables upon startup; or better yet, you drop them.
+
+### What if the cut-over (table switch) is unable to proceed due to locks/timeout?
+
+There is a `lock_wait_timeout` explicitly associated with the cut-over operation. If your table suddenly suffers from a long running query, the cut-over (involving `LOCK` and `RENAME` statements) may be unable to proceed. There's a finite number of retries, and if none of these succeeds, `gh-ost` bails out.
+
+### What if the migration is causing a high load on my master?
+
+This is where `gh-ost` shines. There is no need to kill it as you may be used to with other tools. You can reconfigure `gh-ost` [on the fly](https://github.com/github/gh-ost/blob/master/doc/interactive-commands.md) to be nicer.
+
+You're always able to actively begin [throttling](throttle.md). Just touch the `throttle-file` or `echo throttle` into `gh-ost`. Otherwise, reconfigure your `max-load`, the `nice-ratio`, the `throttle-query` to gain better thresholds that would suit your needs.

doc/why-triggerless.md

@@ -12,7 +12,6 @@ are all using [triggers](http://dev.mysql.com/doc/refman/5.6/en/triggers.html) t
 Use of triggers simplifies a lot of the flow in doing a live table migration, but also poses some limitations or difficulties. Here are reasons why we choose to [design a triggerless solution](triggerless-design.md) to schema migrations.
 
 
-
 ## Overview
 
 Triggers are stored routines which are invoked on a per-row operation upon `INSERT`, `DELETE`, `UPDATE` on a table.