* Cancel any row count queries before attempting to cut over
Closes#830. Switches from using `QueryRow` to `QueryRowContext`, and
stores a context.CancelFunc in the migration context, which is called to
halt any running row count query before beginning the cut over.
* Make it threadsafe
* Kill the count query on the database side as well
* Explicitly grab a connection to run the count, store its connection id
* When the query context is canceled, run a `KILL QUERY ?` on that connection id
* Rewrite these to use the threadsafe functions, stop exporting the cancel func
* Update logger
* Update logger
Co-authored-by: Tim Vaillancourt <timvaillancourt@github.com>
Co-authored-by: Tim Vaillancourt <tim@timvaillancourt.com>
Co-authored-by: dm-2 <45519614+dm-2@users.noreply.github.com>
* Add context/timeout to HTTP throttle check
* Dont run `.GetThrottleHTTPInterval()` on every loop
* Update help message
* Var rename
* 2022
* Add timeout flag
* Add unix/tcp server commands, use ParseInt() for string->int64
* Var rename
* Re-check http timeout on every loop iteration
* Remove stale comment
* Make throttle interval idempotent
* var rename
* Usage grammar
* Make http timeout idempotent too
* Parse time.Duration once
* Move timeout to NewThrottler
* Help update
* Set User-Agent header
* Re-add newline
Co-authored-by: dm-2 <45519614+dm-2@users.noreply.github.com>
* Add a go.mod file
* run go mod vendor again
* Move to a well-supported ini file reader
* Remove GO111MODULE=off
* Use go 1.16
* Rename github.com/outbrain/golib -> github.com/openark/golib
* Remove *.go-e files
* Fix for `strconv.ParseInt: parsing "": invalid syntax` error
* Add test for '[osc]' section
Co-authored-by: Nate Wernimont <nate.wernimont@workiva.com>
* v1.1.0
* WIP: copying AUTO_INCREMENT value to ghost table
Initial commit: towards setting up a test suite
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
* greping for 'expect_table_structure' content
* Adding simple test for 'expect_table_structure' scenario
* adding tests for AUTO_INCREMENT value after row deletes. Should initially fail
* clear event beforehand
* parsing AUTO_INCREMENT from alter query, reading AUTO_INCREMENT from original table, applying AUTO_INCREMENT value onto ghost table if applicable and user has not specified AUTO_INCREMENT in alter statement
* support GetUint64
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
* minor update to test
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
* adding test for user defined AUTO_INCREMENT statement
* Generated column as part of UNIQUE (or PRIMARY) KEY
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
* skip analysis of generated column data type in unique key
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
* All MySQL DBs limited to max 3 concurrent/idle connections
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
* hooks: reporting GH_OST_ETA_SECONDS. ETA stored as part of migration context
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
* GH_OST_ETA_NANOSECONDS
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
* N/A denoted by negative value
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
* ETAUnknown constant
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
* v1.1.0
* WIP: copying AUTO_INCREMENT value to ghost table
Initial commit: towards setting up a test suite
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
* greping for 'expect_table_structure' content
* Adding simple test for 'expect_table_structure' scenario
* adding tests for AUTO_INCREMENT value after row deletes. Should initially fail
* clear event beforehand
* parsing AUTO_INCREMENT from alter query, reading AUTO_INCREMENT from original table, applying AUTO_INCREMENT value onto ghost table if applicable and user has not specified AUTO_INCREMENT in alter statement
* support GetUint64
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
* minor update to test
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
* adding test for user defined AUTO_INCREMENT statement
Co-authored-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Related issue: https://github.com/github/gh-ost/issues/521
- Add --ssl-cert and --ssl-key options to specify SSL public/private
key files
- Allow combining --ssl-allow-insecure with other --ssl* flags.
`mysql.RegisterTLSConfig` allows combining the corresponding
parameters in the `tls.Config` it receives, so gh-ost should
allow this. I found being able to pass --ssl-allow-insecure along
with --ssl-ca, --ssl-cert, and --ssl-key useful in testing.
- Use the same TLS config everywhere. Since the CLI only supports
a single set of --ssl* configuration parameters, this should be
fine -- `mysql.RegisterTLSConfig` documentation indicates the
TLS config given will not be modified, so it can safely be used
in many goroutines provided we also do not modify it. The previous
implementation did not work when the TLS config was duplicated,
which happens when gh-ost walks up the replication chain trying
to find the master. This is because, when the config is duplicated,
we must call `RegisterTLSConfig` again with the new config. This
config is exactly the same, so it's easiest to side-step the issue
by registering the TLS config once and using it everywhere.
- Adding a command line option for users to enforce tls/ssl connections
for the applier, inspector, and binlog reader.
- The user can optionally request server certificate verification through
a command line option to specify the ca cert via a file path.
- Fixes an existing bug appending the timeout option to the singleton
applier connection.
