Commit Graph

137 Commits

Author SHA1 Message Date
Tim Vaillancourt
1fa3d4f75a
Add basic tests for applier (#1165)
* Add basic tests for applier

* Add header
2022-09-06 15:06:14 +02:00
Tim Vaillancourt
3c946e97d7
Improve applier .ReadMigrationRangeValues() func accuracy (#1164)
* Use a transaction in applier `ReadMigrationRangeValues` func

* Private func names
2022-09-06 14:07:47 +02:00
Abirdcfly
ca8aef5d6d
chore: remove duplicate word in comments (#1175)
Signed-off-by: Abirdcfly <fp544037857@gmail.com>

Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-08-30 23:49:13 +02:00
Tim Vaillancourt
bee009b9e3
Allow zero in dates (#1161)
* Merge pull request #31 from openark/zero-date

Support zero date and zero in date, via dedicated command line flag

* Merge pull request #32 from openark/existing-date-with-zero

Support tables with existing zero dates

* Remove un-needed ignore_versions file

* Fix new lint errors from golang-ci update

Co-authored-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
2022-08-10 21:50:59 +02:00
Tim Vaillancourt
ffe54f48ed
Enable more golang-ci linters (#1149) 2022-07-18 18:37:18 +02:00
dm-2
f9fa4231d7
Merge branch 'master' into golang-ci-staticcheck-linter 2022-07-07 15:47:59 +01:00
dm-2
8eb9708755
Merge branch 'master' into changlog-table-comment 2022-07-07 15:34:30 +01:00
Tim Vaillancourt
eead61cb6d cleanup v4 2022-07-07 05:05:37 +02:00
Tim Vaillancourt
500fbefdb9 golang-ci: fix staticcheck linter warnings 2022-07-07 03:12:44 +02:00
Tim Vaillancourt
6bf32f2015
Ensure mysql rows responses are closed (#1132)
Co-authored-by: dm-2 <45519614+dm-2@users.noreply.github.com>
2022-07-06 23:32:15 +02:00
shaohk
87970f6312 fix(lost data in mysql two-phase commit): lost data in mysql two-phase commit 2022-06-24 11:38:05 +08:00
Tim Vaillancourt
387b32594e Var rename 2022-06-07 01:49:18 +02:00
Tim Vaillancourt
bf0c13b46d Add table comment to changelog 2022-06-07 01:47:51 +02:00
Andrew Mason
6e1daf90ee
Check RowsAffected when applying DML events to get more accurate statistics (#844)
* Check RowsAffected when applying DML events to get more accurate statistics

Addresses #600.

When applying a DML event, check the RowsAffected on the `Result`
struct. Since all DML event queries are point queries, this will only
ever be 0 or 1. The applier then takes this value and multiplies by
the `rowsDelta` of the event, resulting in a properly-signed, accurate
row delta to use in the statistics.

If an error occurs here, log it, but do not surface this as an
actual error .. simply assume the DML affected a row and move on. It
will be inaccurate, but this is already the case.

* Fix import

* update wording to warning log message

Co-authored-by: Tim Vaillancourt <timvaillancourt@github.com>

Co-authored-by: Tim Vaillancourt <timvaillancourt@github.com>
2021-07-14 16:48:03 +02:00
Tim Vaillancourt
47d49c6b92
Add go mod (#935)
* Add a go.mod file

* run go mod vendor again

* Move to a well-supported ini file reader

* Remove GO111MODULE=off

* Use go 1.16

* Rename github.com/outbrain/golib -> github.com/openark/golib

* Remove *.go-e files

* Fix for `strconv.ParseInt: parsing "": invalid syntax` error

* Add test for '[osc]' section

Co-authored-by: Nate Wernimont <nate.wernimont@workiva.com>
2021-06-24 20:19:37 +02:00
Tim Vaillancourt
c71dbf9ef3
Copy auto increment (#967)
* v1.1.0

* WIP: copying AUTO_INCREMENT value to ghost table
Initial commit: towards setting up a test suite

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* greping for 'expect_table_structure' content

* Adding simple test for 'expect_table_structure' scenario

* adding tests for AUTO_INCREMENT value after row deletes. Should initially fail

* clear event beforehand

* parsing AUTO_INCREMENT from alter query, reading AUTO_INCREMENT from original table, applying AUTO_INCREMENT value onto ghost table if applicable and user has not specified AUTO_INCREMENT in alter statement

* support GetUint64

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* minor update to test

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* adding test for user defined AUTO_INCREMENT statement

Co-authored-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
2021-05-14 15:32:56 +02:00
Tim Vaillancourt
048d5838db
Fix whitespace after merge-conflict fix 2021-02-08 13:37:39 +01:00
Tim Vaillancourt
edbc8d6733
Merge branch 'master' into improve-conn-log 2021-02-08 13:35:30 +01:00
Tim Vaillancourt
88e59dd2a3
Merge branch 'master' into master 2020-10-22 23:22:32 +02:00
Tim Vaillancourt
0e2d33ad86 Merge in https://github.com/github/gh-ost/pull/755 2020-10-20 16:08:49 +02:00
Tim Vaillancourt
f268259f5e
Merge branch 'master' into handle_driver_timeout_error 2020-08-19 22:04:55 +02:00
Tim Vaillancourt
4bbc8deb77
Fix tabs from merge conflict 2020-08-13 15:50:38 +02:00
Tim Vaillancourt
5b6da5b7f3
Merge branch 'master' into handle_driver_timeout_error 2020-08-13 15:49:34 +02:00
Shlomi Noach
c9249f2b71 Updating and using AlterTableOptions 2020-07-23 11:38:05 +03:00
Andrew Mason
90ad7a061f Handle the rest of rows.Err cases 2020-04-21 12:50:23 -04:00
Andrew Mason
61de098072 Add a check to rows.Err after processing all rows
Closes #822.

In https://github.com/go-sql-driver/mysql/issues/1075, @acharis notes
that the way the go-sql driver is written, query timeout errors don't
get set in `rows.Err()` until _after_ a call to `rows.Next()` is made.

Because this kind of error means there will be no rows in the result
set, the `for rows.Next()` will never enter the for loop, so we must
check the value of `rows.Err()` after the loop, and surface the error up
appropriately.
2020-03-31 16:25:16 -04:00
Shuode Li
ae762694fb Support Azure Database for MySQL. 2019-12-17 03:47:14 +00:00
Abeyu M
f4676bf463 implement a logging interface 2019-10-07 11:10:36 -04:00
Shlomi Noach
f0ed7aeb0b Always use NO_AUTO_VALUE_ON_ZERO 2019-03-24 11:32:42 +02:00
babinomec
7d5749b84a add --skip-strict-mode option 2019-02-14 15:58:49 +01:00
Brandon Bodnar
23617f287f Add initial support for ssl encryption connections to database servers.
- Adding a command line option for users to enforce tls/ssl connections
  for the applier, inspector, and binlog reader.
- The user can optionally request server certificate verification through
  a command line option to specify the ca cert via a file path.
- Fixes an existing bug appending the timeout option to the singleton
  applier connection.
2019-01-31 15:22:03 -06:00
Shlomi Noach
97c1e72dbe
Merge branch 'master' into minor_changes 2018-09-17 08:58:04 +03:00
Shlomi Noach
32f1cf1dbe Support for GCP (Google Cloud Platform) 2018-09-02 09:51:40 +03:00
MOON_CLJ
21d455013e minor changes and typo correction 2018-05-23 13:13:25 +08:00
Shlomi Noach
db871b42c4 Support for GENERATED (aka virtual) columns 2018-05-22 12:36:52 +03:00
Shlomi Noach
dfeefed5dc avoid prepared statements for changelog table 2018-05-20 13:46:33 +03:00
zj118228
d3b98d9e74 Refactor the code and remove the global variable of
support-aliyun-rds branch.
2018-03-29 12:20:41 +08:00
zj118228
6f3d54a136 Aliyun RDS hide some mysql variables for secrity so gh-ost will get
ilegal values. Add --aliyun-rds flag to avoid getting them.
2018-01-23 14:44:22 +08:00
Nikhil Mathew
ec6ceffbcb Pass in a migrationContext UUID for a migration specific connections cache 2017-12-14 15:53:29 -08:00
Shlomi Noach
0530b09700
Merge branch 'master' into nm-refactor-migration-context 2017-12-07 13:59:12 +02:00
Shlomi Noach
58c381f0a0 some cleanup 2017-11-21 09:14:04 +02:00
Shlomi Noach
203ea6c2cd refactor, simplifid code 2017-11-20 10:24:28 +02:00
Shlomi Noach
3898d49f6c supporting update to columns of migration key 2017-11-20 08:17:20 +02:00
Nikhil Mathew
538833ea84
Merge branch 'master' into nm-refactor-migration-context 2017-11-14 10:29:49 -08:00
Nikhil Mathew
2ad65ee1d7 Refactor usages of GetDB to use mysql 2017-11-10 15:44:22 -08:00
Shlomi Noach
8a59d7e823 failing on PK value change 2017-11-08 11:11:17 +02:00
Josh Soref
e2df0b80eb spelling: iterating 2017-11-08 00:49:23 +00:00
Josh Soref
418462a6ca spelling: dropping 2017-11-08 00:47:15 +00:00
Akshay Chhajed
ccb7654235 Improved connection type logging 2017-10-29 19:53:32 +05:30
Nikhil Mathew
84bdfdb1ad Cache DB connection pools on the migrationContext where applicable 2017-09-22 16:06:06 -07:00