Commit Graph

15 Commits

Author SHA1 Message Date
Shlomi Noach
9ac468c97e failing on DATETIME-to-TIMESTAMP conversion within chosen key 2016-10-14 09:33:53 +02:00
Shlomi Noach
882959ba83 column is type-aware; is timezone-conversion aware
also refactored some functions
2016-10-13 13:12:33 +02:00
Shlomi Noach
f4693274fb Extracted and generalized encoding logic to encoding.go 2016-09-08 09:37:22 +02:00
Shlomi Noach
791d963ea0 Character set recognition and manipulation
- Identifying textual characters sets; converting into specific type when applying dml events
- Refactored `ColumnsList`: introducing `Column` type
- Refactored `unsigned` handling, as part of `Column`
- `Column` type supports `convertArg()`: converting value of argument according to column data type
- DB URI attempts `utf8mb4,utf8,latin1` charsets in that order (first one to be recognized wins)
- Local tests filter by pattern
- Local tests append table schema on failure
- Local tests do not have postpone flag file
- Added character set local tests: `utf8`, `utf8mb4`, `latin1`
2016-09-07 14:24:11 +02:00
Shlomi Noach
7a0e2cfe3f adding UnsignedFlags to ColumnsList 2016-08-17 06:49:20 +02:00
Daniël van Eeden
7786356a45 Fix typo: s/auto_incrmenet/auto_increment/ 2016-08-05 17:46:38 +02:00
Shlomi Noach
9d055dbda7 renaming to gh-ost 2016-05-16 11:09:17 +02:00
Shlomi Noach
eeffa701d6 - Added ok-to-drop-table flag
- Added `switch-to-rbr` flag; applying binlog format change if needed
- Using dedicated db instance for locking & renaming on applier (must be used from within same connection)
- Heartbeat now uses `time.RFC3339Nano`
- Swap tables works! Caveat: short table outage
- `--test-on-replica` works!
- retries: using `panicAbort`: from any goroutine, regardless of context, it is possible to terminate the operation
- Reintroduced changelog events listener on streamer. This is the correct implementation.
2016-04-18 10:57:18 -07:00
Shlomi Noach
04525887f3 - Throttling-check is now an async routine running once per second
- Throttling variables protected by mutex
- Added `--throttle-additional-flag-file`: `operation pauses when this file exists; hint: keep default, use for throttling multiple gh-osc operations`
- ColumnList is not a `struct` which contains ordinal mapping
- More implicit write changelog + audit changelog
- builder now builds `DELETE` and `INSERT` queries from data it will eventually get from DML event
- Sanity check for binlog_row_image
- Restarting replication to be sure binlog settings apply
- Prepare for accepting `SIGHUP` (reloading configuration)
2016-04-11 17:27:16 +02:00
Shlomi Noach
a1a34b8150 ongoing development:
- accepts --max-load
- accepts multiple conditions in --max-load
- throttle includes reason
- chunk-size sanity check
- change log state writes both in appending (history) mode and in replacing (current) mode
- more atomic checks
- inspecting ghost table columns, unique key
- comparing unique keys between tables; sanity
- intersecting columns between tables
- prettify status
- refactored throttle() and retries()
2016-04-08 14:35:06 +02:00
Shlomi Noach
0e7b23e6fe - Creating an populating Changelog table
- Using heartbeat
- Throttling works based on heartbeat
- Refactored binlog_reader stuff. Now streaming events (into golang channel, which makes for nice buffering and throttling)
- Binlog table listeners work
- More Migrator logic; existing logic for waiting on `state` events (e.g. `TablesCreatedState`)
2016-04-07 15:57:12 +02:00
Shlomi Noach
4dd5a93ed7 - More work on go-mysql reader: now properly notes binlog coordinates as it makes progress
- Properly creating multiple BinlogEntry-ies when reading multi statement binlog event (Still todo: transactions)
2016-04-06 18:44:54 +02:00
Shlomi Noach
d8fefb3d6f exploded args on range query building; iteration works 2016-04-05 19:50:49 +02:00
Shlomi Noach
ea0906f4e5 reading table (range) min/max values, right now according to hardcoded unique key 2016-04-04 18:19:46 +02:00
Shlomi Noach
c75cd998fb a bunch of 'inspector' initial tests on the replica 2016-04-04 12:27:51 +02:00