Commit Graph

868 Commits

Author SHA1 Message Date
Daniel Poelzleithner
a244d49ea4 do not export inhert functions 2022-04-19 12:41:06 +02:00
Daniel Poelzleithner
0f9a99f6ea Implement batchSizeLimit for rsync based transfers
If the batchSizeLimit is set, only files lower then this limit will
be grouped in one rsync transfer.
Each file larger then this limit will spawn their own transfer process.
This will cause large files to no longer block small file transfers under the
circumstance the maxProcess limit on the sync is larger then 1

A very optimized, very secure transfer configuration based on a
pool of ssh connection looks like this:

```
sync {
        default.rsync,
        tunnel = tunnel {
            command = {"ssh", "-N", "-L", "localhost:${localport}:localhost:873", "user@targetmachine"},
            mode = "pool",
            parallel = 2,
        },
        source = "/tmp/src",
        target    = "rsync://localhost:${localport}/test",
        delay = 5,
        batchSizeLimit = 1024 * 1024 * 30,
        maxProcesses = 4,
        rsync = {
                verbose = true,
                inplace = true,
        }
}
```

If you configure remote ssh configuration only allows portforwarding and your rsync daemon
is configured correctly, you can very securely transfer data without giving shell access.
2022-04-14 14:05:49 +02:00
Daniel Poelzleithner
89c05fe2a9 Add function to return file size 2022-04-14 13:55:07 +02:00
Daniel Poelzleithner
597f5e71d2 Add script to simulate fs changes 2022-04-13 01:55:59 +02:00
Daniel Poelzleithner
81325b0de5 fix logic error in alarm compare 2022-04-01 02:09:08 +02:00
Daniel Poelzleithner
883143f204 Fix wrong alarm selection 2022-03-31 03:54:59 +02:00
Daniel Poelzleithner
66b038e394 Finish full sync cron trigger 2022-03-31 03:54:21 +02:00
Daniel Poelzleithner
5b42d69193 Only check Tunnels if there is at least one tunnel defined 2022-03-30 23:07:45 +02:00
Daniel Poelzleithner
8607a84cc1 Log source of next alarm as well 2022-03-30 23:03:12 +02:00
Daniel Poelzleithner
1f79032c28 Add debug functions and timestamp2string helpers 2022-03-30 23:02:03 +02:00
Daniel Poelzleithner
4a03dab872 [WIP] implement crontab full sync 2022-03-25 19:01:19 +01:00
Daniel Poelzleithner
7a44958347 Fixes for diffrent lua versions 2022-03-24 14:48:58 +01:00
Daniel Poelzleithner
e2a4dc0e87 Merge branch 'tunnel' into periodic-full 2022-03-24 13:41:13 +01:00
Daniel Poelzleithner
9bfcaac15d Add lua-crontab to dependencies 2022-03-24 11:21:24 +01:00
Daniel Poelzleithner
bfc604a591 Implement tunnel pool mode.
In this mode, multiple tunnel processes are started and connection a load
balanced on the pool of connections.

Example config:

...
sync {
  default.rsync,
  tunnel = tunnel {
    command = {"ssh", "-N", "-L", "localhost:${localport}:localhost:873", "user@testmachine"},
    mode = "pool",
    parallel = 2,
  },
  target    = "rsync://localhost:${localport}/test",
  ...
}
2022-03-23 23:44:19 +01:00
Daniel Poelzleithner
07ec98174a Call runner cleanup method when exiting through error codes 2022-03-23 23:41:02 +01:00
Daniel Poelzleithner
b712aea0c7 rename traceback function 2022-03-17 02:29:23 +01:00
Daniel Poelzleithner
d708dca0f2 Call user provideable check function 2022-03-17 02:27:02 +01:00
Daniel Poelzleithner
cf7cfe1bda Add function to return a free port 2022-03-17 02:26:00 +01:00
Daniel Poelzleithner
f722ec14f8 Report tunnel status in status file 2022-03-17 02:25:27 +01:00
Daniel Poelzleithner
60e6505473 Add function to substitude commands with placeholders 2022-03-16 17:38:03 +01:00
Daniel Poelzleithner
cce6e5423b Updates to tunnel logic.
Delay before tunnel is considered up.
Working delay if restart fails.
Disabled state.
Kill all tunnel processes on graceful exit
2022-03-16 01:16:33 +01:00
Daniel Poelzleithner
dcf611d4f8 bump version 2022-03-16 01:16:33 +01:00
Daniel Poelzleithner
acff33211c [WIP] Add tunnel support 2022-03-16 01:16:33 +01:00
Daniel Poelzleithner
2841ef8528 add .editorconfig 2022-03-11 07:12:39 +01:00
Daniel Poelzleithner
a410ddebb8 Install man page to correct subfolder
fixes #655
2022-03-04 12:10:32 +01:00
Daniel Poelzleithner
8e002d64b5
Merge pull request #656 from LeonGaultier/patch-lua_version_mismatch_error_start_binary
fix error - bad binary format (version mismatch)
2022-02-28 21:12:09 +01:00
Marko Oldenburg
b7d11f6b00 change header discription 2022-02-24 14:40:46 +01:00
Marko Oldenburg
efa8155b68 fix error - bad binary format (version mismatch)
The problem is that the generated make files end up referencing a mix of lua versions
This patch will fix it.
2022-02-14 12:55:05 +01:00
Daniel Poelzleithner
e2a27af6e7 use relative sh path 2022-01-12 20:46:41 +01:00
Daniel Poelzleithner
f65cdd282d finish lua2.4 changes 2022-01-12 20:46:14 +01:00
Daniel Poelzleithner
a609f34971 fix compilation with lua 5.4
fixes #621
2021-12-14 16:45:25 +01:00
Daniel Poelzleithner
012b0cdd3e Merge branch 'master' of github.com:lsyncd/lsyncd 2021-12-13 13:26:14 +01:00
Daniel Poelzleithner
0b59ae7652 Merge remote-tracking branch 'origin/pr/595' 2021-12-10 15:22:23 +01:00
Daniel Poelzleithner
b7aa3d9a75
Merge pull request #538 from schneiderl/master
Fixed typo at readme.md
2021-12-10 15:19:17 +01:00
Daniel Poelzleithner
39f2f3a373 Finish -onepass option to exit when all syncs ran sucessfully once 2021-12-10 15:05:30 +01:00
Daniel Poelzleithner
d737c03c03 Merge remote-tracking branch 'origin/pr/541' 2021-12-10 12:51:01 +01:00
Daniel Poelzleithner
f812cd4cd5 Merge remote-tracking branch 'origin/pr/567' 2021-12-10 12:25:17 +01:00
Daniel Poelzleithner
938c702525 cleanup ssh server when aborting ci run 2021-12-10 12:20:18 +01:00
Daniel Poelzleithner
e5f71ea3ae Merge remote-tracking branch 'origin/pr/568' 2021-12-10 12:20:08 +01:00
Daniel Poelzleithner
dcebacb554
Merge pull request #615 from ajdavis/catalina-inotify-fix
Remove /System/Volumes/Data from fsevent paths on macOS
2021-12-09 15:43:27 +01:00
Daniel Poelzleithner
92af746a73 Enable CI 2021-12-09 14:37:30 +01:00
Daniel Poelzleithner
5212fd4794 don't depend on which. Add CI script 2021-12-09 13:50:15 +01:00
Daniel Poelzleithner
aea80964f3 install examples into docs 2021-12-08 18:54:02 +01:00
Daniel Poelzleithner
99a682f4cd add advanced ssh example 2021-12-08 18:28:03 +01:00
Daniel Poelzleithner
75bb0fb084 ignore tests/ssh folder 2021-12-08 18:27:32 +01:00
Daniel Poelzleithner
3e8aad3b2e Use sshopts for rsyncssh tests 2021-12-08 18:26:11 +01:00
Daniel Poelzleithner
deb425a075 use relative paths for direct mode 2021-12-08 18:25:11 +01:00
Daniel Poelzleithner
5b0d266669 add cmd option to pass ssh options 2021-12-08 18:24:13 +01:00
Daniel Poelzleithner
bb247e0267 Merge remote-tracking branch 'origin/pr/583'
fixes #583
2021-12-06 20:31:12 +01:00