Commit Graph

12 Commits

Author SHA1 Message Date
Jakob Borg
ac190b2e39 Change DB label format (index folders, devices) 2016-01-03 19:32:40 +01:00
Jakob Borg
faf4267c73 Refactor: the various db key functions should be instance methods 2015-10-31 11:27:04 +01:00
Jakob Borg
c4ad97136f Move leveldb instance and transactions into separate files 2015-10-29 08:07:51 +01:00
Jakob Borg
fd7b8ec77e Neater transaction handling 2015-10-28 22:04:00 +01:00
Jakob Borg
e28c991331 Create an instance type to tie database methods to 2015-10-28 21:03:05 +01:00
Jakob Borg
0c0c69f0cf The GC runs are legacy and slows things down quite a bit
benchmark                         old ns/op      new ns/op      delta
BenchmarkReplaceAll-8             2942370526     2866418930     -2.58%
BenchmarkUpdateOneChanged-8       7402489        226635         -96.94%
BenchmarkUpdateOneUnchanged-8     7298777        229090         -96.86%
BenchmarkNeedHalf-8               113608416      104483393      -8.03%
BenchmarkHave-8                   29834263       29288220       -1.83%
BenchmarkGlobal-8                 162773699      159269126      -2.15%
BenchmarkNeedHalfTruncated-8      111943400      108235000      -3.31%
BenchmarkHaveTruncated-8          29490369       28945489       -1.85%
BenchmarkGlobalTruncated-8        165841081      149355833      -9.94%

benchmark                         old allocs     new allocs     delta
BenchmarkReplaceAll-8             1054942        1054944        +0.00%
BenchmarkUpdateOneChanged-8       1149           1135           -1.22%
BenchmarkUpdateOneUnchanged-8     1135           1135           +0.00%
BenchmarkNeedHalf-8               374774         374777         +0.00%
BenchmarkHave-8                   151995         151995         +0.00%
BenchmarkGlobal-8                 530042         530063         +0.00%
BenchmarkNeedHalfTruncated-8      374697         374699         +0.00%
BenchmarkHaveTruncated-8          151834         151834         +0.00%
BenchmarkGlobalTruncated-8        530050         530021         -0.01%

benchmark                         old bytes      new bytes      delta
BenchmarkReplaceAll-8             5074294728     5074297112     +0.00%
BenchmarkUpdateOneChanged-8       141048         135097         -4.22%
BenchmarkUpdateOneUnchanged-8     134976         134976         +0.00%
BenchmarkNeedHalf-8               44734813       44759436       +0.06%
BenchmarkHave-8                   11911634       11911138       -0.00%
BenchmarkGlobal-8                 80436854       81609867       +1.46%
BenchmarkNeedHalfTruncated-8      46514673       46588024       +0.16%
BenchmarkHaveTruncated-8          11348357       11348354       -0.00%
BenchmarkGlobalTruncated-8        81730740       79485168       -2.75%
2015-10-21 23:05:22 +02:00
Jakob Borg
1eca4170f7 Add test for LocalSize/GlobalSize results 2015-10-21 21:58:48 +02:00
Jakob Borg
c268e4ad1b Also keep GlobalSize in RAM 2015-10-21 21:58:48 +02:00
Jakob Borg
d4f81e8791 Keep LocalSize data in RAM 2015-10-21 21:58:48 +02:00
Jakob Borg
76af9ba53d Implement facility based logger, debugging via REST API
This implements a new debug/trace infrastructure based on a slightly
hacked up logger. Instead of the traditional "if debug { ... }" I've
rewritten the logger to have no-op Debugln and Debugf, unless debugging
has been enabled for a given "facility". The "facility" is just a
string, typically a package name.

This will be slightly slower than before; but not that much as it's
mostly a function call that returns immediately. For the cases where it
matters (the Debugln takes a hex.Dump() of something for example, and
it's not in a very occasional "if err != nil" branch) there is an
l.ShouldDebug(facility) that is fast enough to be used like the old "if
debug".

The point of all this is that we can now toggle debugging for the
various packages on and off at runtime. There's a new method
/rest/system/debug that can be POSTed a set of facilities to enable and
disable debug for, or GET from to get a list of facilities with
descriptions and their current debug status.

Similarly a /rest/system/log?since=... can grab the latest log entries,
up to 250 of them (hardcoded constant in main.go) plus the initial few.

Not implemented in this commit (but planned) is a simple debug GUI
available on /debug that shows the current log in an easily pasteable
format and has checkboxes to enable the various debug facilities.

The debug instructions to a user then becomes "visit this URL, check
these boxes, reproduce your problem, copy and paste the log". The actual
log viewer on the hypothetical /debug URL can poll regularly for new log
entries and this bypass the 250 line limit.

The existing STTRACE=foo variable is still obeyed and just sets the
start state of the system.
2015-10-03 18:09:53 +02:00
Jakob Borg
4581c57478 Fix import paths 2015-09-22 19:38:46 +02:00
Jakob Borg
7705a6c1f1 mv internal lib 2015-08-09 09:35:26 +02:00