Compare commits

...
Author SHA1 Message Date
John Sully c801414b40 Perf: remove mfence 2020-10-04 03:15:18 +00:00
John Sully f4c317d285 Initial implementation of replication-db-disable config 2020-10-03 23:57:35 +00:00
John Sully 420ccdfbae Fix module test failures due to locking 2020-10-01 00:08:54 +00:00
John Sully 41466a4147 Fix TLS test failure due to bad merge 2020-09-30 20:21:40 +00:00
John Sully 9798f2d711 Keep redis-cli.c C89 compatible 2020-09-30 20:17:51 +00:00
John Sully 5f3543921b Mac build break fix 2020-09-30 20:17:02 +00:00
John Sully a41366cc3a Fix stream replication failure with active replication, issue #238 2020-09-30 20:12:54 +00:00
John Sully 4c7e4b91a6 Merge tag '6.0.8' into unstable 2020-09-30 19:47:55 +00:00
John Sully 28183f4b66 Test RDB merge on load with active replication 2020-09-29 03:26:06 +00:00
John Sully a0abc1eddd Drop min-clients-per-thread to a more reasonable number 2020-09-28 19:17:36 +00:00
John Sully 676644f2db Support read only replicas attaching to active replicas (Bug #229) 2020-09-23 04:16:08 +00:00
John Sully 22ac56d62d Active replicas memory leak 2020-09-22 21:38:35 +00:00
John Sully 143f7f634d Merge branch 'unstable' of https://github.com/JohnSully/KeyDB into unstable 2020-09-21 04:42:46 +00:00
Dmitry Dorofeev 955faef23a Fix to support FreeBSD pthread_getthreadid_np call 2020-09-21 00:41:41 -04:00
John Sully 2e159e6ea3 Fix TSAN errors in fastlock 2020-09-21 03:26:50 +00:00
John Sully 02b533fec3 Fix issue #222 2020-09-21 01:09:58 +00:00
John Sully de3099e6d8 Make multimaster replication info statelessly paresable 2020-09-20 23:57:06 +00:00
John Sully 368f67f422 Dramatically improve the performance of subkey expires 2020-09-20 23:30:21 +00:00
John Sully 13c9a094d1 Reduce test noise 2020-09-19 22:00:17 +00:00
John Sully 0f8b7a19ef Fix CI failure due to apt host changes 2020-09-19 21:49:59 +00:00
John Sully 8f7d977f90 Avoid memory overhead in situations it won't be used (active rep only) 2020-09-19 21:44:59 +00:00
John Sully 064440ee84 Update endurance test for ubuntu 20 2020-09-17 02:10:18 +00:00
John Sully 50641d0569 Fix build break 2020-09-17 00:53:56 +00:00
John Sully f5ecc77e29 master_global_link_status should respect the quorum 2020-09-17 00:31:10 +00:00
John Sully 655bb6e21c Allow reads during an active replica load, optionally allow writes as an experimental feature 2020-09-17 00:13:03 +00:00
John Sully ab4cdd2ed4 Implement replica-quorum config 2020-09-16 03:26:31 +00:00
Yossi Gottlieb d3834c5069 TLS: Session caching configuration support. (#7420)
* TLS: Session caching configuration support.
* TLS: Remove redundant config initialization.
2020-09-16 02:10:29 +00:00
John Sully 7a1b2c58c3 Protocol error in client command 2020-09-11 20:48:18 +00:00
John Sully 7fc6d6c6b1 Fix more protocol errors 2020-09-11 20:47:27 +00:00
John Sully dec5db4e71 Fix protocol difference with Redis 2020-09-11 20:08:26 +00:00
Oran Agra 03b59cd5f0 6.0.8 2020-09-10 14:09:00 +03:00
Oran Agra 5380716a70 Fix RESP3 response for HKEYS/HVALS on non-existing key 2020-09-10 14:09:00 +03:00
Oran Agra 18b30467f7 Fix leak in new blockedclient module API test
(cherry picked from commit 0e20ad14a6a857cb168b808f94721df19b23dc0c)
2020-09-10 14:09:00 +03:00
Yossi Gottlieb 0052d5196d Tests: fix oom-score-adj false positives. (#7772)
The key save delay is too short and on certain systems the child process
is gone before we have a chance to inspect it.

(cherry picked from commit b2a73c404b)
2020-09-10 14:09:00 +03:00
杨博东 b55b0ea0f2 Tests: Add aclfile load and save tests (#7765)
improves test coverage

(cherry picked from commit 0666267d27)
2020-09-10 14:09:00 +03:00
Roi Lipman 25e211bcf0 RM_ThreadSafeContextTryLock a non-blocking method for acquiring GIL (#7738)
Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit 042189fd87)
2020-09-10 14:09:00 +03:00
Yossi Gottlieb 635704d424 Tests: validate CONFIG REWRITE for all params. (#7764)
This is a catch-all test to confirm that that rewrite produces a valid
output for all parameters and that this process does not introduce
undesired configuration changes.

(cherry picked from commit a8b7268911)
2020-09-10 14:09:00 +03:00
Oran Agra 47364bce69 Change THP warning to use madvise rather than never (#7771)
completes b2419c31c1

(cherry picked from commit 1461f02deb)
2020-09-10 14:09:00 +03:00
Itamar Haber 3cd6c26056 Documents RM_Call's fmt (#5448)
Improve RM_Call inline documentation about the fmt argument
so that we don't completely depend on the web docs.

Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit ce15620dc1)
2020-09-10 14:09:00 +03:00
Jan-Erik Rediger 8b0d797ecd Check that THP is not set to always (madvise is ok) (#4001)
THP can also be set to madvise, in which case it shouldn't cause
problems for Redis since redis (or the allocator) doesn't use madvise
to activate it.

(cherry picked from commit b2419c31c1)
2020-09-10 14:09:00 +03:00
Yossi Gottlieb 3c8b394511 Tests: clean up stale .cli files. (#7768)
(cherry picked from commit 918abd7276)
2020-09-10 14:09:00 +03:00
Eran Liberty 84ba2ec82b Allow exec with read commands on readonly replica in cluster (#7766)
There was a bug. Although cluster replicas would allow read commands,
they would not allow a MULTI-EXEC that's composed solely of read commands.
Adds tests for coverage.

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Eran Liberty <eranl@amazon.com>
(cherry picked from commit b120366d48)
2020-09-10 14:09:00 +03:00
Yossi Gottlieb cf3375bf7c Fix CONFIG REWRITE of oom-score-adj-values. (#7761)
(cherry picked from commit 750acf3a45)
2020-09-10 14:09:00 +03:00
Oran Agra 643d3e0b79 handle cur_test for nested tests
if there are nested tests and nested servers, we need to restore the
previous value of cur_test when a test exist.

example:
```
test{test 1} {
	start_server {
		test{test 1.1 - master only} {
		}
		start_server {
		    test{test 1.2 - with replication} {
            }
		}
	}
}
```
when `test 1.1 - master only exists`, we're still inside `test 1`

(cherry picked from commit 0a1e734193)
2020-09-10 14:09:00 +03:00
Oran Agra 874c3a9978 Add daily CI for MacOS (#7759)
(cherry picked from commit 5496b4a7cd)
2020-09-10 14:09:00 +03:00
bodong.ybd 781e50d41f Tests: Some fixes for macOS
1) cur_test: when restart_server, "no such variable" error occurs
  ./runtest --single integration/rdb
  test {client freed during loading}
      SET ::cur_test
      restart_server
        kill_server
          test "Check for memory leaks (pid $pid)"
          SET ::cur_test
          UNSET ::cur_test
      UNSET ::cur_test // This global variable has been unset.

2) `ps --ppid` not available on macOS platform, can be replaced with
`pgrep -P pid`.

(cherry picked from commit f22fa9594d)
2020-09-10 14:09:00 +03:00
Oran Agra 57dbd6e90d Fix cluster consistency-check test (#7754)
This test was failing from time to time see discussion at the bottom of #7635
This was probably due to timing, the DEBUG SLEEP executed by redis-cli
didn't sleep for enough time.

This commit changes:
1) use SET-ACTIVE-EXPIRE instead of DEBUG SLEEP
2) reduce many `after` sleeps with retry loops to speed up the test.
3) add many comment explaining the different steps of the test and
   it's purpose.
4) config appendonly before populating the volatile keys, so that they'll
   be part of the AOF command stream rather than the preamble RDB portion.

other complications: recently kill_instance switched from SIGKILL to
SIGTERM, and this would sometimes fail since there was an AOFRW running
in the background. now we wait for it to end before attempting the kill.

(cherry picked from commit b491d477c3)
2020-09-10 14:09:00 +03:00
Yossi Gottlieb e9fef49e12 Tests: fix unmonitored servers. (#7756)
There is an inherent race condition in port allocation for spawned
servers. If a server fails to start because a port is taken, a new port
is allocated. This fixes a problem where the logs are not truncated and
as a result a large number of unmonitored servers are started.

(cherry picked from commit 2df4cb93ac)
2020-09-10 14:09:00 +03:00
Oran Agra a227bc8053 fix broken cluster/sentinel tests by recent commit (#7752)
2b998de46 added a file for stderr to keep valgrind log but i forgot to
add a similar thing when valgrind isn't being used.
the result is that `glob */err.txt` fails.

(cherry picked from commit 42ba7a1b75)
2020-09-10 14:09:00 +03:00
Oran Agra d410dc3162 Improve valgrind support for cluster tests (#7725)
- redirect valgrind reports to a dedicated file rather than console
- try to avoid killing instances with SIGKILL so that we get the memory
  leak report (killing with SIGTERM before resorting to SIGKILL)
- search for valgrind reports when done, print them and fail the tests
- add --dont-clean option to keep the logs on exit
- fix exit error code when crash is found (would have exited with 0)

changes that affect the normal redis test suite:
- refactor check_valgrind_errors into two functions one to search and
  one to report
- move the search half into util.tcl to serve the cluster tests too
- ignore "address range perms" valgrind warnings which seem non relevant.

(cherry picked from commit 2b998de460)
2020-09-10 14:09:00 +03:00
Oran Agra 41c7c7919c test infra - add durable mode to work around test suite crashing
in some cases a command that returns an error possibly due to a timing
issue causes the tcl code to crash and thus prevents the rest of the
tests from running. this adds an option to make the test proceed despite
the crash.
maybe it should be the default mode some day.

(cherry picked from commit fe5da2e60d)
2020-09-10 14:09:00 +03:00
Oran Agra db6c763d8b test infra - wait_done_loading
reduce code duplication in aof.tcl.
move creation of clients into the test so that it can be skipped

(cherry picked from commit 1b7ba44e79)
2020-09-10 14:09:00 +03:00
Oran Agra 72d6f966ac test infra - flushall between tests in external mode
(cherry picked from commit b65e5aca86)
2020-09-10 14:09:00 +03:00
Oran Agra 28e074608c test infra - improve test skipping ability
- skip full units
- skip a single test (not just a list of tests)
- when skipping tag, skip spinning up servers, not just the tests
- skip tags when running against an external server too
- allow using multiple tags (split them)

(cherry picked from commit 677d14c213)
2020-09-10 14:09:00 +03:00
Oran Agra 5b8de5b7f2 test infra - reduce disk space usage
this is important when running a test with --loop

(cherry picked from commit e3e69c25fd)
2020-09-10 14:09:00 +03:00
Oran Agra bce350c666 test infra - write test name to logfile
(cherry picked from commit 9d527d076b)
2020-09-10 14:09:00 +03:00
Yossi Gottlieb 332aa2f919 redis-cli: fix writeConn() buffer handling. (#7749)
Fix issues with writeConn() which resulted with corruption of the stream by leaving an extra byte in the buffer. The trigger for this is partial writes or write errors which were not experienced on Linux but reported on macOS.

(cherry picked from commit 58e5feb3f4)
2020-09-10 14:09:00 +03:00
Oran Agra e4db67ea57 Print server startup messages after daemonization (#7743)
When redis isn't configured to have a log file, having these prints
before damonization puts them in the calling process stdout rather than
/dev/null

(cherry picked from commit 8b0747d657)
2020-09-10 14:09:00 +03:00
Thandayuthapani b983015dd4 Add masters/replicas options to redis-cli --cluster call command (#6491)
* Add master/slave option in --cluster call command

* Update src/redis-cli.c

* Update src/redis-cli.c

Co-authored-by: Itamar Haber <itamar@redislabs.com>
(cherry picked from commit f22f64f0db)
2020-09-10 14:09:00 +03:00
Oran Agra 9b718d41f0 fix README about BUILD_WITH_SYSTEMD usage (#7739)
BUILD_WITH_SYSTEMD is an internal variable. Users should use USE_SYSTEMD=yes.

(cherry picked from commit 747b4004ea)
2020-09-10 14:09:00 +03:00
Yossi Gottlieb da33a18588 Fix double-make issue with make && make install. (#7734)
All user-supplied variables that affect the build should be explicitly
persisted.

Fixes #7254

(cherry picked from commit b35d6e5cff)
2020-09-10 14:09:00 +03:00
John Sully d47aeb1fc8 Fix whitespace 2020-09-07 03:35:46 +00:00
John Sully e47584b286 Dramatically improve perf by blocking commands 2020-09-07 00:49:53 +00:00
Oran Agra dbea5f7a8d Redis 6.0.7 2020-09-01 09:27:58 +03:00
Oran Agra 4bb40a9688 Reduce the probability of failure when start redis in runtest-cluster #7554 (#7635)
When runtest-cluster, at first, we need to create a cluster use spawn_instance,
a port which is not used is choosen, however sometimes we can't run server on
the port. possibley due to a race with another process taking it first.
such as redis/redis/runs/896537490. It may be due to the machine problem or
In order to reduce the probability of failure when start redis in
runtest-cluster, we attemp to use another port when find server do not start up.

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: yanhui13 <yanhui13@meituan.com>
(cherry picked from commit e2d64485b8)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb d2532d1335 Backport Lua 5.2.2 stack overflow fix. (#7733)
This fixes the issue described in CVE-2014-5461. At this time we cannot
confirm that the original issue has a real impact on Redis, but it is
included as an extra safety measure.

(cherry picked from commit d75ad774a9)
2020-09-01 09:27:58 +03:00
Leoš Literák 00d0d870d2 Update README.md with instructions how to build with systemd support (#7730)
#7728 - update instructions for systemd support

(cherry picked from commit 571571ca19)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb f38e2802b6 Fix oom-score-adj on older distros. (#7724)
Don't assume `ps` handles `-h` to display output without headers and
manually trim headers line from output.

(cherry picked from commit b61b663895)
2020-09-01 09:27:58 +03:00
Wang Yuan 1b100a167a Fix wrong format specifiers of 'sdscatfmt' for the INFO command (#7706)
unlike printf, sdscatfmt doesn't take %d

(cherry picked from commit 43af28f5b4)
2020-09-01 09:27:58 +03:00
Wen Hui d4e1c88052 fix make warnings (#7692)
(cherry picked from commit e61adc0d89)
2020-09-01 09:27:58 +03:00
Nathan Scott 2c0c5a809c Annotate module API functions in redismodule.h for use with -fno-common (#6900)
In order to keep the redismodule.h self-contained but still usable with
gcc v10 and later, annotate each API function tentative definition with
the __common__ attribute.  This avoids the 'multiple definition' errors
modules will otherwise see for all API functions at link time.

Further details at gcc.gnu.org/gcc-10/porting_to.html

Turn the existing __attribute__ ((unused)), ((__common__)) and ((print))
annotations into conditional macros for any compilers not accepting this
syntax.  These macros only expand to API annotations under gcc.

Provide a pre- and post- macro for every API function, so that they can
be defined differently by the file that includes redismodule.h.

Removing REDISMODULE_API_FUNC in the interest of keeping the function
declarations readable.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit 11cd983d58)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb f6d04d01b9 Add oom-score-adj configuration option to control Linux OOM killer. (#1690)
Add Linux kernel OOM killer control option.

This adds the ability to control the Linux OOM killer oom_score_adj
parameter for all Redis processes, depending on the process role (i.e.
master, replica, background child).

A oom-score-adj global boolean flag control this feature. In addition,
specific values can be configured using oom-score-adj-values if
additional tuning is required.

(cherry picked from commit 2530dc0ebd)
2020-09-01 09:27:58 +03:00
Meir Shpilraien (Spielrein) 2fc915f509 see #7544, added RedisModule_HoldString api. (#7577)
Added RedisModule_HoldString that either returns a
shallow copy of the given String (by increasing
the String ref count) or a new deep copy of String
in case its not possible to get a shallow copy.

Co-authored-by: Itamar Haber <itamar@redislabs.com>
(cherry picked from commit 3f494cc49d)
2020-09-01 09:27:58 +03:00
ShooterIT e3cfc92180 [Redis-benchmark] Remove zrem test, add zpopmin test
(cherry picked from commit 6a06a5a597)
2020-09-01 09:27:58 +03:00
ShooterIT 7fcf29e615 [Redis-benchmark] Support zset type
(cherry picked from commit e5a50ed3c4)
2020-09-01 09:27:58 +03:00
Arun Ranganathan e81bac32fd Show threading configuration in INFO output (#7446)
Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit f6cad30bb6)
2020-09-01 09:27:58 +03:00
Meir Shpilraien (Spielrein) 2257f38b68 This PR introduces a new loaded keyspace event (#7536)
Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Itamar Haber <itamar@redislabs.com>
(cherry picked from commit 8d82639319)
2020-09-01 09:27:58 +03:00
Oran Agra 2165a78d10 Fix rejectCommand trims newline in shared error objects, hung clients (#7714)
65a3307bc (released in 6.0.6) has a side effect, when processCommand
rejects a command with pre-made shared object error string, it trims the
newlines from the end of the string. if that string is later used with
addReply, the newline will be missing, breaking the protocol, and
leaving the client hung.

It seems that the only scenario which this happens is when replying with
-LOADING to some command, and later using that reply from the CONFIG
SET command (still during loading). this will result in hung client.

Refactoring the code in order to avoid trimming these newlines from
shared string objects, and do the newline trimming only in other cases
where it's needed.

Co-authored-by: Guy Benoish <guy.benoish@redislabs.com>
(cherry picked from commit 9fcd9e191e)
2020-09-01 09:27:58 +03:00
valentinogeron e6f6731c66 EXEC with only read commands should not be rejected when OOM (#7696)
If the server gets MULTI command followed by only read
commands, and right before it gets the EXEC it reaches OOM,
the client will get OOM response.

So, from now on, it will get OOM response only if there was
at least one command that was tagged with `use-memory` flag

(cherry picked from commit b7289e912c)
2020-09-01 09:27:58 +03:00
Itamar Haber 5a9a653f3e Expands lazyfree's effort estimate to include Streams (#5794)
Otherwise, it is treated as a single allocation and freed synchronously. The following logic is used for estimating the effort in constant-ish time complexity:

1. Check the number of nodes.
1. Add an allocation for each consumer group registered inside the stream.
1. Check the number of PELs in the first CG, and then add this count times the number of CGs.
1. Check the number of consumers in the first CG, and then add this count times the number of CGs.

(cherry picked from commit 5b0a06af48)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb ff0846cbb7 Add language servers stuff, test/tls to gitignore. (#7698)
(cherry picked from commit 5449a2a8b5)
2020-09-01 09:27:58 +03:00
Valentino Geron 3e2ed279fd Assert that setDeferredAggregateLen isn't called with negative value
In case the redis is about to return broken reply we want to crash
with assert so that we are notified about the bug. see #7687.

(cherry picked from commit 8b428cf0f7)
2020-09-01 09:27:58 +03:00
Valentino Geron 19ef1f371d Fix LPOS command when RANK is greater than matches
When calling to LPOS command when RANK is higher than matches,
the return value is non valid response. For example:
```
LPUSH l a
:1
LPOS l b RANK 5 COUNT 10
*-4
```
It may break client-side parser.

Now, we count how many replies were replied in the array.
```
LPUSH l a
:1
LPOS l b RANK 5 COUNT 10
*0
```

(cherry picked from commit 9204a9b2c2)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb 8d79702d8a Tests: fix redis-cli with remote hosts. (#7693)
(cherry picked from commit f80f3f492a)
2020-09-01 09:27:58 +03:00
huangzhw 6cfcf76e0b RedisModuleEvent_LoadingProgress always at 100% progress (#7685)
It was also using the wrong struct, but luckily RedisModuleFlushInfo and RedisModuleLoadingProgress
are identical.

(cherry picked from commit a3d4d7bf68)
2020-09-01 09:27:58 +03:00
guybe7 c9478a3b4e Modules: Invalidate saved_oparray after use (#7688)
We wanna avoid a chance of someone using the pointer in it after it'll be freed / realloced.

(cherry picked from commit 65c24bd3d4)
2020-09-01 09:27:58 +03:00
杨博东 113d5ae872 Fix flock cluster config may cause failure to restart after kill -9 (#7674)
After fork, the child process(redis-aof-rewrite) will get the fd opened
by the parent process(redis), when redis killed by kill -9, it will not
graceful exit(call prepareForShutdown()), so redis-aof-rewrite thread may still
alive, the fd(lock) will still be held by redis-aof-rewrite thread, and
redis restart will fail to get lock, means fail to start.

This issue was causing failures in the cluster tests in github actions.

Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit cbaf3c5bba)
2020-09-01 09:27:58 +03:00
Raghav Muddur 1bd75aafdb Update clusterMsgDataPublish to clusterMsgModule (#7682)
Correcting the variable to clusterMsgModule.

(cherry picked from commit 34c3be365a)
2020-09-01 09:27:58 +03:00
Madelyn Olson ba48e7e9cd Fixed hset error since it's shared with hmset (#7678)
(cherry picked from commit cbd9af8583)
2020-09-01 09:27:58 +03:00
guybe7 4e19358d65 PERSIST should signalModifiedKey (Like EXPIRE does) (#7671)
(cherry picked from commit b87c288016)
2020-09-01 09:27:58 +03:00
Oran Agra 43bc1ea065 OOM Crash log include size of allocation attempt. (#7670)
Since users often post just the crash log in github issues, the log
print that's above it is missing.
No reason not to include the size in the panic message itself.

(cherry picked from commit 0f741a9e2d)
2020-09-01 09:27:58 +03:00
Wen Hui 3842bfd899 [module] using predefined REDISMODULE_NO_EXPIRE in RM_GetExpire (#7669)
It was already defined in the API header and the documentation, but not used by the implementation.

(cherry picked from commit 93d87d6d4c)
2020-09-01 09:27:58 +03:00
Oran Agra 6dcd641681 Trim trailing spaces in error replies coming from rejectCommand (#7668)
65a3307bc9 added rejectCommand which takes an robj reply and passes it
through addReplyErrorSafe to addReplyErrorLength.
The robj contains newline at it's end, but addReplyErrorSafe converts it
to spaces, and passes it to addReplyErrorLength which adds the protocol
newlines.

The result was that most error replies (like OOM) had extra two trailing
spaces in them.

(cherry picked from commit cdd925b289)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb 1180537589 Module API: fix missing RM_CLIENTINFO_FLAG_SSL. (#7666)
The `REDISMODULE_CLIENTINFO_FLAG_SSL` flag was already a part of the `RedisModuleClientInfo` structure but was not implemented.

(cherry picked from commit 64c360c515)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb 2e7ad58918 TLS: relax verification on CONFIG SET. (#7665)
Avoid re-configuring (and validating) SSL/TLS configuration on `CONFIG
SET` when TLS is not actively enabled for incoming connections, cluster
bus or replication.

This fixes failures when tests run without `--tls` on binaries that were
built with TLS support.

An additional benefit is that it's now possible to perform a multi-step
configuration process while TLS is disabled. The new configuration will
be verified and applied only when TLS is effectively enabled.

(cherry picked from commit fb2a94af3f)
2020-09-01 09:27:58 +03:00
Madelyn Olson 39ffc3702f Fixed timer warning (#5953)
(cherry picked from commit 79c506ebf0)
2020-09-01 09:27:58 +03:00
Wagner Francisco Mezaroba 31415cb27a allow --pattern to be used along with --bigkeys (#3586)
Adds --pattern option to cli's --bigkeys, --hotkeys & --scan modes

(cherry picked from commit e2a71338eb)
2020-09-01 09:27:58 +03:00
zhaozhao.zz 840a9951b5 redis-benchmark: fix wrong random key for hset (#4895)
(cherry picked from commit ff1e4a7063)
2020-09-01 09:27:58 +03:00
zhaozhao.zz 3c87d572b9 CLIENT_MASTER should ignore server.proto_max_bulk_len
(cherry picked from commit 589e610ebc)
2020-09-01 09:27:58 +03:00
zhaozhao.zz d3831559cf config: proto-max-bulk-len must be 1mb or greater
(cherry picked from commit bd4b33d7a2)
2020-09-01 09:27:58 +03:00
zhaozhao.zz 44ef19743e using proto-max-bulk-len in checkStringLength for SETRANGE and APPEND
(cherry picked from commit 2e69bfe44d)
2020-09-01 09:27:58 +03:00
YoongHM 171f83306e Start redis after network is online (#7639)
The two lines allow systemd to start redis.service after the network is online. Only after the network is online that Redis could bind to IP address other than 127.0.0.1 during initial boot up process.

(cherry picked from commit 8e937ce4cc)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb 0314429c8f Run daily workflow on main repo only (no forks). (#7646)
(cherry picked from commit 944cca98c8)
2020-09-01 09:27:58 +03:00
WuYunlong 086707315f see #7250, fix signature of RedisModule_DeauthenticateAndCloseClient (#7645)
In redismodule.h, RedisModule_DeauthenticateAndCloseClient returns void
`void REDISMODULE_API_FUNC(RedisModule_DeauthenticateAndCloseClient)(RedisModuleCtx *ctx, uint64_t client_id);`
But in module.c, RM_DeauthenticateAndCloseClient returns int
`int RM_DeauthenticateAndCloseClient(RedisModuleCtx *ctx, uint64_t client_id)`

It it safe to change return value from `void` to `int` from the user's perspective.

(cherry picked from commit d6220f12a9)
2020-09-01 09:27:58 +03:00
Wang Yuan 9fd0c027e7 Print error info if failed opening config file (#6943)
(cherry picked from commit ea7eeb2fd2)
2020-09-01 09:27:58 +03:00
Wen Hui 0e581e0044 fix memory leak in ACLLoadFromFile error handling (#7623)
(cherry picked from commit 3f67b03378)
2020-09-01 09:27:58 +03:00
Oran Agra e8801f7eaa redis-cli --cluster-yes - negate force flag for clarity
this internal flag is there so that some commands do not comply to `--cluster-yes`

(cherry picked from commit 1aa31e4da9)
2020-09-01 09:27:58 +03:00
Frank Meier 3a677b45ac reintroduce REDISCLI_CLUSTER_YES env variable in redis-cli
the variable was introduced only in the 5.0 branch in #5879 bc6c1c40db

(cherry picked from commit 51077c8212)
2020-09-01 09:27:58 +03:00
Frank Meier dc46f6f041 add force option to 'create-cluster create' script call (#7612)
(cherry picked from commit 3244fae7d4)
2020-09-01 09:27:58 +03:00
Oran Agra 916b215fc5 fix new rdb test failing on timing issues (#7604)
apparenlty on github actions sometimes 500ms is not enough

(cherry picked from commit 824bd2ac11)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb 9f0e8f7849 Fix test-centos7-tls daily job. (#7598)
(cherry picked from commit f2da3efc78)
2020-09-01 09:27:58 +03:00
Oran Agra a5294c4e52 module hook for master link up missing on successful psync (#7584)
besides, hooks test was time sensitive. when the replica managed to
reconnect quickly after the client kill, the test would fail

(cherry picked from commit f7e7775990)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb e511ab2cc6 CI: Add daily CentOS 7.x jobs. (#7582)
(cherry picked from commit 4ac1f9ac55)
2020-09-01 09:27:58 +03:00
WuYunlong 7200b3aa0f Fix running single test 14-consistency-check.tcl (#7587)
(cherry picked from commit f3352daf4f)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb e8efcdee0a Clarify RM_BlockClient() error condition. (#6093)
(cherry picked from commit 7af05f07ff)
2020-09-01 09:27:58 +03:00
namtsui 2bcc056a41 Avoid an out-of-bounds read in the redis-sentinel (#7443)
The Redis sentinel would crash with a segfault after a few minutes
because it tried to read from a page without read permissions. Check up
front whether the sds is long enough to contain redis:slave or
redis:master before memcmp() as is done everywhere else in
sentinelRefreshInstanceInfo().

Bug report and commit message from Theo Buehler. Fix from Nam Nguyen.

Co-authored-by: Nam Nguyen <namn@berkeley.edu>
(cherry picked from commit 63dae52324)
2020-09-01 09:27:58 +03:00
Wen Hui b2bd3c0653 Add SignalModifiedKey hook in XGROUP CREATE with MKSTREAM option (#7562)
(cherry picked from commit f33acb3f02)
2020-09-01 09:27:58 +03:00
Wen Hui 4238df5436 fix leak in error handling of debug populate command (#7062)
valsize was not modified during the for loop below instead of getting from c->argv[4], therefore there is no need to put inside the for loop.. Moreover, putting the check outside loop will also avoid memory leaking, decrRefCount(key) should be called in the original code if we put the check in for loop

(cherry picked from commit c69a9b2f61)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb c68b3908a6 Fix TLS cluster tests. (#7578)
Fix consistency test added in af5167b7f without considering TLS
redis-cli configuration.

(cherry picked from commit bedf1b2126)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb 1c05b87e30 TLS: Propagate and handle SSL_new() failures. (#7576)
The connection API may create an accepted connection object in an error
state, and callers are expected to check it before attempting to use it.

Co-authored-by: mrpre <mrpre@163.com>
(cherry picked from commit 784ceeb90d)
2020-09-01 09:27:58 +03:00
Oran Agra 67750ce3b3 Fix failing tests due to issues with wait_for_log_message (#7572)
- the test now waits for specific set of log messages rather than wait for
  timeout looking for just one message.
- we don't wanna sample the current length of the log after an action, due
  to a race, we need to start the search from the line number of the last
  message we where waiting for.
- when attempting to trigger a full sync, use multi-exec to avoid a race
  where the replica manages to re-connect before we completed the set of
  actions that should force a full sync.
- fix verify_log_message which was broken and unused

(cherry picked from commit 109b5ccdcd)
2020-09-01 09:27:58 +03:00
Jiayuan Chen 096285ab64 Add optional tls verification (#7502)
Adds an `optional` value to the previously boolean `tls-auth-clients` configuration keyword.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
(cherry picked from commit f31260b044)
2020-09-01 09:27:58 +03:00
Oran Agra 0b2f7c6407 Daily github action: run cluster and sentinel tests with tls (#7575)
(cherry picked from commit 6d92eee69b)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb 8a4240b1fa TLS: support cluster/replication without tls-port.
Initialize and configure OpenSSL even when tls-port is not used, because
we may still have tls-cluster or tls-replication.

Also, make sure to reconfigure OpenSSL when these parameters are changed
as TLS could have been enabled for the first time.

(cherry picked from commit c75512d89d)
2020-09-01 09:27:58 +03:00
grishaf 395d339234 Fix prepareForShutdown function declaration (#7566)
(cherry picked from commit 4126ca466f)
2020-09-01 09:27:58 +03:00
Oran Agra 6daa8b9adb Stabilize bgsave test that sometimes fails with valgrind (#7559)
on ci.redis.io the test fails a lot, reporting that bgsave didn't end.
increaseing the timeout we wait for that bgsave to get aborted.
in addition to that, i also verify that it indeed got aborted by
checking that the save counter wasn't reset.

add another test to verify that a successful bgsave indeed resets the
change counter.

(cherry picked from commit 8a57969fd7)
2020-09-01 09:27:58 +03:00
Madelyn Olson e00b6248c9 Properly reset errno for rdbLoad (#7542)
(cherry picked from commit 818dc3a089)
2020-09-01 09:27:58 +03:00
Oran Agra e8aa5583d0 testsuite may leave servers alive on error (#7549)
in cases where you have
test name {
  start_server {
    start_server {
      assert
    }
  }
}

the exception will be thrown to the test proc, and the servers are
supposed to be killed on the way out. but it seems there was always a
bug of not cleaning the server stack, and recently (#7404) we started
relying on that stack in order to kill them, so with that bug sometimes
we would have tried to kill the same server twice, and leave one alive.

luckly, in most cases the pattern is:
start_server {
  test name {
  }
}

(cherry picked from commit 36b9494385)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb f1d5d5d28e Tests: drop TCL 8.6 dependency. (#7548)
This re-implements the redis-cli --pipe test so it no longer depends on a close feature available only in TCL 8.6.

Basically what this test does is run redis-cli --pipe, generates a bunch of commands and pipes them through redis-cli, and inspects the result in both Redis and the redis-cli output.

To do that, we need to close stdin for redis-cli to indicate we're done so it can flush its buffers and exit. TCL has bi-directional channels can only offers a way to "one-way close" a channel with TCL 8.6. To work around that, we now generate the commands into a file and feed that file to redis-cli directly.

As we're writing to an actual file, the number of commands is now reduced.

(cherry picked from commit f57e844b2e)
2020-09-01 09:27:58 +03:00
Oran Agra cc7f4ba67e Fixes to release scripts (#7547)
(cherry picked from commit 343dd9bcce)
2020-09-01 09:27:58 +03:00
Remi Collet af907e4b6d Fix deprecated tail syntax in tests (#7543)
(cherry picked from commit 3f2fbc4c61)
2020-09-01 09:27:58 +03:00
Wen Hui d2f09a054c Add missing calls to raxStop (#7532)
Since the dynamic allocations in raxIterator are only used for deep walks, memory
leak due to missing call to raxStop can only happen for rax with key names longer
than 32 bytes.

Out of all the missing calls, the only ones that may lead to a leak are the rax
for consumer groups and consumers, and these were only in AOFRW and rdbSave, which
normally only happen in fork or at shutdown.

(cherry picked from commit 4e8f2d6881)
2020-09-01 09:27:58 +03:00
Wen Hui cef46edebc add missing caching command in client help (#7399)
(cherry picked from commit 2fbd0271f6)
2020-09-01 09:27:58 +03:00
zhaozhao.zz 9e997cd0d1 replication: need handle -NOPERM error after send ping (#7538)
(cherry picked from commit 13e50935a8)
2020-09-01 09:27:58 +03:00
Scott Brenner 171aa22b0b GitHub Actions workflows - use latest version of actions/checkout (#7534)
(cherry picked from commit 2f4e9c3f9f)
2020-09-01 09:27:58 +03:00
John Sully f17d178d03 Add build flag to disable MVCC tstamps 2020-08-10 06:10:24 +00:00
John Sully d88fd1588d Fix lock after free in module API 2020-08-10 06:10:24 +00:00
Oran Agra 7bf665f125 Redis 6.0.6. 2020-07-20 21:08:26 +03:00
Oran Agra a5696bdf4f Run daily CI on PRs to release a branch 2020-07-20 21:08:26 +03:00
Itamar Haber e15528bf1d Adds SHA256SUM to redis-stable tarball upload
(cherry picked from commit 5df0a64d30)
2020-07-20 21:08:26 +03:00
yoav-steinberg e28aa99af1 Support passing stack allocated module strings to moduleCreateArgvFromUserFormat (#7528)
Specifically, the key passed to the module aof_rewrite callback is a stack allocated robj. When passing it to RedisModule_EmitAOF (with appropriate "s" fmt string) redis used to panic when trying to inc the ref count of the stack allocated robj. Now support such robjs by coying them to a new heap robj. This doesn't affect performance because using the alternative "c" or "b" format strings also copies the input to a new heap robj.

(cherry picked from commit d484b8a04e)
2020-07-20 21:08:26 +03:00
Luke Palmer 3051430040 Send null for invalidate on flush (#7469)
(cherry picked from commit 5f716ea467)
2020-07-20 21:08:26 +03:00
dmurnane 29b20fd528 Notify systemd on sentinel startup (#7168)
Co-authored-by: Daniel Murnane <dmurnane@eitccorp.com>
(cherry picked from commit 9242ccf238)
2020-07-20 21:08:26 +03:00
Developer-Ecosystem-Engineering 5b36681213 Add registers dump support for Apple silicon (#7453)
Export following environment variables before building on macOS on Apple silicon

export ARCH_FLAGS="-arch arm64"
export SDK_NAME=macosx
export SDK_PATH=$(xcrun --show-sdk-path --sdk $SDK_NAME)
export CFLAGS="$ARCH_FLAGS -isysroot $SDK_PATH -I$SDK_PATH/usr/include"
export CXXFLAGS=$CFLAGS
export LDFLAGS="$ARCH_FLAGS"
export CC="$(xcrun -sdk $SDK_PATH --find clang) $CFLAGS"
export CXX="$(xcrun -sdk $SDK_PATH --find clang++) $CXXFLAGS"
export LD="$(xcrun -sdk $SDK_PATH --find ld) $LDFLAGS"

make
make test
..
All tests passed without errors!

Backtrack logging assumes x86 and required updating

(cherry picked from commit c2b5f1c15b)
2020-07-20 21:08:26 +03:00
Wen Hui 0f75036c07 correct error msg for num connections reaching maxclients in cluster mode (#7444)
(cherry picked from commit d85af4d6f5)
2020-07-20 21:08:26 +03:00
WuYunlong b1a01fda91 Fix command help for unexpected options (#7476)
(cherry picked from commit 93bdbf5aa4)
2020-07-20 21:08:26 +03:00
WuYunlong 83f55f61a6 Refactor RM_KeyType() by using macro. (#7486)
(cherry picked from commit dc690161d5)
2020-07-20 21:08:26 +03:00
Oran Agra f89f50dbd0 diskless master disconnect replicas when rdb child failed (#7518)
in case the rdb child failed, crashed or terminated unexpectedly redis
would have marked the replica clients with repl_put_online_on_ack and
then kill them only after a minute when no ack was received.

it would not stream anything to these connections, so the only effect of
this bug is a delay of 1 minute in the replicas attempt to re-connect.

(cherry picked from commit a176cb56a3)
2020-07-20 21:08:26 +03:00
Oran Agra 05f8975d21 redis-cli tests, fix valgrind timing issue (#7519)
this test when run with valgrind on github actions takes 160 seconds

(cherry picked from commit 254c962554)
2020-07-20 21:08:26 +03:00
WuYunlong 4780cc5efa Fix out of update help info in tcl tests. (#7516)
Before this commit, the output of "./runtest-cluster --help" is incorrect.
After this commit, the format of the following 3 output is consistent:
./runtest --help
./runtest-cluster --help
./runtest-sentinel --help

(cherry picked from commit 8128d39737)
2020-07-20 21:08:26 +03:00
Qu Chen 9bc6af2449 Replica always reports master's config epoch in CLUSTER NODES output. (#7235)
(cherry picked from commit 938c35302f)
2020-07-20 21:08:26 +03:00
Oran Agra c4b428a388 RESTORE ABSTTL skip expired keys - leak (#7511)
(cherry picked from commit 6a81450144)
2020-07-20 21:08:26 +03:00
Oran Agra aea4db2f5a fix recently added time sensitive tests failing with valgrind (#7512)
interestingly the latency monitor test fails because valgrind is slow
enough so that the time inside PEXPIREAT command from the moment of
the first mstime() call to get the basetime until checkAlreadyExpired
calls mstime() again is more than 1ms, and that test was too sensitive.

using this opportunity to speed up the test (unrelated to the failure)
the fix is just the longer time passed to PEXPIRE.

(cherry picked from commit e5227aab89)
2020-07-20 21:08:26 +03:00
Oran Agra b5c5f870a4 runtest --stop pause stops before terminating the redis server (#7513)
in the majority of the cases (on this rarely used feature) we want to
stop and be able to connect to the shard with redis-cli.
since these are two different processes interracting with the tty we
need to stop both, and we'll have to hit enter twice, but it's not that
bad considering it is rarely used.

(cherry picked from commit 02ef355f98)
2020-07-20 21:08:26 +03:00
Oran Agra 23bf3d1cb9 update release scripts for new hosts, and CI to run more tests (#7480)
* update daily CI to include cluster and sentinel tests
* update daily CI to run when creating a new release
* update release scripts to work on the new redis.io hosts

(cherry picked from commit 7f19a04f0f)
2020-07-20 21:08:26 +03:00
jimgreen2013 1be6dbbdf6 fix description about ziplist, the code is ok (#6318)
* fix description about ZIP_BIG_PREVLEN(the code is ok), it's similar to
antirez#4705

* fix description about ziplist entry encoding field (the code is ok),
the max length should be 2^32 - 1 when encoding is 5 bytes

(cherry picked from commit 67660881ed)
2020-07-20 21:08:26 +03:00
马永泽 1acdc84d10 fix benchmark in cluster mode fails to authenticate (#7488)
Co-authored-by: Oran Agra <oran@redislabs.com> (styling)
(cherry picked from commit 279b4a1464)
2020-07-20 21:08:26 +03:00
Abhishek Soni 52929b61f8 fix: typo in CI job name (#7466)
(cherry picked from commit d5648d617e)
2020-07-20 21:08:26 +03:00
Jiayuan Chen b0faf6113e Fix typo in deps README (#7500)
(cherry picked from commit af39d750cd)
2020-07-20 21:08:26 +03:00
WuYunlong f838df92d2 Add missing latency-monitor tcl test to test_helper.tcl. (#6782)
(cherry picked from commit d792db7948)
2020-07-20 21:08:26 +03:00
Yossi Gottlieb 7a536c2912 TLS: Session caching configuration support. (#7420)
* TLS: Session caching configuration support.
* TLS: Remove redundant config initialization.

(cherry picked from commit 3e6f2b1a45)
2020-07-20 21:08:26 +03:00
Yossi Gottlieb 6f1e828454 TLS: Ignore client cert when tls-auth-clients off. (#7457)
(cherry picked from commit 5266293a0f)
2020-07-20 21:08:26 +03:00
James Hilliard 2d3687655b Use pkg-config to properly detect libssl and libcrypto libraries (#7452)
(cherry picked from commit 6a014af79a)
2020-07-20 21:08:26 +03:00
Yossi Gottlieb b057ff81ee TLS: Add missing redis-cli options. (#7456)
* Tests: fix and reintroduce redis-cli tests.

These tests have been broken and disabled for 10 years now!

* TLS: add remaining redis-cli support.

This adds support for the redis-cli --pipe, --rdb and --replica options
previously unsupported in --tls mode.

* Fix writeConn().

(cherry picked from commit d9f970d8d3)
2020-07-20 21:08:26 +03:00
Oran Agra a9d1014fb2 redis-cli --hotkeys fixed to handle non-printable key names
(cherry picked from commit b23e251036)
2020-07-20 21:08:26 +03:00
Oran Agra 183bdc11c0 redis-cli --bigkeys fixed to handle non-printable key names
(cherry picked from commit 6f8a8647de)
2020-07-20 21:08:26 +03:00
Oran Agra 95ba01b538 RESTORE ABSTTL won't store expired keys into the db (#7472)
Similarly to EXPIREAT with TTL in the past, which implicitly deletes the
key and return success, RESTORE should not store key that are already
expired into the db.
When used together with REPLACE it should emit a DEL to keyspace
notification and replication stream.

(cherry picked from commit 5977a94842)
2020-07-20 21:08:26 +03:00
huangzhw 7b21b8c3fb defrag.c activeDefragSdsListAndDict when defrag sdsele, We can't use (#7492)
it to calculate hash, we should use newsds.

(cherry picked from commit d6180c8c86)
2020-07-20 21:08:26 +03:00
Oran Agra 33ca884cc5 skip a test that uses +inf on valgrind (#7440)
On some platforms strtold("+inf") with valgrind returns a non-inf result

[err]: INCRBYFLOAT does not allow NaN or Infinity in tests/unit/type/incr.tcl
Expected 'ERR*would produce*' to equal or match '1189731495357231765085759.....'

(cherry picked from commit 909bc97c52)
2020-07-20 21:08:26 +03:00
Oran Agra 2b5f23197c stabilize tests that look for log lines (#7367)
tests were sensitive to additional log lines appearing in the log
causing the search to come empty handed.

instead of just looking for the n last log lines, capture the log lines
before performing the action, and then search from that offset.

(cherry picked from commit 8e76e13472)
2020-07-20 21:08:26 +03:00
Oran Agra 1104113c07 tests/valgrind: don't use debug restart (#7404)
* tests/valgrind: don't use debug restart

DEBUG REATART causes two issues:
1. it uses execve which replaces the original process and valgrind doesn't
   have a chance to check for errors, so leaks go unreported.
2. valgrind report invalid calls to close() which we're unable to resolve.

So now the tests use restart_server mechanism in the tests, that terminates
the old server and starts a new one, new PID, but same stdout, stderr.

since the stderr can contain two or more valgrind report, it is not enough
to just check for the absence of leaks, we also need to check for some known
errors, we do both, and fail if we either find an error, or can't find a
report saying there are no leaks.

other changes:
- when killing a server that was already terminated we check for leaks too.
- adding DEBUG LEAK which was used to test it.
- adding --trace-children to valgrind, although no longer needed.
- since the stdout contains two or more runs, we need slightly different way
  of checking if the new process is up (explicitly looking for the new PID)
- move the code that handles --wait-server to happen earlier (before
  watching the startup message in the log), and serve the restarted server too.

* squashme - CR fixes

(cherry picked from commit 69ade87325)
2020-07-20 21:08:26 +03:00
Oran Agra 62f8583445 change references to the github repo location (#7479)
(cherry picked from commit 9bbf768d3c)
2020-07-20 21:08:26 +03:00
zhaozhao.zz 3155adb299 BITOP: propagate only when it really SET or DEL targetkey (#5783)
For example:
    BITOP not targetkey sourcekey

If targetkey and sourcekey doesn't exist, BITOP has no effect,
we do not propagate it, thus can save aof and replica flow.

(cherry picked from commit 1978f996d8)
2020-07-20 21:08:26 +03:00
antirez 14a59d4ce7 Update comment to clarify change in #7398.
(cherry picked from commit ad0a9df77a)
2020-07-20 21:08:26 +03:00
antirez 17aaf5ec97 LPOS: option FIRST renamed RANK.
(cherry picked from commit a5a3a7bbc6)
2020-07-20 21:08:26 +03:00
Dave Nielsen 3f5cf24da9 updated copyright year
Changed "2015" to "2020"

(cherry picked from commit 2d6d9f7552)
2020-07-20 21:08:26 +03:00
Oran Agra 05e483cbb3 EXEC always fails with EXECABORT and multi-state is cleared
In order to support the use of multi-exec in pipeline, it is important that
MULTI and EXEC are never rejected and it is easy for the client to know if the
connection is still in multi state.

It was easy to make sure MULTI and DISCARD never fail (done by previous
commits) since these only change the client state and don't do any actual
change in the server, but EXEC is a different story.

Since in the past, it was possible for clients to handle some EXEC errors and
retry the EXEC, we now can't affort to return any error on EXEC other than
EXECABORT, which now carries with it the real reason for the abort too.

Other fixes in this commit:
- Some checks that where performed at the time of queuing need to be re-
  validated when EXEC runs, for instance if the transaction contains writes
  commands, it needs to be aborted. there was one check that was already done
  in execCommand (-READONLY), but other checks where missing: -OOM, -MISCONF,
  -NOREPLICAS, -MASTERDOWN
- When a command is rejected by processCommand it was rejected with addReply,
  which was not recognized as an error in case the bad command came from the
  master. this will enable to count or MONITOR these errors in the future.
- make it easier for tests to create additional (non deferred) clients.
- add tests for the fixes of this commit.

(cherry picked from commit 65a3307bc9)
2020-07-20 21:08:26 +03:00
antirez c8f250f8dd Include cluster.h for getClusterConnectionsCount().
(cherry picked from commit 21f62c3346)
2020-07-20 21:08:26 +03:00
antirez 9412094f25 Fix BITFIELD i64 type handling, see #7417.
(cherry picked from commit 746297314f)
2020-07-20 21:08:26 +03:00
antirez 8312aa27d4 Clarify maxclients and cluster in conf. Remove myself too.
(cherry picked from commit 59fd178014)
2020-07-20 21:08:26 +03:00
hwware d3aa3791fe fix memory leak in sentinel connection sharing
(cherry picked from commit 1bfa2d27a6)
2020-07-20 21:08:26 +03:00
chenhui0212 2b189f098e Fix comments in function raxLowWalk of listpack.c
(cherry picked from commit f800b52172)
2020-07-20 21:08:26 +03:00
Tomasz Poradowski 3328b7a514 ensure SHUTDOWN_NOSAVE in Sentinel mode
- enforcing of SHUTDOWN_NOSAVE flag in one place to make it consitent
  when running in Sentinel mode

(cherry picked from commit 4ee011adb5)
2020-07-20 21:08:26 +03:00
chenhui0212 88e1f80eee fix comments in listpack.c
(cherry picked from commit 71fafd761a)
2020-07-20 21:08:26 +03:00
antirez 0ebbc36059 Use cluster connections too, to limit maxclients.
See #7401.

(cherry picked from commit 4b8d8826af)
2020-07-20 21:08:26 +03:00
antirez 43ed3c3589 Tracking: fix enableBcastTrackingForPrefix() invalid sdslen() call.
Related to #7387.

(cherry picked from commit 784479939d)
2020-07-20 21:08:26 +03:00
root 8095daea4a cluster.c remove if of clusterSendFail in markNodeAsFailingIfNeeded
(cherry picked from commit c92464db69)
2020-07-20 21:08:26 +03:00
meir@redislabs.com 51e178454d Fix RM_ScanKey module api not to return int encoded strings
The scan key module API provides the scan callback with the current
field name and value (if it exists). Those arguments are RedisModuleString*
which means it supposes to point to robj which is encoded as a string.
Using createStringObjectFromLongLong function might return robj that
points to an integer and so break a module that tries for example to
use RedisModule_StringPtrLen on the given field/value.

The PR introduces a fix that uses the createObject function and sdsfromlonglong function.
Using those function promise that the field and value pass to the to the
scan callback will be Strings.

The PR also changes the Scan test module to use RedisModule_StringPtrLen
to catch the issue. without this, the issue is hidden because
RedisModule_ReplyWithString knows to handle integer encoding of the
given robj (RedisModuleString).

The PR also introduces a new test to verify the issue is solved.

(cherry picked from commit a89bf734a9)
2020-07-20 21:08:26 +03:00
antirez 82b2bfd20b Fix LCS object type checking. Related to #7379.
(cherry picked from commit 1055398849)
2020-07-20 21:08:26 +03:00
hwware ec1faeec72 fix memory leak
(cherry picked from commit 7008a0ba66)
2020-07-20 21:08:26 +03:00
hwware c1326d7b10 fix server crash in STRALGO command
(cherry picked from commit 2a05fa0d48)
2020-07-20 21:08:26 +03:00
Benjamin Sergeant 845fb2d1c1 Update redis-cli.c
(cherry picked from commit 93021da221)
2020-07-20 21:08:26 +03:00
Jamie Scott 930fac8058 minor fix
(cherry picked from commit 4f3b15e6ea)
2020-07-20 21:08:26 +03:00
John Sully 119797014c Remove gitter, we don't check it often enough 2020-07-13 21:14:03 -04:00
John Sully e8f568644b Merge branch 'unstable' of https://github.com/JohnSully/KeyDB into unstable 2020-07-13 01:05:17 +00:00
John Sully b736a95b0d Merge tag '6.0.5' into unstable
Redis 6.0.5
2020-07-13 00:55:23 +00:00
benschermel d909d12cb3 Merge branch 'unstable' of https://github.com/JohnSully/KeyDB into unstable 2020-07-12 18:04:40 -04:00
benschermel 201ffa9d62 update rpm permissions 2020-07-12 18:03:21 -04:00
John Sully 5619f51528 Add the KEYDB.MEXISTS command, see issue #203 2020-07-12 21:42:11 +00:00
John Sully d99d06b125 Add multi-master-no-forward command to reduce bus traffic with multi-master 2020-07-12 19:25:19 +00:00
John Sully 1b9ade7f65 Document min-clients-per-thread 2020-07-12 18:49:02 +00:00
John Sully e01312642b Prevent deadlock in RM_ThreadSafeContextLock() when triggered as part of a module callback in a server thread 2020-07-12 18:17:53 +00:00
John Sully cd9514f4c8 Fix failure to merge databases on active replica sync, due to bad merge with Redis 6 2020-07-12 01:13:22 +00:00
John Sully f111c23415 Support missing Redis 6 config options 2020-07-11 21:39:20 +00:00
John Sully 6857c4c208 Latency fixes 2020-07-01 21:59:38 -04:00
Muhammad Zahalqa 52897f8f9a replace memcpy with cctor on objects 2020-06-24 23:20:59 -04:00
Muhammad Zahalqa 8bc70e9840 unique lock used with cv.wait already owns mutex 2020-06-24 23:18:53 -04:00
Muhammad Zahalqa 5b81303f04 Fix bug in condition_variabe.wait usage
consition variable wait should always be called with a locked mutex.
This code break the pre-conditions of using wait.
2020-06-24 23:18:53 -04:00
Muhammad Zahalqa ede524c064 fixes for robj_sharedptr
1. fix cases where null pointer might be accessed
2. make assignmnet op safe
3. make operator bool explicit (safe bool idiom)
4. make comparison operators symetric

fix robj_sharedptr use in rdb.cpp
2020-06-24 23:18:27 -04:00
Muhammad Zahalqa aeed09c346 aeCommand objects no need for memset and missing init of some memebers. 2020-06-24 23:15:29 -04:00
antirez 0cae0900cf help.h updated. 2020-06-12 12:16:24 +02:00
antirez e0022d8cfe LPOS: tests + crash fix. 2020-06-12 12:08:06 +02:00
antirez 495327c020 LPOS: update to latest proposal.
See https://gist.github.com/antirez/3591c5096bc79cad8b5a992e08304f48
2020-06-12 12:08:06 +02:00
antirez afb8b3c030 LPOS: implement the final design. 2020-06-12 12:08:06 +02:00
Paul Spooren 31c9cd8604 LRANK: Add command (the command will be renamed LPOS).
The `LRANK` command returns the index (position) of a given element
within a list. Using the `direction` argument it is possible to specify
going from head to tail (acending, 1) or from tail to head (decending,
-1). Only the first found index is returend. The complexity is O(N).

When using lists as a queue it can be of interest at what position a
given element is, for instance to monitor a job processing through a
work queue. This came up within the Python `rq` project which is based
on Redis[0].

[0]: https://github.com/rq/rq/issues/1197

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-06-12 12:08:06 +02:00
John Sully 70b80aa5fa Implement replicaof remove as requested in issue #192 2020-06-09 12:43:58 -04:00
antirez 51efb7fe25 Redis 6.0.5. 2020-06-09 12:19:30 +02:00
antirez a7d3670da5 Adapt EVAL+busy script test to new behavior. 2020-06-09 12:19:30 +02:00
antirez ee8dd01bbc Temporary fix for #7353 issue about EVAL during -BUSY. 2020-06-09 11:53:01 +02:00
xhe a4a856d532 return the correct proto version
HELLO should return the current proto version, while the code hardcoded
3
2020-06-09 11:53:01 +02:00
Oran Agra e2046b3003 Don't queue commands in an already aborted MULTI state 2020-06-09 11:53:01 +02:00
Oran Agra b35fdf1de1 Avoid rejecting WATCH / UNWATCH, like MULTI/EXEC/DISCARD
Much like MULTI/EXEC/DISCARD, the WATCH and UNWATCH are not actually
operating on the database or server state, but instead operate on the
client state. the client may send them all in one long pipeline and check
all the responses only at the end, so failing them may lead to a
mismatch between the client state on the server and the one on the
client end, and execute the wrong commands (ones that were meant to be
discarded)

the watched keys are not actually stored in the client struct, but they
are in fact part of the client state. for instance, they're not cleared
or moved in SWAPDB or FLUSHDB.
2020-06-09 11:53:01 +02:00
zhaozhao.zz 1d7bf208ce AOF: append origin SET if no expire option 2020-06-09 11:53:01 +02:00
Oran Agra 676445ad95 fix disconnectSlaves, to try to free each slave.
the recent change in that loop (iteration rather than waiting for it to
be empty) was intended to avoid an endless loop in case some slave would
refuse to be freed.

but the lookup of the first client remained, which would have caused it
to try the first one again and again instead of moving on.
2020-06-09 11:53:01 +02:00
zhaozhao.zz 4846c0c8af donot free protected client in freeClientsInAsyncFreeQueue
related #7234
2020-06-09 11:53:01 +02:00
John Sully e8d4d095cc Update gcov for C++ 2020-06-08 16:48:44 -04:00
Ben Schermel c50a3568a5 update Dockerfile 2020-06-08 15:34:22 -04:00
Ben Schermel 17e96b91b5 updating changelog 2020-06-08 15:34:22 -04:00
Ben Schermel c5e7a8517b dh9 for stretch deb pkg 2020-06-08 15:34:22 -04:00
Ben Schermel c36773777d fix missing sentinel 2020-06-08 15:34:22 -04:00
Ben Schermel d69e0b8561 update changelog 2020-06-08 15:34:22 -04:00
Ben Schermel 19ccc22a43 update changelog 2020-06-08 15:34:22 -04:00
Ben Schermel 6968a0cfcb update changelog 2020-06-08 15:34:22 -04:00
Ben Schermel df618fbd9f include support for dists < dh11 2020-06-08 15:34:22 -04:00
Ben Schermel 94acd43340 fix rpmbuild dirs 2020-06-08 15:34:22 -04:00
Ben Schermel 14bdd745d1 fix rpmbuild dirs 2020-06-08 15:34:22 -04:00
Ben Schermel aa3f5de16f adding dh_installsystemd 2020-06-08 15:34:22 -04:00
Ben Schermel 9f7890cb1a strip unused from deb source 2020-06-08 15:34:22 -04:00
benschermel 1dc34c16b5 add deb/rpm/docker support 2020-06-08 15:34:22 -04:00
benschermel b0e7be7ce2 add deb/rpm/docker support 2020-06-08 15:34:22 -04:00
benschermel 026ab4c922 update github CI for deb 2020-06-08 15:34:22 -04:00
Ben Schermel 07ba64c200 update rpm versioning 2020-06-08 15:34:22 -04:00
benschermel b4b35cf904 add package utilities 2020-06-08 15:34:22 -04:00
John Sully ba974c9810 Some valgrind fixes 2020-06-07 23:20:10 -04:00
John Sully 9817db7edd fix valgrind false error 2020-06-07 20:17:47 -04:00
John Sully 515bb0e6bd Endurance mode for tests ignore flaky ones 2020-06-07 16:59:59 -04:00
John Sully bc7ea8fb20 Solo tests should work with loopn 2020-06-07 15:06:50 -04:00
John Sully 2caf8e4c80 Stream tests rely on deferring clients, mitigate races caused by them 2020-06-07 13:59:41 -04:00
John Sully e1f3cd6ec3 replication test race 2020-06-07 01:14:57 -04:00
John Sully 02e821530a fix race in test 2020-06-07 00:44:11 -04:00
John Sully b6c3d80edb Make lazyfree more reliable 2020-06-07 00:40:58 -04:00
Oran Agra f33de403ed fix pingoff test race 2020-06-06 11:44:21 +02:00
Kevin Fwu 49af4d07e4 Fix TLS certificate loading for chained certificates.
This impacts client verification for chained certificates (such as Lets
Encrypt certificates). Client Verify requires the full chain in order to
properly verify the certificate.
2020-06-06 11:44:21 +02:00
antirez 329fddbda1 Revert "Implements sendfile for redis."
This reverts commit 9cf500a3f6.
2020-06-06 11:43:29 +02:00
antirez 925a2cd5aa Revert "avoid using sendfile if tls-replication is enabled"
This reverts commit b9abecfc4c.
2020-06-06 11:43:29 +02:00
Liu Zhen 84a7a90586 fix clusters mixing accidentally by gossip
`clusterStartHandshake` will start hand handshake
and eventually send CLUSTER MEET message, which is strictly prohibited
in the REDIS CLUSTER SPEC.
Only system administrator can initiate CLUSTER MEET message.
Futher, according to the SPEC, rather than IP/PORT pairs, only nodeid
can be trusted.
2020-06-06 11:43:29 +02:00
antirez cd63359a17 Fix handling of special chars in ACL LOAD.
Now it is also possible for ACL SETUSER to accept empty strings
as valid operations (doing nothing), so for instance

    ACL SETUSER myuser ""

Will have just the effect of creating a user in the default state.

This should fix #7329.
2020-06-06 11:43:29 +02:00
John Sully 0042586190 Detect issues like #189 2020-06-05 22:18:28 -04:00
John Sully c6dce838b7 Ensure CRON jobs run in a clean environment 2020-06-05 21:35:47 -04:00
John Sully 5b885bb164 Undecorated new is OK to use 2020-06-01 23:36:01 -04:00
John Sully 95f46e6077 maxmem tests should be run solo for higher reliability 2020-06-01 16:50:30 -04:00
John Sully 31f58311e3 Remove debug logs that shouldn't have been checked in 2020-06-01 16:34:05 -04:00
John Sully 130613e166 PSYNC test shouldn't wait forever 2020-06-01 16:13:58 -04:00
John Sully 50fd4fa7e6 PSYNC test reliability improvements (test only issue) 2020-06-01 16:01:26 -04:00
John Sully e6a9a6b428 Fix for issue #187 we need to properly handle the case where a key with a subkey expirey itself expires during load 2020-06-01 15:33:19 -04:00
John Sully d6be93537e Update endurance.yml 2020-06-01 00:25:31 -04:00
John Sully 1ef35cf466 Fix module multithreaded test failures 2020-05-31 23:46:12 -04:00
John Sully 1eb0e3c1c6 sendfile has high latency in some scenarios, don't use it 2020-05-31 23:22:25 -04:00
John Sully e50313b207 we must always respect the output of sdscat, also change the string so its not a substring of other config params 2020-05-31 23:22:10 -04:00
John Sully ac3f8d1ac7 Do another iteration for endurance 2020-05-31 22:51:02 -04:00
John Sully 202b97eff7 Don't start multimaster tests until all nodes are connected 2020-05-31 22:50:30 -04:00
John Sully f25aebf269 Fix client race in test (test only issue) 2020-05-31 21:39:50 -04:00
John Sully 9b1e636a66 --loop should run forever 2020-05-31 21:27:15 -04:00
John Sully 5147f0153a PUBSUB test reliability: A client race was erroneously failing tests (test only issue) 2020-05-31 21:27:15 -04:00
John Sully 88f7739ae0 Update endurance.yml 2020-05-31 17:50:02 -04:00
John Sully c0c033a0c1 Active replicas referencing eachother should connect one at a time 2020-05-31 16:03:50 -04:00
John Sully 73300c5700 Fix race in futex_sleep 2020-05-31 16:03:50 -04:00
John Sully 37033179fe Update endurance.yml 2020-05-31 14:18:12 -04:00
John Sully 3e6a5ac9cf Update endurance.yml 2020-05-31 03:43:06 -04:00
John Sully dcd297eb5b Endurance tests need more cores 2020-05-30 22:16:56 -04:00
John Sully 4a9742e696 Update endurance.yml 2020-05-30 20:03:04 -04:00
John Sully 8edbae2e04 Auto tune lock for high CPU tension scenarios 2020-05-30 18:22:27 -04:00
John Sully 63c6735ffc Ensure endurance run can complete in an hour 2020-05-30 15:31:54 -04:00
John Sully 8a0441c144 We only want one master connection in progress at a time, ensure that if the connection fails we try a different master 2020-05-30 15:13:28 -04:00
John Sully 460c5ed2b3 daily CI config fix 2020-05-30 03:27:37 -04:00
John Sully 8f44f0937b More clients for endurance 2020-05-30 03:13:04 -04:00
John Sully 1803b1587b Endurance tests need to be faster, reduce loops 2020-05-30 02:40:44 -04:00
John Sully a5325061f2 Only CI ARM daily 2020-05-30 02:34:37 -04:00
John Sully a10a0df09f Also test with multiple threads 2020-05-30 02:10:00 -04:00
John Sully 031044c7ea Reduce clients for endurance test 2020-05-30 02:07:30 -04:00
John Sully ca755e4905 Fix cron syntax 2020-05-30 01:53:21 -04:00
John Sully 3063a028b3 Add endurance CI tests 2020-05-30 01:47:26 -04:00
John Sully 2c659acc7d Update daily CI job 2020-05-30 01:21:50 -04:00
John Sully 341a8caef9 TLS requires we explicitly marshal connected sockets when using them on a different thread 2020-05-30 01:14:53 -04:00
John Sully 366f8c2dc6 Merge branch 'unstable' of https://github.com/JohnSully/KeyDB into unstable 2020-05-30 00:06:23 -04:00
John Sully 5803540422 Fix memory test failures with multithreading enabled 2020-05-30 00:06:15 -04:00
Ben Schermel f17c492a63 centos build fix 2020-05-29 22:23:48 -04:00
John Sully aad6a7ce15 sendFile blocks too long for use with active replication 2020-05-29 20:27:45 -04:00
John Sully 1cb48c7bf6 Unify beforeSleep handling 2020-05-29 17:44:55 -04:00
John Sully f25c405ad2 Cluster crash 2020-05-29 03:40:52 -04:00
John Sully c9920849dd active replica tests on slow computers 2020-05-29 01:58:15 -04:00
John Sully 81b240f81d Fix test issue with TLS 2020-05-29 01:44:52 -04:00
John Sully 6a902b29e1 Fix CI 2020-05-29 01:01:41 -04:00
John Sully 9c31ac7925 Merge tag '6.0.4' into unstable
Redis 6.0.4.
2020-05-29 00:57:07 -04:00
John Sully f0d8d1680f Fix TLS tests 2020-05-29 00:16:06 -04:00
John Sully 6dcbcacb10 gencert its own step 2020-05-28 23:49:13 -04:00
John Sully 3196717cee CI yml syntax error 2020-05-28 23:18:02 -04:00
John Sully f95578ac02 Fix TLS tests in CI 2020-05-28 23:16:42 -04:00
John Sully 57ca6facc3 fix TLS test failure 2020-05-28 23:16:27 -04:00
antirez 886d287c40 Redis 6.0.4. 2020-05-28 12:18:38 +02:00
antirez 59cd4c9f65 Test: take PSYNC2 test master timeout high during switch.
This will likely avoid false positives due to trailing pings.
2020-05-28 10:56:14 +02:00
antirez 6c1bb7b194 Test: add the tracking unit as default. 2020-05-28 10:22:29 +02:00
Oran Agra 1aee695e52 tests: find_available_port start search from next port
i.e. don't start the search from scratch hitting the used ones again.
this will also reduce the likelihood of collisions (if there are any
left) by increasing the time until we re-use a port we did use in the
past.
2020-05-28 10:09:51 +02:00
Oran Agra a2ae463520 tests: each test client work on a distinct port range
apparently when running tests in parallel (the default of --clients 16),
there's a chance for two tests to use the same port.
specifically, one test might shutdown a master and still have the
replica up, and then another test will re-use the port number of master
for another master, and then that replica will connect to the master of
the other test.

this can cause a master to count too many full syncs and fail a test if
we run the tests with --single integration/psync2 --loop --stop

see Probmem 2 in #7314
2020-05-28 10:09:51 +02:00
Oran Agra 86e562d691 32bit CI needs to build modules correctly 2020-05-28 10:09:51 +02:00
Oran Agra ab2984b1e2 adjust revived meaningful offset tests
these tests create several edge cases that are otherwise uncovered (at
least not consistently) by the test suite, so although they're no longer
testing what they were meant to test, it's still a good idea to keep
them in hope that they'll expose some issue in the future.
2020-05-28 10:09:51 +02:00
Oran Agra 1ff5a222de revive meaningful offset tests 2020-05-28 10:09:51 +02:00
antirez cc549b46a4 Replication: showLatestBacklog() refactored out. 2020-05-28 10:09:51 +02:00
antirez 377dd0515c Drop useless line from replicationCacheMaster(). 2020-05-28 10:09:51 +02:00
antirez 3f8d113f1b Another meaningful offset test removed. 2020-05-28 10:09:51 +02:00
antirez d4541349dc Remove the PSYNC2 meaningful offset test. 2020-05-28 10:09:51 +02:00
antirez 2112a57028 Remove the meaningful offset feature.
After a closer look, the Redis core devleopers all believe that this was
too fragile, caused many bugs that we didn't expect and that were very
hard to track. Better to find an alternative solution that is simpler.
2020-05-28 10:09:51 +02:00
antirez d2eb6e0b47 Set a protocol error if master use the inline protocol.
We want to react a bit more aggressively if we sense that the master is
sending us some corrupted stream. By setting the protocol error we both
ensure that the replica will disconnect, and avoid caching the master so
that a full SYNC will be required. This is protective against
replication bugs.
2020-05-28 10:09:51 +02:00
Oran Agra 9c1df3b769 daily CI test with tls 2020-05-28 10:09:51 +02:00
Oran Agra 115ed1911e avoid using sendfile if tls-replication is enabled
this obviously broke the tests, but went unnoticed so far since tls
wasn't often tested.
2020-05-28 10:09:51 +02:00
antirez 11c748aaca Replication: log backlog creation event. 2020-05-28 10:09:51 +02:00
antirez 8f10137227 Test: PSYNC2 test can now show server logs. 2020-05-28 10:09:51 +02:00
John Sully 561a1f8d8a Test TLS as part of CI 2020-05-26 01:39:54 -04:00
antirez 2e591fc4a4 Clarify what is happening in PR #7320. 2020-05-25 12:08:01 +02:00
zhaozhao.zz cbb51fb8f9 PSYNC2: second_replid_offset should be real meaningful offset
After adjustMeaningfulReplOffset(), all the other related variable
should be updated, including server.second_replid_offset.

Or the old version redis like 5.0 may receive wrong data from
replication stream, cause redis 5.0 can sync with redis 6.0,
but doesn't know meaningful offset.
2020-05-25 12:08:01 +02:00
Oran Agra e0fc88b4d9 add CI for 32bit build 2020-05-25 12:08:01 +02:00
antirez e3f864b5fd Make disconnectSlaves() synchronous in the base case.
Otherwise we run into that:

Backtrace:
src/redis-server 127.0.0.1:21322(logStackTrace+0x45)[0x479035]
src/redis-server 127.0.0.1:21322(sigsegvHandler+0xb9)[0x4797f9]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fd373c5e390]
src/redis-server 127.0.0.1:21322(_serverAssert+0x6a)[0x47660a]
src/redis-server 127.0.0.1:21322(freeReplicationBacklog+0x42)[0x451282]
src/redis-server 127.0.0.1:21322[0x4552d4]
src/redis-server 127.0.0.1:21322[0x4c5593]
src/redis-server 127.0.0.1:21322(aeProcessEvents+0x2e6)[0x42e786]
src/redis-server 127.0.0.1:21322(aeMain+0x1d)[0x42eb0d]
src/redis-server 127.0.0.1:21322(main+0x4c5)[0x42b145]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fd3738a3830]
src/redis-server 127.0.0.1:21322(_start+0x29)[0x42b409]

Since we disconnect all the replicas and free the replication backlog in
certain replication paths, and the code that will free the replication
backlog expects that no replica is connected.

However we still need to free the replicas asynchronously in certain
cases, as documented in the top comment of disconnectSlaves().
2020-05-25 12:08:01 +02:00
ShooterIT 8af1e513f6 Implements sendfile for redis. 2020-05-25 12:08:01 +02:00
antirez 3c21418cd8 Fix #7306 less aggressively.
Citing from the issue:

btw I suggest we change this fix to something else:
* We revert the fix.
* We add a call that disconnects chained replicas in the place where we trim the replica (that is a master i this case) offset.
This way we can avoid disconnections when there is no trimming of the backlog.

Note that we now want to disconnect replicas asynchronously in
disconnectSlaves(), because it's in general safer now that we can call
it from freeClient(). Otherwise for instance the command:

    CLIENT KILL TYPE master

May crash: clientCommand() starts running the linked of of clients,
looking for clients to kill. However it finds the master, kills it
calling freeClient(), but this in turn calls replicationCacheMaster()
that may also call disconnectSlaves() now. So the linked list iterator
of the clientCommand() will no longer be valid.
2020-05-25 12:08:01 +02:00
Madelyn Olson e201f83cee EAGAIN for tls during diskless load 2020-05-22 12:37:59 +02:00
Qu Chen 58fc456cbd Disconnect chained replicas when the replica performs PSYNC with the master always to avoid replication offset mismatch between master and chained replicas. 2020-05-22 12:37:59 +02:00
hwware 3febc5c29e using moreargs variable 2020-05-22 12:37:59 +02:00
hwware 8d67385593 fix server crash for STRALGO command 2020-05-22 12:37:49 +02:00
ShooterIT 7a35eec540 Replace addDeferredMultiBulkLength with addReplyDeferredLen in comment 2020-05-22 12:37:49 +02:00
Yossi Gottlieb f93e1417b9 TLS: Improve tls-protocols clarity in redis.conf. 2020-05-22 12:37:49 +02:00
ShooterIT d0c9e44545 Fix reply bytes calculation error
Fix #7275.
2020-05-22 12:37:49 +02:00
zhaozhao.zz 1cde6a0603 Tracking: flag CLIENT_TRACKING_BROKEN_REDIR when redir broken 2020-05-22 12:37:49 +02:00
Oran Agra 436be34986 fix a rare active defrag edge case bug leading to stagnation
There's a rare case which leads to stagnation in the defragger, causing
it to keep scanning the keyspace and do nothing (not moving any
allocation), this happens when all the allocator slabs of a certain bin
have the same % utilization, but the slab from which new allocations are
made have a lower utilization.

this commit fixes it by removing the current slab from the overall
average utilization of the bin, and also eliminate any precision loss in
the utilization calculation and move the decision about the defrag to
reside inside jemalloc.

and also add a test that consistently reproduce this issue.
2020-05-22 12:37:49 +02:00
Oran Agra f9d2ffdc5d improve DEBUG MALLCTL to be able to write to write only fields.
also support:
  debug mallctl-str thread.tcache.flush VOID
2020-05-22 12:37:49 +02:00
hujie d7968ee920 fix clear USER_FLAG_ALLCOMMANDS flag in acl
in ACLSetUserCommandBit, when the command bit overflows, no operation
is performed, so no need clear the USER_FLAG_ALLCOMMANDS flag.

in ACLSetUser, when adding subcommand, we don't need to call
ACLGetCommandID ahead since subcommand may be empty.
2020-05-22 12:37:49 +02:00
ShooterIT a902e6b251 Redis Benchmark: generate random test data
The function of generating random data is designed by antirez. See #7196.
2020-05-22 12:37:49 +02:00
hwware 9564ed7c39 Redis-Benchmark: avoid potentical memmory leaking 2020-05-22 12:37:49 +02:00
WuYunlong 2e41827435 Handle keys with hash tag when computing hash slot using tcl cluster client. 2020-05-22 12:37:49 +02:00
WuYunlong eb2c8b2c61 Add a test to prove current tcl cluster client can not handle keys with hash tag. 2020-05-22 12:37:49 +02:00
ShooterIT 928e6976bb Use dictSize to get the size of dict in dict.c 2020-05-22 12:37:49 +02:00
Madelyn Olson cdcf5af5aa Converge hash validation for adding and removing 2020-05-22 12:37:49 +02:00
Benjamin Sergeant e8b09d2203 do not handle --cluster-yes for cluster fix mode 2020-05-22 12:37:49 +02:00
Benjamin Sergeant 57b4fb0d84 fix typo ... 2020-05-22 12:37:49 +02:00
Benjamin Sergeant 29f25e4111 Redis-cli 6.0.1 --cluster-yes doesn't work (fix #7246)
This make it so that all prompts for all redis-cli --cluster commands are automatically answered with a yes.
2020-05-22 12:37:49 +02:00
Oran Agra 00d8b92b89 fix valgrind test failure in replication test
in b4416280c i added more keys to that test to make it run longer
but in valgrind this now means the test times out, give valgrind more
time.
2020-05-22 12:37:49 +02:00
Oran Agra 5e17e6276c add regression test for the race in #7205
with the original version of 6.0.0, this test detects an excessive full
sync.
with the fix in 1a7cd2c0e, this test detects memory corruption,
especially when using libc allocator with or without valgrind.
2020-05-22 12:37:49 +02:00
antirez 96e7c011e2 Improve the PSYNC2 test reliability. 2020-05-22 12:37:49 +02:00
254 changed files with 13387 additions and 1953 deletions

No files matched your search

+24 -25
View File
@@ -3,57 +3,56 @@ name: CI
on: [push, pull_request]
jobs:
test-ubuntu-latest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions/checkout@v2
- name: make
run: |
sudo apt-get update
sudo apt-get -y install uuid-dev libcurl4-openssl-dev
make -j2
- name: test
make BUILD_TLS=yes -j2
- name: gen-cert
run: ./utils/gen-test-certs.sh
- name: test-tls
run: |
sudo apt-get -y install tcl8.5
./runtest --clients 2 --verbose
sudo apt-get -y install tcl8.5 tcl-tls
./runtest --clients 2 --verbose --tls
- name: cluster-test
run: |
./runtest-cluster
./runtest-cluster --tls
- name: sentinel test
run: |
./runtest-sentinel
- name: module tests
run: |
./runtest-moduleapi
test-ubuntu-arm:
runs-on: [self-hosted, linux, arm]
steps:
- uses: actions/checkout@v1
- name: make
run: |
sudo apt-get -y install uuid-dev libcurl4-openssl-dev
make -j4
- name: test
run: |
sudo apt-get -y install tcl8.5
./runtest --clients 2 --verbose
- name: module tests
run: |
./runtest-moduleapi
build-ubuntu-old:
runs-on: ubuntu-16.04
steps:
- uses: actions/checkout@v1
- name: make -j2
run: |
sudo apt-get update
sudo apt-get -y install uuid-dev libcurl4-openssl-dev
make
make -j2
build-macos-latest:
runs-on: macos-latest
steps:
- uses: actions/checkout@v1
- uses: actions/checkout@v2
- name: make
run: make -j2
build-libc-malloc:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: make
run: |
sudo apt-get update
sudo apt-get -y install uuid-dev libcurl4-openssl-dev
make MALLOC=libc -j2
+133 -8
View File
@@ -1,15 +1,21 @@
name: Daily
on:
pull_request:
branches:
# any PR to a release branch.
- '[0-9].[0-9]'
schedule:
- cron: '0 7 * * *'
- cron: '0 0 * * *'
jobs:
test-jemalloc:
test-ubuntu-jemalloc:
runs-on: ubuntu-latest
timeout-minutes: 1200
if: github.repository == 'redis/redis'
timeout-minutes: 14400
steps:
- uses: actions/checkout@v1
- uses: actions/checkout@v2
- name: make
run: |
sudo apt-get -y install uuid-dev libcurl4-openssl-dev
@@ -20,12 +26,17 @@ jobs:
./runtest --accurate --verbose
- name: module api test
run: ./runtest-moduleapi --verbose
- name: sentinel tests
run: ./runtest-sentinel
- name: cluster tests
run: ./runtest-cluster
test-libc-malloc:
test-ubuntu-libc-malloc:
runs-on: ubuntu-latest
timeout-minutes: 1200
if: github.repository == 'redis/redis'
timeout-minutes: 14400
steps:
- uses: actions/checkout@v1
- uses: actions/checkout@v2
- name: make
run: |
sudo apt-get -y install uuid-dev libcurl4-openssl-dev
@@ -36,12 +47,53 @@ jobs:
./runtest --accurate --verbose
- name: module api test
run: ./runtest-moduleapi --verbose
- name: sentinel tests
run: ./runtest-sentinel
- name: cluster tests
run: ./runtest-cluster
test:
runs-on: ubuntu-latest
if: github.repository == 'redis/redis'
timeout-minutes: 14400
steps:
- uses: actions/checkout@v2
- name: make
run: |
sudo apt-get -y install uuid-dev libcurl4-openssl-dev
make BUILD_TLS=yes -j2
- name: "test (tls)"
run: |
sudo apt-get install tcl8.5 tcl-tls
./utils/gen-test-certs.sh
./runtest --accurate --verbose --tls
- name: test
run: ./runtest --accurate --verbose
- name: module api test (tls)
run: ./runtest-moduleapi --verbose --tls
test-ubuntu-arm:
runs-on: [self-hosted, linux, arm]
steps:
- uses: actions/checkout@v2
- name: make
run: |
sudo apt-get -y install uuid-dev libcurl4-openssl-dev
make -j4
- name: test
run: |
sudo apt-get -y install tcl8.5
./runtest --clients 2 --verbose
- name: module tests
run: |
./runtest-moduleapi
test-valgrind:
runs-on: ubuntu-latest
if: github.repository == 'redis/redis'
timeout-minutes: 14400
steps:
- uses: actions/checkout@v1
- uses: actions/checkout@v2
- name: make
run: |
sudo apt-get -y install uuid-dev libcurl4-openssl-dev
@@ -52,3 +104,76 @@ jobs:
./runtest --valgrind --verbose --clients 1
- name: module api test
run: ./runtest-moduleapi --valgrind --verbose --clients 1
test-centos7-jemalloc:
runs-on: ubuntu-latest
if: github.repository == 'redis/redis'
container: centos:7
timeout-minutes: 14400
steps:
- uses: actions/checkout@v2
- name: make
run: |
yum -y install centos-release-scl
yum -y install devtoolset-7
scl enable devtoolset-7 "make"
- name: test
run: |
yum -y install tcl
./runtest --accurate --verbose
- name: module api test
run: ./runtest-moduleapi --verbose
- name: sentinel tests
run: ./runtest-sentinel
- name: cluster tests
run: ./runtest-cluster
test-centos7-tls:
runs-on: ubuntu-latest
if: github.repository == 'redis/redis'
container: centos:7
timeout-minutes: 14400
steps:
- uses: actions/checkout@v2
- name: make
run: |
yum -y install centos-release-scl epel-release
yum -y install devtoolset-7 openssl-devel openssl
scl enable devtoolset-7 "make BUILD_TLS=yes"
- name: test
run: |
yum -y install tcl tcltls
./utils/gen-test-certs.sh
./runtest --accurate --verbose --tls
./runtest --accurate --verbose
- name: module api test
run: |
./runtest-moduleapi --verbose --tls
./runtest-moduleapi --verbose
- name: sentinel tests
run: |
./runtest-sentinel --tls
./runtest-sentinel
- name: cluster tests
run: |
./runtest-cluster --tls
./runtest-cluster
test-macos-latest:
runs-on: macos-latest
if: github.repository == 'redis/redis'
timeout-minutes: 14400
steps:
- uses: actions/checkout@v2
- name: make
run: make
- name: test
run: |
./runtest --accurate --verbose
- name: module api test
run: ./runtest-moduleapi --verbose
- name: sentinel tests
run: ./runtest-sentinel
- name: cluster tests
run: ./runtest-cluster
+22
View File
@@ -0,0 +1,22 @@
name: Endurance
on:
schedule:
- cron: '30 * * * *'
jobs:
test-endurance:
runs-on: octocore
timeout-minutes: 60
steps:
- uses: actions/checkout@v1
- name: make
run: |
sudo apt-get -y install uuid-dev libcurl4-openssl-dev
make -j8
- name: test-multithread (5X)
run: |
sudo apt-get install -y tcl tcl-tls
./runtest --loopn 5 --config server-threads 3 --clients 5 --endurance
+20 -3
View File
@@ -4,8 +4,24 @@ core
*.xo
*.so
*.d
!**/bash_completion.d
!**/logrotate.d
!**/keydb.service.d
!**/keydb-sentinel.service.d
*.log
dump.rdb
src/keydb-server
**/bin/keydb-server
**/app/keydb-server
*.deb
*.rpm
keydb-pro-server
src/keydb-cli
**/bin/keydb-cli
**/app/keydb-cli
src/keydb-sentinel
**/bin/keydb-sentinel
**/app/keydb-sentinel
redis-benchmark
keydb-benchmark
redis-check-aof
@@ -15,11 +31,8 @@ keydb-check-rdb
redis-check-dump
keydb-check-dump
redis-cli
keydb-cli
redis-sentinel
keydb-sentinel
redis-server
keydb-server
doc-tools
release
misc/*
@@ -35,9 +48,13 @@ src/nodes.conf
deps/lua/src/lua
deps/lua/src/luac
deps/lua/src/liblua.a
tests/tls/*
.make-*
.prerequisites
*.dSYM
Makefile.dep
.vscode/*
.idea/*
.ccls
.ccls-cache/*
compile_commands.json
+1064
View File
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -1 +1 @@
Please check https://github.com/antirez/redis/issues
Please check https://github.com/redis/redis/issues
+2 -2
View File
@@ -1,5 +1,5 @@
Copyright (c) 2006-2015, Salvatore Sanfilippo
Copyright (C) 2019, John Sully
Copyright (c) 2006-2020, Salvatore Sanfilippo
Copyright (C) 2019-2020, John Sully
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
-10
View File
@@ -1,10 +0,0 @@
FROM ubuntu:18.04
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -qqy \
build-essential nasm autotools-dev autoconf libcurl4-openssl-dev libjemalloc-dev tcl tcl-dev uuid-dev libcurl4-openssl-dev \
&& apt-get clean
CMD make
+4 -3
View File
@@ -1,6 +1,5 @@
![Current Release](https://img.shields.io/github/release/JohnSully/KeyDB.svg)
![CI](https://github.com/JohnSully/KeyDB/workflows/CI/badge.svg?branch=unstable)
[![Join the chat at https://gitter.im/KeyDB/community](https://badges.gitter.im/KeyDB/community.svg)](https://gitter.im/KeyDB/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
[![StackShare](http://img.shields.io/badge/tech-stack-0690fa.svg?style=flat)](https://stackshare.io/eq-alpha-technology-inc/eq-alpha-technology-inc)
##### New! Want to extend KeyDB with Javascript? Try [ModJS](https://github.com/JohnSully/ModJS)
@@ -96,9 +95,11 @@ libssl-dev on Debian/Ubuntu) and run:
% make BUILD_TLS=yes
You can enable flash support with:
To build with systemd support, you'll need systemd development libraries (such
as libsystemd-dev on Debian/Ubuntu or systemd-devel on CentOS) and run:
% make USE_SYSTEMD=yes
% make MALLOC=memkind
***Note that the following dependencies may be needed:
% sudo apt-get install autoconf autotools-dev libnuma-dev libtool
-2
View File
@@ -56,8 +56,6 @@ Note that unlike Redis, KeyDB fully supports multithreading of TLS connections.
To-Do List
----------
- [ ] Add session caching support. Check if/how it's handled by clients to
assess how useful/important it is.
- [ ] redis-benchmark support. The current implementation is a mix of using
hiredis for parsing and basic networking (establishing connections), but
directly manipulating sockets for most actions. This will need to be cleaned
+1 -1
View File
@@ -47,7 +47,7 @@ Hiredis
Hiredis uses the SDS string library, that must be the same version used inside Redis itself. Hiredis is also very critical for Sentinel. Historically Redis often used forked versions of hiredis in a way or the other. In order to upgrade it is advised to take a lot of care:
1. Check with diff if hiredis API changed and what impact it could have in Redis.
2. Make sure thet the SDS library inside Hiredis and inside Redis are compatible.
2. Make sure that the SDS library inside Hiredis and inside Redis are compatible.
3. After the upgrade, run the Redis Sentinel test.
4. Check manually that redis-cli and redis-benchmark behave as expecteed, since we have no tests for CLI utilities currently.
-1
Submodule deps/depot_tools deleted from aaf5669995.
@@ -216,7 +216,7 @@ ixalloc(tsdn_t *tsdn, void *ptr, size_t oldsize, size_t size, size_t extra,
}
JEMALLOC_ALWAYS_INLINE int
iget_defrag_hint(tsdn_t *tsdn, void* ptr, int *bin_util, int *run_util) {
iget_defrag_hint(tsdn_t *tsdn, void* ptr) {
int defrag = 0;
rtree_ctx_t rtree_ctx_fallback;
rtree_ctx_t *rtree_ctx = tsdn_rtree_ctx(tsdn, &rtree_ctx_fallback);
@@ -232,11 +232,22 @@ iget_defrag_hint(tsdn_t *tsdn, void* ptr, int *bin_util, int *run_util) {
malloc_mutex_lock(tsdn, &bin->lock);
/* don't bother moving allocations from the slab currently used for new allocations */
if (slab != bin->slabcur) {
const bin_info_t *bin_info = &bin_infos[binind];
size_t availregs = bin_info->nregs * bin->stats.curslabs;
*bin_util = ((long long)bin->stats.curregs<<16) / availregs;
*run_util = ((long long)(bin_info->nregs - extent_nfree_get(slab))<<16) / bin_info->nregs;
defrag = 1;
int free_in_slab = extent_nfree_get(slab);
if (free_in_slab) {
const bin_info_t *bin_info = &bin_infos[binind];
int curslabs = bin->stats.curslabs;
size_t curregs = bin->stats.curregs;
if (bin->slabcur) {
/* remove slabcur from the overall utilization */
curregs -= bin_info->nregs - extent_nfree_get(bin->slabcur);
curslabs -= 1;
}
/* Compare the utilization ratio of the slab in question to the total average,
* to avoid precision lost and division, we do that by extrapolating the usage
* of the slab as if all slabs have the same usage. If this slab is less used
* than the average, we'll prefer to evict the data to hopefully more used ones */
defrag = (bin_info->nregs - free_in_slab) * curslabs <= curregs;
}
}
malloc_mutex_unlock(tsdn, &bin->lock);
}
+3 -5
View File
@@ -3326,12 +3326,10 @@ jemalloc_postfork_child(void) {
/******************************************************************************/
/* Helps the application decide if a pointer is worth re-allocating in order to reduce fragmentation.
* returns 0 if the allocation is in the currently active run,
* or when it is not causing any frag issue (large or huge bin)
* returns the bin utilization and run utilization both in fixed point 16:16.
* returns 1 if the allocation should be moved, and 0 if the allocation be kept.
* If the application decides to re-allocate it should use MALLOCX_TCACHE_NONE when doing so. */
JEMALLOC_EXPORT int JEMALLOC_NOTHROW
get_defrag_hint(void* ptr, int *bin_util, int *run_util) {
get_defrag_hint(void* ptr) {
assert(ptr != NULL);
return iget_defrag_hint(TSDN_NULL, ptr, bin_util, run_util);
return iget_defrag_hint(TSDN_NULL, ptr);
}
+1 -1
View File
@@ -274,7 +274,7 @@ int luaD_precall (lua_State *L, StkId func, int nresults) {
CallInfo *ci;
StkId st, base;
Proto *p = cl->p;
luaD_checkstack(L, p->maxstacksize);
luaD_checkstack(L, p->maxstacksize + p->numparams);
func = restorestack(L, funcr);
if (!p->is_vararg) { /* no varargs? */
base = func + 1;
+80 -7
View File
@@ -159,9 +159,12 @@ tcp-keepalive 300
# By default, clients (including replica servers) on a TLS port are required
# to authenticate using valid client side certificates.
#
# It is possible to disable authentication using this directive.
# If "no" is specified, client certificates are not required and not accepted.
# If "optional" is specified, client certificates are accepted and must be
# valid if provided, but are not required.
#
# tls-auth-clients no
# tls-auth-clients optional
# By default, a Redis replica does not attempt to establish a TLS connection
# with its master.
@@ -176,9 +179,10 @@ tcp-keepalive 300
# tls-cluster yes
# Explicitly specify TLS versions to support. Allowed values are case insensitive
# and include "TLSv1", "TLSv1.1", "TLSv1.2", "TLSv1.3" (OpenSSL >= 1.1.1)
# and include "TLSv1", "TLSv1.1", "TLSv1.2", "TLSv1.3" (OpenSSL >= 1.1.1) or
# any combination. To enable only TLSv1.2 and TLSv1.3, use:
#
# tls-protocols TLSv1.2
# tls-protocols "TLSv1.2 TLSv1.3"
# Configure allowed ciphers. See the ciphers(1ssl) manpage for more information
# about the syntax of this string.
@@ -198,6 +202,22 @@ tcp-keepalive 300
#
# tls-prefer-server-ciphers yes
# By default, TLS session caching is enabled to allow faster and less expensive
# reconnections by clients that support it. Use the following directive to disable
# caching.
#
# tls-session-caching no
# Change the default number of TLS sessions cached. A zero value sets the cache
# to unlimited size. The default size is 20480.
#
# tls-session-cache-size 5000
# Change the default timeout of cached TLS sessions. The default timeout is 300
# seconds.
#
# tls-session-cache-timeout 60
################################# GENERAL #####################################
# By default KeyDB does not run as a daemon. Use 'yes' if you need it.
@@ -400,6 +420,20 @@ dir ./
#
replica-serve-stale-data yes
# Active Replicas will allow read only data access while loading remote RDBs
# provided they are permitted to serve stale data. As an option you may also
# permit them to accept write commands. This is an EXPERIMENTAL feature and
# may result in commands not being fully synchronized
#
# allow-write-during-load no
# You can modify the number of masters necessary to form a replica quorum when
# multi-master is enabled and replica-serve-stale-data is "no". By default
# this is set to -1 which implies the number of known masters (e.g. those
# you added with replicaof)
#
# replica-quorum -1
# You can configure a replica instance to accept writes or not. Writing against
# a replica instance may be useful to store some ephemeral data (because data
# written on a replica will be easily deleted after resync with the master) but
@@ -744,9 +778,8 @@ replica-priority 100
#
# The ACL Log tracks failed commands and authentication events associated
# with ACLs. The ACL Log is useful to troubleshoot failed commands blocked
# by ACLs. The ACL Log is stored in and consumes memory. There is no limit
# to its length.You can reclaim memory with ACL LOG RESET or set a maximum
# length below.
# by ACLs. The ACL Log is stored in memory. You can reclaim memory with
# ACL LOG RESET. Define the maximum entry length of the ACL Log below.
acllog-max-len 128
# Using an external ACL file
@@ -805,6 +838,11 @@ acllog-max-len 128
# Once the limit is reached KeyDB will close all the new connections sending
# an error 'max number of clients reached'.
#
# IMPORTANT: When Redis Cluster is used, the max number of connections is also
# shared with the cluster bus: every node in the cluster will use two
# connections, one incoming and another outgoing. It is important to size the
# limit accordingly in case of very large clusters.
#
# maxclients 10000
############################## MEMORY MANAGEMENT ################################
@@ -1015,6 +1053,32 @@ lazyfree-lazy-user-del no
# --threads option to match the number of Redis theads, otherwise you'll not
# be able to notice the improvements.
############################ KERNEL OOM CONTROL ##############################
# On Linux, it is possible to hint the kernel OOM killer on what processes
# should be killed first when out of memory.
#
# Enabling this feature makes Redis actively control the oom_score_adj value
# for all its processes, depending on their role. The default scores will
# attempt to have background child processes killed before all others, and
# replicas killed before masters.
oom-score-adj no
# When oom-score-adj is used, this directive controls the specific values used
# for master, replica and background child processes. Values range -1000 to
# 1000 (higher means more likely to be killed).
#
# Unprivileged processes (not root, and without CAP_SYS_RESOURCE capabilities)
# can freely increase their value, but not decrease it below its initial
# settings.
#
# Values are used relative to the initial value of oom_score_adj when the server
# starts. Because typically the initial value is 0, they will often match the
# absolute values.
oom-score-adj-values 0 200 800
############################## APPEND ONLY MODE ###############################
# By default KeyDB asynchronously dumps the dataset on disk. This mode is
@@ -1617,7 +1681,7 @@ client-output-buffer-limit pubsub 32mb 8mb 60
# In the KeyDB protocol, bulk requests, that are, elements representing single
# strings, are normally limited ot 512 mb. However you can change this limit
# here.
# here, but must be 1mb or greater
#
# proto-max-bulk-len 512mb
@@ -1810,6 +1874,15 @@ jemalloc-bg-thread yes
# Set bgsave child process to cpu affinity 1,10,11
# bgsave_cpulist 1,10-11
# The minimum number of clients on a thread before KeyDB assigns new connections to a different thread
# Tuning this parameter is a tradeoff between locking overhead and distributing the workload over multiple cores
# min-clients-per-thread 50
# Avoid forwarding RREPLAY messages to other masters?
# WARNING: This setting is dangerous! You must be certain all masters are connected to each
# other in a true mesh topology or data loss will occur!
# This command can be used to reduce multimaster bus traffic
# multi-master-no-forward no
# Path to directory for file backed scratchpad. The file backed scratchpad
# reduces memory requirements by storing rarely accessed data on disk
+3
View File
@@ -0,0 +1,3 @@
### KeyDB Packaging
The following directories contain packaging code used to create rpm and deb packages as well as docker images.
+21
View File
@@ -0,0 +1,21 @@
### KeyDB DEB Package Source Builds
This directory contains scripts and components needed to generate debian packages on different distributions/architectures from source
The 'debian' directory contains debian source code for bionic, buster and later distributions as it uses functions only available with debhelper11+. 'debian_dh9' is used for xenial, stretch and earlier distributions using debhelper9.
You will need to install pbuilder `sudo apt install pbuilder` along with other distribution specific dependancies
Generate deb packages with the following script command run from this directory:
```
$ ./deb-buildsource.sh
```
This generates a directory structure, .dsc file, original.tar.gz, and new changelog for the distribution and architecture installed.
When complete the produced debian packages will be located in deb_files_generated directory.
Arguments for the deb-buildsource.sh script that can be passed are either 'None' or '"your custom comments"'. 'None' assumes a permanent changelog entry has been made listing the correct build for the version. If you quote and enter your own comment string, it will be appended to the changelog for that deb package. If no argument is passed a default comment is generated saying that this deb package was generated by this script.
This script has been tested on xenial/bionic/stretch/buster
File diff suppressed because it is too large. Load diff
+244
View File
@@ -0,0 +1,244 @@
# Example sentinel.conf
# *** IMPORTANT ***
#
# By default Sentinel will not be reachable from interfaces different than
# localhost, either use the 'bind' directive to bind to a list of network
# interfaces, or disable protected mode with "protected-mode no" by
# adding it to this configuration file.
#
# Before doing that MAKE SURE the instance is protected from the outside
# world via firewalling or other means.
#
# For example you may use one of the following:
#
# bind 127.0.0.1 192.168.1.1
#
# protected-mode no
# port <sentinel-port>
# The port that this sentinel instance will run on
port 26379
# By default Redis Sentinel does not run as a daemon. Use 'yes' if you need it.
# Note that Redis will write a pid file in /var/run/keydb-sentinel.pid when
# daemonized.
daemonize yes
# When running daemonized, Redis Sentinel writes a pid file in
# /var/run/keydb-sentinel.pid by default. You can specify a custom pid file
# location here.
pidfile /var/run/sentinel/keydb-sentinel.pid
# Specify the log file name. Also the empty string can be used to force
# Sentinel to log on the standard output. Note that if you use standard
# output for logging but daemonize, logs will be sent to /dev/null
logfile /var/log/keydb/keydb-sentinel.log
# sentinel announce-ip <ip>
# sentinel announce-port <port>
#
# The above two configuration directives are useful in environments where,
# because of NAT, Sentinel is reachable from outside via a non-local address.
#
# When announce-ip is provided, the Sentinel will claim the specified IP address
# in HELLO messages used to gossip its presence, instead of auto-detecting the
# local address as it usually does.
#
# Similarly when announce-port is provided and is valid and non-zero, Sentinel
# will announce the specified TCP port.
#
# The two options don't need to be used together, if only announce-ip is
# provided, the Sentinel will announce the specified IP and the server port
# as specified by the "port" option. If only announce-port is provided, the
# Sentinel will announce the auto-detected local IP and the specified port.
#
# Example:
#
# sentinel announce-ip 1.2.3.4
# dir <working-directory>
# Every long running process should have a well-defined working directory.
# For Redis Sentinel to chdir to /tmp at startup is the simplest thing
# for the process to don't interfere with administrative tasks such as
# unmounting filesystems.
dir /var/lib/keydb
# sentinel monitor <master-name> <ip> <keydb-port> <quorum>
#
# Tells Sentinel to monitor this master, and to consider it in O_DOWN
# (Objectively Down) state only if at least <quorum> sentinels agree.
#
# Note that whatever is the ODOWN quorum, a Sentinel will require to
# be elected by the majority of the known Sentinels in order to
# start a failover, so no failover can be performed in minority.
#
# Replicas are auto-discovered, so you don't need to specify replicas in
# any way. Sentinel itself will rewrite this configuration file adding
# the replicas using additional configuration options.
# Also note that the configuration file is rewritten when a
# replica is promoted to master.
#
# Note: master name should not include special characters or spaces.
# The valid charset is A-z 0-9 and the three characters ".-_".
sentinel monitor mymaster 127.0.0.1 6379 2
# sentinel auth-pass <master-name> <password>
#
# Set the password to use to authenticate with the master and replicas.
# Useful if there is a password set in the Redis instances to monitor.
#
# Note that the master password is also used for replicas, so it is not
# possible to set a different password in masters and replicas instances
# if you want to be able to monitor these instances with Sentinel.
#
# However you can have Redis instances without the authentication enabled
# mixed with Redis instances requiring the authentication (as long as the
# password set is the same for all the instances requiring the password) as
# the AUTH command will have no effect in Redis instances with authentication
# switched off.
#
# Example:
#
# sentinel auth-pass mymaster MySUPER--secret-0123passw0rd
# sentinel down-after-milliseconds <master-name> <milliseconds>
#
# Number of milliseconds the master (or any attached replica or sentinel) should
# be unreachable (as in, not acceptable reply to PING, continuously, for the
# specified period) in order to consider it in S_DOWN state (Subjectively
# Down).
#
# Default is 30 seconds.
sentinel down-after-milliseconds mymaster 30000
# sentinel parallel-syncs <master-name> <numreplicas>
#
# How many replicas we can reconfigure to point to the new replica simultaneously
# during the failover. Use a low number if you use the replicas to serve query
# to avoid that all the replicas will be unreachable at about the same
# time while performing the synchronization with the master.
sentinel parallel-syncs mymaster 1
# sentinel failover-timeout <master-name> <milliseconds>
#
# Specifies the failover timeout in milliseconds. It is used in many ways:
#
# - The time needed to re-start a failover after a previous failover was
# already tried against the same master by a given Sentinel, is two
# times the failover timeout.
#
# - The time needed for a replica replicating to a wrong master according
# to a Sentinel current configuration, to be forced to replicate
# with the right master, is exactly the failover timeout (counting since
# the moment a Sentinel detected the misconfiguration).
#
# - The time needed to cancel a failover that is already in progress but
# did not produced any configuration change (SLAVEOF NO ONE yet not
# acknowledged by the promoted replica).
#
# - The maximum time a failover in progress waits for all the replicas to be
# reconfigured as replicas of the new master. However even after this time
# the replicas will be reconfigured by the Sentinels anyway, but not with
# the exact parallel-syncs progression as specified.
#
# Default is 3 minutes.
sentinel failover-timeout mymaster 180000
# SCRIPTS EXECUTION
#
# sentinel notification-script and sentinel reconfig-script are used in order
# to configure scripts that are called to notify the system administrator
# or to reconfigure clients after a failover. The scripts are executed
# with the following rules for error handling:
#
# If script exits with "1" the execution is retried later (up to a maximum
# number of times currently set to 10).
#
# If script exits with "2" (or an higher value) the script execution is
# not retried.
#
# If script terminates because it receives a signal the behavior is the same
# as exit code 1.
#
# A script has a maximum running time of 60 seconds. After this limit is
# reached the script is terminated with a SIGKILL and the execution retried.
# NOTIFICATION SCRIPT
#
# sentinel notification-script <master-name> <script-path>
#
# Call the specified notification script for any sentinel event that is
# generated in the WARNING level (for instance -sdown, -odown, and so forth).
# This script should notify the system administrator via email, SMS, or any
# other messaging system, that there is something wrong with the monitored
# Redis systems.
#
# The script is called with just two arguments: the first is the event type
# and the second the event description.
#
# The script must exist and be executable in order for sentinel to start if
# this option is provided.
#
# Example:
#
# sentinel notification-script mymaster /var/redis/notify.sh
# CLIENTS RECONFIGURATION SCRIPT
#
# sentinel client-reconfig-script <master-name> <script-path>
#
# When the master changed because of a failover a script can be called in
# order to perform application-specific tasks to notify the clients that the
# configuration has changed and the master is at a different address.
#
# The following arguments are passed to the script:
#
# <master-name> <role> <state> <from-ip> <from-port> <to-ip> <to-port>
#
# <state> is currently always "failover"
# <role> is either "leader" or "observer"
#
# The arguments from-ip, from-port, to-ip, to-port are used to communicate
# the old address of the master and the new address of the elected replica
# (now a master).
#
# This script should be resistant to multiple invocations.
#
# Example:
#
# sentinel client-reconfig-script mymaster /var/redis/reconfig.sh
# SECURITY
#
# By default SENTINEL SET will not be able to change the notification-script
# and client-reconfig-script at runtime. This avoids a trivial security issue
# where clients can set the script to anything and trigger a failover in order
# to get the program executed.
sentinel deny-scripts-reconfig yes
# REDIS COMMANDS RENAMING
#
# Sometimes the Redis server has certain commands, that are needed for Sentinel
# to work correctly, renamed to unguessable strings. This is often the case
# of CONFIG and SLAVEOF in the context of providers that provide Redis as
# a service, and don't want the customers to reconfigure the instances outside
# of the administration console.
#
# In such case it is possible to tell Sentinel to use different command names
# instead of the normal ones. For example if the master "mymaster", and the
# associated replicas, have "CONFIG" all renamed to "GUESSME", I could use:
#
# SENTINEL rename-command mymaster CONFIG GUESSME
#
# After such configuration is set, every time Sentinel would use CONFIG it will
# use GUESSME instead. Note that there is no actual need to respect the command
# case, so writing "config guessme" is the same in the example above.
#
# SENTINEL SET can also be used in order to perform this configuration at runtime.
#
# In order to set a command back to its original name (undo the renaming), it
# is possible to just rename a command to itsef:
#
# SENTINEL rename-command mymaster CONFIG CONFIG
+63
View File
@@ -0,0 +1,63 @@
#! /bin/bash
# define parameters used in this deb package build
# enter comments as an argument string quoted. If you do not want comments to be added (ie. you have updated in git repo) type None
if [ $# -eq 0 ]; then
changelog_comments="This is a new source build generated from the deb_build_source.sh script"
else
changelog_comments=$1
fi
echo $changelog_comments
build=1 #change if updated build number required. Default is 1 as convention for keydb is to update minor version
version=$(grep KEYDB_REAL_VERSION ../../src/version.h | awk '{ printf $3 }' | tr -d \")
majorv="${version:0:1}"
distributor=$(lsb_release --id --short)
if [ "$distributor" == "Debian" ]; then
distname=+deb$(lsb_release --release --short | cut -d. -f1)u1
elif [ "$distributor" == "Ubuntu" ]; then
distname=~$(lsb_release --codename --short)1
fi
codename=$(lsb_release --codename --short)
date=$(date +%a," "%d" "%b" "%Y" "%T)
pkg_name=keydb-$majorv:$version$distname
# create build tree
cd ../../../
tar -czvf keydb_$version.orig.tar.gz --force-local KeyDB
cd KeyDB/pkg/deb/
mkdir -p $pkg_name/tmp
if [[ "$codename" == "xenial" ]] || [[ "$codename" == "stretch" ]]; then
cp -r debian_dh9 $pkg_name/tmp/debian
else
cp -r debian $pkg_name/tmp
fi
cp master_changelog $pkg_name/tmp/debian/changelog
mv ../../../keydb_$version.orig.tar.gz ./$pkg_name
cd $pkg_name/tmp
changelog_str="keydb ($majorv:$version-$build$distname) $codename; urgency=medium\n\n * $version $changelog_comments \n\n -- Ben Schermel <ben@eqalpha.com> $date +0000\n\n"
if [ $# -eq 0 ]; then
sed -i "1s/^/$changelog_str\n/" debian/changelog
elif [ $# -eq 1 ] && [ "$1" != "None" ]; then
sed -i "1s/^/$changelog_str\n/" debian/changelog
fi
sed -i "s/distribution_placeholder/$distname/g" debian/changelog
sed -i "s/codename_placeholder/$codename/g" debian/changelog
# generate required files and structure for pbuilder including .dsc file
debuild -S -sa
cd ../
# create pbuilder chrooted environment and build the deb package
if [ "$codename" == "xenial" ]; then
sudo pbuilder create --distribution $codename --othermirror "deb http://archive.ubuntu.com/ubuntu $codename universe multiverse"
else
sudo pbuilder create --distribution $codename
fi
sudo pbuilder --update
sudo pbuilder --build *.dsc
# move new packages to deb_files_generated and clean up
cp /var/cache/pbuilder/result/*$version*.deb ../deb_files_generated
sudo pbuilder clean
cd ../
rm -rf $pkg_name
Whitespace-only changes.
@@ -0,0 +1,30 @@
# -*- sh -*-
#
# Bash completion function for the 'keydb-cli' command.
#
# Steve
# --
# http://www.steve.org.uk
#
_keydb-cli()
{
COMPREPLY=()
cur=${COMP_WORDS[COMP_CWORD]}
prev=${COMP_WORDS[COMP_CWORD-1]}
#
# All known commands accepted. Sorted.
#
opts='bgrewriteaof bgsave dbsize debug decr decrby del echo exists expire expireat flushall flushdb get getset incr incrby info keys lastsave lindex llen lpop lpush lrange lrem lset ltrim mget move mset msetnx ping randomkey rename renamenx rewriteaof rpop rpoplpush rpush sadd save scard sdiff sdiffstore select set setnx shutdown sinter sinterstore sismember slaveof smembers smove sort spop srandmember srem sunion sunionstore ttl type zadd zcard zincrby zrange zrangebyscore zrem zremrangebyscore zrevrange zscore'
#
# Only complete on the first term.
#
if [ $COMP_CWORD -eq 1 ]; then
COMPREPLY=( $(compgen -W "${opts}" -- ${cur}) )
return 0
fi
}
complete -F _keydb-cli keydb-cli
+129
View File
@@ -0,0 +1,129 @@
#!/bin/sh
SYSTEMD_VER=$(systemctl --version | head -1 | cut -d' ' -f2)
if [ ${SYSTEMD_VER} -lt 237 ]
then
SYSTEMD_EXTRA=""
else
SYSTEMD_EXTRA=$(cat <<EOF
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
#MemoryDenyWriteExecute=true
ProtectKernelModules=true
ProtectKernelTunables=true
ProtectControlGroups=true
RestrictRealtime=true
RestrictNamespaces=true
EOF
)
fi
for BINARY in keydb-server keydb-sentinel
do
for MODE in default templated
do
case "${BINARY}" in
keydb-server)
NAME="keydb"
;;
keydb-sentinel)
NAME="sentinel"
;;
esac
case "${MODE}" in
default)
EXTRA="Alias=${NAME}.service"
TARGET="debian/${BINARY}.service"
NAMESPACED="${NAME}"
DESCRIPTION="Advanced key-value store"
;;
templated)
EXTRA=""
TARGET="debian/${BINARY}@.service"
NAMESPACED="${NAME}-%i"
DESCRIPTION="Advanced key-value store (%I)"
;;
esac
: >${TARGET}
if [ "${MODE}" = "templated" ]
then
cat >> ${TARGET} <<EOF
# Templated service file for ${BINARY}(1)
#
# Each instance of ${BINARY} requires its own configuration file:
#
# $ cp /etc/keydb/${NAME}.conf /etc/keydb/${NAME}-myname.conf
# $ chown keydb:keydb /etc/keydb/${NAME}-myname.conf
#
# Ensure each instance is using their own database:
#
# $ sed -i -e 's@^dbfilename .*@dbfilename dump-myname.rdb@' /etc/keydb/${NAME}-myname.conf
#
# We then listen exlusively on UNIX sockets to avoid TCP port collisions:
#
# $ sed -i -e 's@^port .*@port 0@' /etc/keydb/${NAME}-myname.conf
# $ sed -i -e 's@^\\(# \\)\\{0,1\\}unixsocket .*@unixsocket /var/run/${NAME}-myname/${BINARY}.sock@' /etc/keydb/${NAME}-myname.conf
#
# ... and ensure we are logging, etc. in a unique location:
#
# $ sed -i -e 's@^logfile .*@logfile /var/log/keydb/${BINARY}-myname.log@' /etc/keydb/${NAME}-myname.conf
# $ sed -i -e 's@^pidfile .*@pidfile /var/run/keydb-myname/${BINARY}.pid@' /etc/keydb/${NAME}-myname.conf
#
# We can then start the service as follows, validating we are using our own
# configuration:
#
# $ systemctl start ${BINARY}@myname.service
# $ keydb-cli -s /var/run/${NAME}-myname/${BINARY}.sock info | grep config_file
EOF
fi
cat >> ${TARGET} <<EOF
[Unit]
Description=${DESCRIPTION}
After=network.target
Documentation=https://docs.keydb.dev, man:${BINARY}(1)
[Service]
Type=forking
ExecStart=/usr/bin/${BINARY} /etc/keydb/${NAMESPACED}.conf
ExecStop=/bin/kill -s TERM \$MAINPID
PIDFile=/var/run/${NAMESPACED}/${BINARY}.pid
TimeoutStopSec=0
Restart=always
User=keydb
Group=keydb
RuntimeDirectory=${NAMESPACED}
RuntimeDirectoryMode=2755
UMask=007
PrivateTmp=yes
LimitNOFILE=65535
PrivateDevices=yes
ProtectHome=yes
ReadOnlyDirectories=/
ReadWriteDirectories=-/var/lib/keydb
ReadWriteDirectories=-/var/log/keydb
ReadWriteDirectories=-/var/run/${NAMESPACED}
NoNewPrivileges=true
CapabilityBoundingSet=CAP_SETGID CAP_SETUID CAP_SYS_RESOURCE${SYSTEMD_EXTRA}
# ${BINARY} can write to its own config file when in cluster mode so we
# permit writing there by default. If you are not using this feature, it is
# recommended that you replace the following lines with "ProtectSystem=full".
ProtectSystem=true
ReadWriteDirectories=-/etc/keydb
[Install]
WantedBy=multi-user.target
EOF
if [ "${EXTRA}" != "" ]
then
echo "${EXTRA}" >> "${TARGET}"
fi
done
done
+32
View File
@@ -0,0 +1,32 @@
keydb (5:5.3.3-1~bionic1) bionic; urgency=medium
* 5.3.3 Updating deb source package and naming convention.
-- Ben Schermel <ben@eqalpha.com> Fri, 25 Oct 2019 8:00:37 +0000
keydb (5:5.1.12-1chl1~bionic1) bionic; urgency=medium
* 5.1.1 update. This update fixes several rare deadlock scenarios. Deadlock detection is also added.
-- Ben Schermel <ben@eqalpha.com> Fri, 25 Oct 2019 8:00:37 +0000
keydb (5:5.1.11-1chl1~bionic1) bionic; urgency=medium
* 5.1 release. This release includes subkey expires (EXPIREMEMBER/EXPIREMEMBERAT), with updates to PTTL/TTL accordingly. New OBJECT LASTMODIFIED, BITIOP LSHIFT & BITOP RSHIFT commands. See https://docs.keydb.dev/blog/2019/10/20/blog-post/ for detailed review of release.
-- Ben Schermel <ben@eqalpha.com> Mon, 21 Oct 2019 8:00:37 +0000
keydb (5:5.0.1-1chl1~bionic1) bionic; urgency=medium
* Arm build now included for bionic package
-- Ben Schermel <ben@eqalpha.com> Wed, 21 Aug 2019 22:58:37 +0000
keydb (5:5.0.0-1chl1~bionic1) bionic; urgency=medium
* Initial release of KeyDB PPA. This PPA was originally derived from https://launchpad.net/~chris-lea/+archive/ubuntu/redis-server
-- Ben Schermel <ben@eqalpha.com> Wed, 21 Aug 2019 2:58:37 +0000
+1
View File
@@ -0,0 +1 @@
9
+77
View File
@@ -0,0 +1,77 @@
Source: keydb
Section: database
Priority: optional
Maintainer: Ben Schermel <ben@eqalpha.com>
Build-Depends:
debhelper (>= 9~),
dpkg-dev (>= 1.17.5),
systemd,
procps <!nocheck>,
build-essential <!nocheck>,
tcl <!nocheck>,
tcl-dev <!nocheck>,
uuid-dev <!nocheck>,
libcurl4-openssl-dev <!nocheck>,
nasm <!nocheck>,
autotools-dev <!nocheck>,
autoconf <!nocheck>,
libjemalloc-dev <!nocheck>,
libssl-dev <!nocheck>
Standards-Version: 4.2.1
Homepage: https://docs.keydb.dev/
Vcs-Git: https://github.com/JohnSully/KeyDB.git
Vcs-Browser: https://github.com/JohnSully/KeyDB
Package: keydb
Architecture: all
Depends:
keydb-server (<< ${binary:Version}.1~),
keydb-server (>= ${binary:Version}),
${misc:Depends},
Description: Persistent key-value database with network interface (metapackage)
keydb is a key-value database in a similar vein to memcache but the dataset
is non-volatile. keydb additionally provides native support for atomically
manipulating and querying data structures such as lists and sets.
.
The dataset is stored entirely in memory and periodically flushed to disk.
.
This package depends on the keydb-server package.
Package: keydb-sentinel
Architecture: any
Depends:
lsb-base (>= 3.2-14),
keydb-tools (= ${binary:Version}),
${misc:Depends},
Description: Persistent key-value database with network interface (monitoring)
keydb is a key-value database in a similar vein to memcache but the dataset
is non-volatile. keydb additionally provides native support for atomically
manipulating and querying data structures such as lists and sets.
.
This package contains the keydb Sentinel monitoring software.
Package: keydb-server
Architecture: any
Depends:
lsb-base (>= 3.2-14),
keydb-tools (= ${binary:Version}),
${misc:Depends},
Description: Persistent key-value database with network interface
keydb is a key-value database in a similar vein to memcache but the dataset
is non-volatile. keydb additionally provides native support for atomically
manipulating and querying data structures such as lists and sets.
.
The dataset is stored entirely in memory and periodically flushed to disk.
Package: keydb-tools
Architecture: any
Depends:
adduser,
${misc:Depends},
${shlibs:Depends},
Description: Persistent key-value database with network interface (client)
keydb is a key-value database in a similar vein to memcache but the dataset
is non-volatile. keydb additionally provides native support for atomically
manipulating and querying data structures such as lists and sets.
.
This package contains the command line client and other tools.
+61
View File
@@ -0,0 +1,61 @@
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Contact: John Sully <john@eqalpha.com>
Upstream-Name: keydb-pro
Source: https://github.com/JohnSully/KeyDB-Pro
Files: *
Copyright: © 2006-2014 Salvatore Sanfilippo <antirez@gmail.com>
Copyright © 2019, John Sully
License: Proprietary
Files: debian/*
Copyright: © 2009 Chris Lamb <lamby@debian.org>
License: BSD-3-clause
License: BSD-2-clause
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
.
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
License: BSD-3-clause
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
.
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of Redis nor the names of its contributors may be
used to endorse or promote products derived from this software
without specific prior written permission.
.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+68
View File
@@ -0,0 +1,68 @@
.TH KEYDB-BENCHMARK 1 "August 17, 2019"
.SH NAME
keydb-benchmark \- Benchmark a KeyDB instance
.SH SYNOPSIS
.B redis-benchmark
[\-h <host>] [\-p <port>] [\-c <clients>] [\-n <requests]> [\-k <boolean>]
.SH DESCRIPTION
KeyDB is a key-value database. It is similar to memcached but the dataset is
not volatile and other datatypes (such as lists and sets) are natively
supported.
.SH OPTIONS
.TP
\-h hostname
Server hostname (default 127.0.0.1)
.TP
\-p port
Server port (default 6379)
.TP
\-s socket
Server socket (overrides host and port)
.TP
\-a password
Password for KeyDB Auth
.TP
\-c clients
Number of parallel connections (default 50)
.TP
\-n requests
Total number of requests (default 100000)
.TP
\-d size
Data size of SET/GET value in bytes (default 2)
.TP
\-dbnum db
SELECT the specified db number (default 0)
.TP
\-k boolean
1=keep alive 0=reconnect (default 1)
.TP
\-r keyspacelen
Use random keys for SET/GET/INCR, random values for SADD Using this option the
benchmark will get/set keys in the form mykey_rand000000012456 instead of
constant keys, the <keyspacelen> argument determines the max number of values
for the random number. For instance if set to 10 only rand000000000000 -
rand000000000009 range will be allowed.
.TP
\-P numreq
Pipeline <numreq> requests. Default 1 (no pipeline).
.TP
\-q
Quiet. Just show query/sec values
.TP
\-\-csv
Ourput in CSV format
.TP
\-l
Loop. Run the tests forever
.TP
\-I
Idle mode. Just open N idle connections and wait.
.TP
\-D
Debug mode. more verbose.
.SH AUTHOR
\fBkeydb-benchmark\fP was written by John Sully, originating as a fork of Redis. Redis was written by Salvatore Sanfilippo.
.PP
This manual page was written by Chris Lamb <lamby@debian.org> for the Debian
project (but may be used by others). Modified by Ben Schermel <ben@eqalpha.com>
+17
View File
@@ -0,0 +1,17 @@
.TH KEYDB-CHECK-AOF 1 "August 17, 2019"
.SH NAME
keydb-check-aof \- Check integrity of a KeyDB .AOF file
.SH SYNOPSIS
.B keydb-check-aof
filename
.SH DESCRIPTION
KeyDB is a key-value database. It is similar to memcached but the dataset is
not volatile and other datatypes (such as lists and sets) are natively
supported.
.PP
This utility checks the integrity of a dumped .AOF file.
.SH AUTHOR
\fBkeydb-check-aof\fP was written by John Sully, originating as a fork of Redis. Redis was written by Salvatore Sanfilippo.
.PP
This manual page was written by Chris Lamb <lamby@debian.org> for the Debian
project (but may be used by others). Modified by Ben Schermel <ben@eqalpha.com>
+17
View File
@@ -0,0 +1,17 @@
.TH KEYDB-CHECK-RDB 1 "August 17, 2019"
.SH NAME
keydb-check-rdb \- Check integrity of KeyDB dumped database file
.SH SYNOPSIS
.B keydb-check-rdb
filename
.SH DESCRIPTION
KeyDB is a key-value database. It is similar to memcached but the dataset is
not volatile and other datatypes (such as lists and sets) are natively
supported.
.PP
This utility checks the integrity of a dumped database file.
.SH AUTHOR
\fBredis-check-rdb\fP was written by John Sully, originating as a fork of Redis. Redis was written by Salvatore Sanfilippo.
.PP
This manual page was written by Chris Lamb <lamby@debian.org> for the Debian
project (but may be used by others). Modified by Ben Schermel <ben@eqalpha.com>
+19
View File
@@ -0,0 +1,19 @@
.TH KEYDB-CLI 1 "August 17, 2019"
.SH NAME
keydb-cli \- Command-line client to keydb-server
.SH SYNOPSIS
.B keydb-cli
.RI [options]
.SH DESCRIPTION
KeyDB is a key-value database. It is similar to memcached but the dataset is
not volatile and other datatypes (such as lists and sets) are natively
supported.
.PP
\fBkeydb-cli\fP provides a simple command-line interface to a KeyDB server.
.SH OPTIONS
See \fBkeydb-doc\fP for more information on the commands KeyDB accepts.
.SH AUTHOR
\fBkeydb-cli\fP was written by John Sully, originating as a fork of Redis. Redis was written by Salvatore Sanfilippo.
.PP
This manual page was written by Chris Lamb <lamby@debian.org> for the Debian
project (but may be used by others). Modified by Ben Schermel <ben@eqalpha.com>
+23
View File
@@ -0,0 +1,23 @@
.TH KEYDB-SENTINEL 1 "August 17, 2019"
.SH NAME
keydb-sentinel \- Persistent key-value database (cluster mode)
.SH SYNOPSIS
.B keydb-sentinel
.RI configfile
.SH DESCRIPTION
KeyDB is a key-value database. It is similar to memcached but the dataset is
not volatile and other datatypes (such as lists and sets) are natively
supported.
.PP
.SH OPTIONS
.IP "configfile"
Read options from specified configuration file.
.SH NOTES
On Debian GNU/Linux systems, \fBkeydb-sentinel\fP is typically started via the
\fB/etc/init.d/keydb-sentinel\fP initscript, not manually. This defaults to using
\fB/etc/keydb/sentinel.conf\fP as a configuration file.
.SH AUTHOR
\fBkeydb-sentinel\fP was written by John Sully, originating as a fork of Redis. Redis was written by Salvatore Sanfilippo.
.PP
This manual page was written by Chris Lamb <lamby@debian.org> for the Debian
project (but may be used by others). Modified by Ben Schermel <ben@eqalpha.com>
+2
View File
@@ -0,0 +1,2 @@
debian/keydb-sentinel.service /lib/systemd/system/
pkg/deb/conf/sentinel.conf /etc/keydb
+7
View File
@@ -0,0 +1,7 @@
/var/log/keydb/keydb-sentinel*.log {
weekly
missingok
rotate 12
compress
notifempty
}
+1
View File
@@ -0,0 +1 @@
debian/keydb-sentinel.1
+26
View File
@@ -0,0 +1,26 @@
#!/bin/sh
set -eu
USER="keydb"
GROUP="$USER"
CONFFILE="/etc/keydb/sentinel.conf"
if [ "$1" = "configure" ]
then
if ! dpkg-statoverride --list ${CONFFILE} >/dev/null 2>&1
then
dpkg-statoverride --update --add ${USER} ${GROUP} 640 ${CONFFILE}
fi
fi
#DEBHELPER#
if [ "$1" = "configure" ]
then
find /etc/keydb -maxdepth 1 -type d -name 'keydb-sentinel.*.d' -empty -delete
fi
systemctl daemon-reload
exit 0
+16
View File
@@ -0,0 +1,16 @@
#!/bin/sh
set -eu
CONFFILE="/etc/keydb/sentinel.conf"
if [ "$1" = "purge" ]
then
dpkg-statoverride --remove ${CONFFILE} || test $? -eq 2
fi
#DEBHELPER#
systemctl daemon-reload
exit 0
+23
View File
@@ -0,0 +1,23 @@
.TH KEYDB-SERVER 1 "August 17, 2019"
.SH NAME
keydb-server \- Persistent key-value database
.SH SYNOPSIS
.B keydb-server
.RI configfile
.SH DESCRIPTION
KeyDB is a key-value database. It is similar to memcached but the dataset is
not volatile and other datatypes (such as lists and sets) are natively
supported.
.PP
.SH OPTIONS
.IP "configfile"
Read options from specified configuration file.
.SH NOTES
On Debian GNU/Linux systems, \fBkeydb-server\fP is typically started via the
\fB/etc/init.d/keydb-server\fP initscript, not manually. This defaults to using
\fB/etc/keydb/keydb.conf\fP as a configuration file.
.SH AUTHOR
\fBkeydb-server\fP was written by John Sully, originating as a fork of Redis. Redis was written by Salvatore Sanfilippo.
.PP
This manual page was written by Chris Lamb <lamby@debian.org> for the Debian
project (but may be used by others). Modified by Ben Schermel <ben@eqalpha.com>
+1
View File
@@ -0,0 +1 @@
README.md
+2
View File
@@ -0,0 +1,2 @@
debian/keydb-server.service /lib/systemd/system/
pkg/deb/conf/keydb.conf /etc/keydb
+7
View File
@@ -0,0 +1,7 @@
/var/log/keydb/keydb-server*.log {
weekly
missingok
rotate 12
compress
notifempty
}
+1
View File
@@ -0,0 +1 @@
debian/keydb-server.1
+26
View File
@@ -0,0 +1,26 @@
#!/bin/sh
set -eu
USER="keydb"
GROUP="$USER"
CONFFILE="/etc/keydb/keydb.conf"
if [ "$1" = "configure" ]
then
if ! dpkg-statoverride --list ${CONFFILE} >/dev/null 2>&1
then
dpkg-statoverride --update --add ${USER} ${GROUP} 640 ${CONFFILE}
fi
fi
#DEBHELPER#
if [ "$1" = "configure" ]
then
find /etc/keydb -maxdepth 1 -type d -name 'keydb-server.*.d' -empty -delete
fi
systemctl daemon-reload
exit 0
+16
View File
@@ -0,0 +1,16 @@
#!/bin/sh
set -eu
CONFFILE="/etc/keydb/keydb.conf"
if [ "${1}" = "purge" ]
then
dpkg-statoverride --remove ${CONFFILE} || test $? -eq 2
fi
#DEBHELPER#
systemctl daemon-reload
exit 0
+2
View File
@@ -0,0 +1,2 @@
src/redis-trib.rb
utils/lru
+7
View File
@@ -0,0 +1,7 @@
debian/bash_completion.d/* /usr/share/bash-completion/completions
src/keydb-benchmark /usr/bin
src/keydb-check-aof /usr/bin
src/keydb-check-rdb /usr/bin
src/keydb-cli /usr/bin
src/keydb-server /usr/bin
src/keydb-sentinel /usr/bin
+4
View File
@@ -0,0 +1,4 @@
debian/keydb-benchmark.1
debian/keydb-check-aof.1
debian/keydb-check-rdb.1
debian/keydb-cli.1
+47
View File
@@ -0,0 +1,47 @@
#!/bin/sh
set -eu
USER="keydb"
Setup_dir () {
DIR="${1}"
MODE="${2}"
GROUP="${3}"
mkdir -p ${DIR}
case "${DIR}" in
/var/log/keydb)
MODE="02750"
GROUP="adm"
;;
*)
MODE="750"
GROUP="${USER}"
;;
esac
if ! dpkg-statoverride --list ${DIR} >/dev/null 2>&1
then
chown ${USER}:${GROUP} ${DIR}
chmod ${MODE} ${DIR}
fi
}
if [ "$1" = "configure" ]
then
adduser \
--system \
--home /var/lib/keydb \
--quiet \
--group \
${USER} || true
Setup_dir /var/log/keydb ${USER}:adm 2750
Setup_dir /var/lib/keydb ${USER}:${USER} 750
fi
#DEBHELPER#
exit 0
+13
View File
@@ -0,0 +1,13 @@
#!/bin/sh
set -eu
if [ "${1}" = "purge" ]
then
userdel keydb || true
rm -rf /var/lib/keydb /var/log/keydb
fi
#DEBHELPER#
exit 0
+39
View File
@@ -0,0 +1,39 @@
#!/usr/bin/make -f
include /usr/share/dpkg/buildflags.mk
export BUILD_TLS=yes
export CFLAGS CPPFLAGS LDFLAGS
export DEB_BUILD_MAINT_OPTIONS = hardening=+all
export DEB_LDFLAGS_MAINT_APPEND = -ldl -latomic $(LUA_LDFLAGS)
ifneq (,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))
NUMJOBS = $(patsubst parallel=%,%,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))
MAKEFLAGS += -j$(NUMJOBS)
export MAKEFLAGS
endif
%:
dh $@
override_dh_auto_install:
debian/bin/generate-systemd-service-files
dh_installsystemd --restart-after-upgrade
override_dh_auto_test:
ifeq (,$(filter nocheck,$(DEB_BUILD_OPTIONS)))
# Avoid race conditions in upstream testsuite.
./runtest || true
./runtest-cluster || true
./runtest-sentinel || true
endif
override_dh_auto_clean:
dh_auto_clean
rm -f src/release.h debian/*.service
override_dh_compress:
dh_compress -Xredis-trib.rb
override_dh_installchangelogs:
dh_installchangelogs --keep 00-RELEASENOTES
+1
View File
@@ -0,0 +1 @@
3.0 (quilt)
+2
View File
@@ -0,0 +1,2 @@
# Upstream do not provide signed tarballs.
keydb-pro source: debian-watch-does-not-check-gpg-signature
+8
View File
@@ -0,0 +1,8 @@
#!/bin/sh
#
# Show the INFO from "keydb-cli"
set -eu
keydb-cli INFO
keydb-cli LOLWUT
+7
View File
@@ -0,0 +1,7 @@
#!/bin/sh
#
# Run the benchmarking
set -eu
keydb-benchmark -P 10
+5
View File
@@ -0,0 +1,5 @@
#!/bin/sh
#
# Smoke test keydb-check-aof
keydb-check-aof 2>&1 | grep -qsi usage:
+10
View File
@@ -0,0 +1,10 @@
#!/bin/sh
#
# Test keydb-check-rdb
set -eu
# Perform a synchronous save to ensure .rdb file eixsts
keydb-cli SAVE
keydb-check-rdb /var/lib/keydb/dump.rdb
+8
View File
@@ -0,0 +1,8 @@
Tests: 0001-keydb-cli
Tests: 0002-benchmark
Tests: 0003-keydb-check-aof
Tests: 0004-keydb-check-rdb
Restrictions: needs-root
+3
View File
@@ -0,0 +1,3 @@
version=6
opts=uversionmangle=s/-?(alpha|beta|rc)/~$1/ \
https://github.com/JohnSully/KeyDB-Pro/releases .*/archive/(.*).tar.gz
+8
View File
@@ -0,0 +1,8 @@
keydb (4:4.0.2-3) unstable; urgency=medium
This version drops the Debian-specific support for the
/etc/keydb/keydb-{server,sentinel}.{pre,post}-{up,down}.d directories in
favour of using systemd's ExecStartPre, ExecStartPost, ExecStopPre,
ExecStopPost commands.
-- Chris Lamb <lamby@debian.org> Wed, 11 Oct 2017 22:55:00 -0400
@@ -0,0 +1,30 @@
# -*- sh -*-
#
# Bash completion function for the 'keydb-cli' command.
#
# Steve
# --
# http://www.steve.org.uk
#
_keydb-cli()
{
COMPREPLY=()
cur=${COMP_WORDS[COMP_CWORD]}
prev=${COMP_WORDS[COMP_CWORD-1]}
#
# All known commands accepted. Sorted.
#
opts='bgrewriteaof bgsave dbsize debug decr decrby del echo exists expire expireat flushall flushdb get getset incr incrby info keys lastsave lindex llen lpop lpush lrange lrem lset ltrim mget move mset msetnx ping randomkey rename renamenx rewriteaof rpop rpoplpush rpush sadd save scard sdiff sdiffstore select set setnx shutdown sinter sinterstore sismember slaveof smembers smove sort spop srandmember srem sunion sunionstore ttl type zadd zcard zincrby zrange zrangebyscore zrem zremrangebyscore zrevrange zscore'
#
# Only complete on the first term.
#
if [ $COMP_CWORD -eq 1 ]; then
COMPREPLY=( $(compgen -W "${opts}" -- ${cur}) )
return 0
fi
}
complete -F _keydb-cli keydb-cli
+131
View File
@@ -0,0 +1,131 @@
#!/bin/sh
SYSTEMD_VER=$(systemctl --version | head -1 | cut -d' ' -f2)
if [ ${SYSTEMD_VER} -lt 237 ]
then
SYSTEMD_EXTRA=""
else
SYSTEMD_EXTRA=$(cat <<EOF
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
#MemoryDenyWriteExecute=true
ProtectKernelModules=true
ProtectKernelTunables=true
ProtectControlGroups=true
RestrictRealtime=true
RestrictNamespaces=true
EOF
)
fi
for BINARY in keydb-server keydb-sentinel
do
for MODE in default templated
do
case "${BINARY}" in
keydb-server)
NAME="keydb"
;;
keydb-sentinel)
NAME="sentinel"
;;
esac
case "${MODE}" in
default)
EXTRA="Alias=${NAME}.service"
TARGET="debian/${BINARY}.service"
NAMESPACED="${NAME}"
DESCRIPTION="Advanced key-value store"
;;
templated)
EXTRA=""
TARGET="debian/${BINARY}@.service"
NAMESPACED="${NAME}-%i"
DESCRIPTION="Advanced key-value store (%I)"
;;
esac
: >${TARGET}
if [ "${MODE}" = "templated" ]
then
cat >> ${TARGET} <<EOF
# Templated service file for ${BINARY}(1)
#
# Each instance of ${BINARY} requires its own configuration file:
#
# $ cp /etc/keydb/${NAME}.conf /etc/keydb/${NAME}-myname.conf
# $ chown keydb:keydb /etc/keydb/${NAME}-myname.conf
#
# Ensure each instance is using their own database:
#
# $ sed -i -e 's@^dbfilename .*@dbfilename dump-myname.rdb@' /etc/keydb/${NAME}-myname.conf
#
# We then listen exlusively on UNIX sockets to avoid TCP port collisions:
#
# $ sed -i -e 's@^port .*@port 0@' /etc/keydb/${NAME}-myname.conf
# $ sed -i -e 's@^\\(# \\)\\{0,1\\}unixsocket .*@unixsocket /var/run/${NAME}-myname/${BINARY}.sock@' /etc/keydb/${NAME}-myname.conf
#
# ... and ensure we are logging, etc. in a unique location:
#
# $ sed -i -e 's@^logfile .*@logfile /var/log/keydb/${BINARY}-myname.log@' /etc/keydb/${NAME}-myname.conf
# $ sed -i -e 's@^pidfile .*@pidfile /var/run/keydb-myname/${BINARY}.pid@' /etc/keydb/${NAME}-myname.conf
#
# We can then start the service as follows, validating we are using our own
# configuration:
#
# $ systemctl start ${BINARY}@myname.service
# $ keydb-cli -s /var/run/${NAME}-myname/${BINARY}.sock info | grep config_file
#
# -- Chris Lamb <lamby@debian.org> Mon, 09 Oct 2017 22:17:24 +0100
EOF
fi
cat >> ${TARGET} <<EOF
[Unit]
Description=${DESCRIPTION}
After=network.target
Documentation=https://docs.keydb.dev, man:${BINARY}(1)
[Service]
Type=forking
ExecStart=/usr/bin/${BINARY} /etc/keydb/${NAMESPACED}.conf
ExecStop=/bin/kill -s TERM \$MAINPID
PIDFile=/var/run/${NAMESPACED}/${BINARY}.pid
TimeoutStopSec=0
Restart=always
User=keydb
Group=keydb
RuntimeDirectory=${NAMESPACED}
RuntimeDirectoryMode=2755
UMask=007
PrivateTmp=yes
LimitNOFILE=65535
PrivateDevices=yes
ProtectHome=yes
ReadOnlyDirectories=/
ReadWriteDirectories=-/var/lib/keydb
ReadWriteDirectories=-/var/log/keydb
ReadWriteDirectories=-/var/run/${NAMESPACED}
NoNewPrivileges=true
CapabilityBoundingSet=CAP_SETGID CAP_SETUID CAP_SYS_RESOURCE${SYSTEMD_EXTRA}
# ${BINARY} can write to its own config file when in cluster mode so we
# permit writing there by default. If you are not using this feature, it is
# recommended that you replace the following lines with "ProtectSystem=full".
ProtectSystem=true
ReadWriteDirectories=-/etc/keydb
[Install]
WantedBy=multi-user.target
EOF
if [ "${EXTRA}" != "" ]
then
echo "${EXTRA}" >> "${TARGET}"
fi
done
done
+32
View File
@@ -0,0 +1,32 @@
keydb (5:5.3.3-1~bionic1) bionic; urgency=medium
* 5.3.3 Updating deb source package and naming convention.
-- Ben Schermel <ben@eqalpha.com> Fri, 25 Oct 2019 8:00:37 +0000
keydb (5:5.1.12-1chl1~bionic1) bionic; urgency=medium
* 5.1.1 update. This update fixes several rare deadlock scenarios. Deadlock detection is also added.
-- Ben Schermel <ben@eqalpha.com> Fri, 25 Oct 2019 8:00:37 +0000
keydb (5:5.1.11-1chl1~bionic1) bionic; urgency=medium
* 5.1 release. This release includes subkey expires (EXPIREMEMBER/EXPIREMEMBERAT), with updates to PTTL/TTL accordingly. New OBJECT LASTMODIFIED, BITIOP LSHIFT & BITOP RSHIFT commands. See https://docs.keydb.dev/blog/2019/10/20/blog-post/ for detailed review of release.
-- Ben Schermel <ben@eqalpha.com> Mon, 21 Oct 2019 8:00:37 +0000
keydb (5:5.0.1-1chl1~bionic1) bionic; urgency=medium
* Arm build now included for bionic package
-- Ben Schermel <ben@eqalpha.com> Wed, 21 Aug 2019 22:58:37 +0000
keydb (5:5.0.0-1chl1~bionic1) bionic; urgency=medium
* Initial release of KeyDB PPA. This PPA was originally derived from https://launchpad.net/~chris-lea/+archive/ubuntu/redis-server
-- Ben Schermel <ben@eqalpha.com> Wed, 21 Aug 2019 2:58:37 +0000
+1
View File
@@ -0,0 +1 @@
9
+77
View File
@@ -0,0 +1,77 @@
Source: keydb
Section: database
Priority: optional
Maintainer: Ben Schermel <ben@eqalpha.com>
Build-Depends:
debhelper (>= 9~),
dpkg-dev (>= 1.17.5),
systemd,
procps <!nocheck>,
build-essential <!nocheck>,
tcl <!nocheck>,
tcl-dev <!nocheck>,
uuid-dev <!nocheck>,
libcurl4-openssl-dev <!nocheck>,
nasm <!nocheck>,
autotools-dev <!nocheck>,
autoconf <!nocheck>,
libjemalloc-dev <!nocheck>,
libssl-dev <!nocheck>
Standards-Version: 4.2.1
Homepage: https://docs.keydb.dev/
Vcs-Git: https://github.com/JohnSully/KeyDB.git
Vcs-Browser: https://github.com/JohnSully/KeyDB
Package: keydb
Architecture: all
Depends:
keydb-server (<< ${binary:Version}.1~),
keydb-server (>= ${binary:Version}),
${misc:Depends},
Description: Persistent key-value database with network interface (metapackage)
keydb is a key-value database in a similar vein to memcache but the dataset
is non-volatile. keydb additionally provides native support for atomically
manipulating and querying data structures such as lists and sets.
.
The dataset is stored entirely in memory and periodically flushed to disk.
.
This package depends on the keydb-server package.
Package: keydb-sentinel
Architecture: any
Depends:
lsb-base (>= 3.2-14),
keydb-tools (= ${binary:Version}),
${misc:Depends},
Description: Persistent key-value database with network interface (monitoring)
keydb is a key-value database in a similar vein to memcache but the dataset
is non-volatile. keydb additionally provides native support for atomically
manipulating and querying data structures such as lists and sets.
.
This package contains the keydb Sentinel monitoring software.
Package: keydb-server
Architecture: any
Depends:
lsb-base (>= 3.2-14),
keydb-tools (= ${binary:Version}),
${misc:Depends},
Description: Persistent key-value database with network interface
keydb is a key-value database in a similar vein to memcache but the dataset
is non-volatile. keydb additionally provides native support for atomically
manipulating and querying data structures such as lists and sets.
.
The dataset is stored entirely in memory and periodically flushed to disk.
Package: keydb-tools
Architecture: any
Depends:
adduser,
${misc:Depends},
${shlibs:Depends},
Description: Persistent key-value database with network interface (client)
keydb is a key-value database in a similar vein to memcache but the dataset
is non-volatile. keydb additionally provides native support for atomically
manipulating and querying data structures such as lists and sets.
.
This package contains the command line client and other tools.
+134
View File
@@ -0,0 +1,134 @@
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Contact: John Sully <john@eqalpha.com>
Upstream-Name: keydb
Source: https://github.com/JohnSully/KeyDB
Files: *
Copyright: © 2006-2014 Salvatore Sanfilippo <antirez@gmail.com>
Copyright © 2019, John Sully
License: BSD-3-clause
Files:
src/rio.*
src/t_zset.c
src/ziplist.h
src/intset.*
src/redis-check-aof.c
deps/hiredis/*
deps/linenoise/*
Copyright:
© 2009-2012 Pieter Noordhuis <pcnoordhuis@gmail.com>
© 2009-2012 Salvatore Sanfilippo <antirez@gmail.com>
License: BSD-3-clause
Files:
src/lzf.h
src/lzfP.h
src/lzf_d.c
src/lzf_c.c
Copyright:
© 2000-2007 Marc Alexander Lehmann <schmorp@schmorp.de>
© 2009-2012 Salvatore Sanfilippo <antirez@gmail.com>
License: BSD-2-clause
Files: src/setproctitle.c
Copyright:
© 2010 William Ahern
© 2013 Salvatore Sanfilippo
© 2013 Stam He
License: BSD-3-clause
Files: src/ae_evport.c
Copyright: © 2012 Joyent, Inc.
License: BSD-3-clause
Files: src/ae_kqueue.c
Copyright: © 2009 Harish Mallipeddi <harish.mallipeddi@gmail.com>
License: BSD-3-clause
Files: utils/install_server.sh
Copyright: © 2011 Dvir Volk <dvirsk@gmail.com>
License: BSD-3-clause
Files: deps/jemalloc/*
Copyright:
© 2002-2012 Jason Evans <jasone@canonware.com>
© 2007-2012 Mozilla Foundation
© 2009-2012 Facebook, Inc.
License: BSD-3-clause
Files: src/pqsort.*
Copyright: © 1992-1993 The Regents of the University of California
License: BSD-3-clause
Files: deps/lua/*
Copyright: © 1994-2012 Lua.org, PUC-Ri
License: MIT
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
.
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
Files: debian/*
Copyright: © 2009 Chris Lamb <lamby@debian.org>
License: BSD-3-clause
License: BSD-2-clause
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
.
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
License: BSD-3-clause
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
.
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of Redis nor the names of its contributors may be
used to endorse or promote products derived from this software
without specific prior written permission.
.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+1
View File
@@ -0,0 +1 @@
keydb_5.3.3-1~bionic1_source.buildinfo database optional
+3
View File
@@ -0,0 +1,3 @@
[DEFAULT]
debian-branch=debian/sid
upstream-branch=upstream/sid
+68
View File
@@ -0,0 +1,68 @@
.TH KEYDB-BENCHMARK 1 "August 17, 2019"
.SH NAME
keydb-benchmark \- Benechmark a KeyDB instance
.SH SYNOPSIS
.B redis-benchmark
[\-h <host>] [\-p <port>] [\-c <clients>] [\-n <requests]> [\-k <boolean>]
.SH DESCRIPTION
KeyDB is a key-value database. It is similar to memcached but the dataset is
not volatile and other datatypes (such as lists and sets) are natively
supported.
.SH OPTIONS
.TP
\-h hostname
Server hostname (default 127.0.0.1)
.TP
\-p port
Server port (default 6379)
.TP
\-s socket
Server socket (overrides host and port)
.TP
\-a password
Password for KeyDB Auth
.TP
\-c clients
Number of parallel connections (default 50)
.TP
\-n requests
Total number of requests (default 100000)
.TP
\-d size
Data size of SET/GET value in bytes (default 2)
.TP
\-dbnum db
SELECT the specified db number (default 0)
.TP
\-k boolean
1=keep alive 0=reconnect (default 1)
.TP
\-r keyspacelen
Use random keys for SET/GET/INCR, random values for SADD Using this option the
benchmark will get/set keys in the form mykey_rand000000012456 instead of
constant keys, the <keyspacelen> argument determines the max number of values
for the random number. For instance if set to 10 only rand000000000000 -
rand000000000009 range will be allowed.
.TP
\-P numreq
Pipeline <numreq> requests. Default 1 (no pipeline).
.TP
\-q
Quiet. Just show query/sec values
.TP
\-\-csv
Ourput in CSV format
.TP
\-l
Loop. Run the tests forever
.TP
\-I
Idle mode. Just open N idle connections and wait.
.TP
\-D
Debug mode. more verbose.
.SH AUTHOR
\fBkeydb-benchmark\fP was written by John Sully, originating as a fork of Redis. Redis was written by Salvatore Sanfilippo.
.PP
This manual page was written by Chris Lamb <lamby@debian.org> for the Debian
project (but may be used by others). Modified by Ben Schermel <ben@eqalpha.com>
+17
View File
@@ -0,0 +1,17 @@
.TH KEYDB-CHECK-AOF 1 "August 17, 2019"
.SH NAME
keydb-check-aof \- Check integrity of a KeyDB .AOF file
.SH SYNOPSIS
.B keydb-check-aof
filename
.SH DESCRIPTION
KeyDB is a key-value database. It is similar to memcached but the dataset is
not volatile and other datatypes (such as lists and sets) are natively
supported.
.PP
This utility checks the integrity of a dumped .AOF file.
.SH AUTHOR
\fBkeydb-check-aof\fP was written by John Sully, originating as a fork of Redis. Redis was written by Salvatore Sanfilippo.
.PP
This manual page was written by Chris Lamb <lamby@debian.org> for the Debian
project (but may be used by others). Modified by Ben Schermel <ben@eqalpha.com>
+17
View File
@@ -0,0 +1,17 @@
.TH KEYDB-CHECK-RDB 1 "August 17, 2019"
.SH NAME
keydb-check-rdb \- Check integrity of KeyDB dumped database file
.SH SYNOPSIS
.B keydb-check-rdb
filename
.SH DESCRIPTION
KeyDB is a key-value database. It is similar to memcached but the dataset is
not volatile and other datatypes (such as lists and sets) are natively
supported.
.PP
This utility checks the integrity of a dumped database file.
.SH AUTHOR
\fBredis-check-rdb\fP was written by John Sully, originating as a fork of Redis. Redis was written by Salvatore Sanfilippo.
.PP
This manual page was written by Chris Lamb <lamby@debian.org> for the Debian
project (but may be used by others). Modified by Ben Schermel <ben@eqalpha.com>
+19
View File
@@ -0,0 +1,19 @@
.TH KEYDB-CLI 1 "August 17, 2019"
.SH NAME
keydb-cli \- Command-line client to keydb-server
.SH SYNOPSIS
.B keydb-cli
.RI [options]
.SH DESCRIPTION
KeyDB is a key-value database. It is similar to memcached but the dataset is
not volatile and other datatypes (such as lists and sets) are natively
supported.
.PP
\fBkeydb-cli\fP provides a simple command-line interface to a KeyDB server.
.SH OPTIONS
See \fBkeydb-doc\fP for more information on the commands KeyDB accepts.
.SH AUTHOR
\fBkeydb-cli\fP was written by John Sully, originating as a fork of Redis. Redis was written by Salvatore Sanfilippo.
.PP
This manual page was written by Chris Lamb <lamby@debian.org> for the Debian
project (but may be used by others). Modified by Ben Schermel <ben@eqalpha.com>
+23
View File
@@ -0,0 +1,23 @@
.TH KEYDB-SENTINEL 1 "August 17, 2019"
.SH NAME
keydb-sentinel \- Persistent key-value database (cluster mode)
.SH SYNOPSIS
.B keydb-sentinel
.RI configfile
.SH DESCRIPTION
KeyDB is a key-value database. It is similar to memcached but the dataset is
not volatile and other datatypes (such as lists and sets) are natively
supported.
.PP
.SH OPTIONS
.IP "configfile"
Read options from specified configuration file.
.SH NOTES
On Debian GNU/Linux systems, \fBkeydb-sentinel\fP is typically started via the
\fB/etc/init.d/keydb-sentinel\fP initscript, not manually. This defaults to using
\fB/etc/keydb/sentinel.conf\fP as a configuration file.
.SH AUTHOR
\fBkeydb-sentinel\fP was written by John Sully, originating as a fork of Redis. Redis was written by Salvatore Sanfilippo.
.PP
This manual page was written by Chris Lamb <lamby@debian.org> for the Debian
project (but may be used by others). Modified by Ben Schermel <ben@eqalpha.com>
+8
View File
@@ -0,0 +1,8 @@
# keydb-sentinel configure options
# ULIMIT: Call ulimit -n with this argument prior to invoking KeyDB Sentinel
# itself. This may be required for high-concurrency environments. KeyDB
# Sentinel itself cannot alter its limits as it is not being run as root.
# (default: 65536)
#
ULIMIT=65536
+89
View File
@@ -0,0 +1,89 @@
#! /bin/sh
### BEGIN INIT INFO
# Provides: keydb-sentinel
# Required-Start: $syslog $remote_fs
# Required-Stop: $syslog $remote_fs
# Should-Start: $local_fs
# Should-Stop: $local_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: keydb-sentinel - Persistent key-value db monitor
# Description: keydb-sentinel - Persistent key-value db monitor
### END INIT INFO
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/bin/keydb-sentinel
DAEMON_ARGS=/etc/keydb/sentinel.conf
NAME=keydb-sentinel
DESC=keydb-sentinel
RUNDIR=/var/run/sentinel
PIDFILE=$RUNDIR/keydb-sentinel.pid
test -x $DAEMON || exit 0
if [ -r /etc/default/$NAME ]
then
. /etc/default/$NAME
fi
. /lib/lsb/init-functions
set -e
if [ "$(id -u)" != "0" ]
then
log_failure_msg "Must be run as root."
exit 1
fi
case "$1" in
start)
echo -n "Starting $DESC: "
mkdir -p $RUNDIR
touch $PIDFILE
chown keydb:keydb $RUNDIR $PIDFILE
chmod 755 $RUNDIR
if [ -n "$ULIMIT" ]
then
ulimit -n $ULIMIT || true
fi
if start-stop-daemon --start --quiet --oknodo --umask 007 --pidfile $PIDFILE --chuid keydb:keydb --exec $DAEMON -- $DAEMON_ARGS
then
echo "$NAME."
else
echo "failed"
fi
;;
stop)
echo -n "Stopping $DESC: "
if start-stop-daemon --stop --retry forever/TERM/1 --quiet --oknodo --pidfile $PIDFILE --exec $DAEMON
then
echo "$NAME."
else
echo "failed"
fi
rm -f $PIDFILE
sleep 1
;;
restart|force-reload)
${0} stop
${0} start
;;
status)
status_of_proc -p ${PIDFILE} ${DAEMON} ${NAME}
;;
*)
echo "Usage: /etc/init.d/$NAME {start|stop|restart|force-reload|status}" >&2
exit 1
;;
esac
exit 0
@@ -0,0 +1,2 @@
debian/keydb-sentinel.service /lib/systemd/system
pkg/deb/conf/sentinel.conf /etc/keydb
@@ -0,0 +1,7 @@
/var/log/keydb/keydb-sentinel*.log {
weekly
missingok
rotate 12
compress
notifempty
}
@@ -0,0 +1,4 @@
rm_conffile /etc/keydb/keydb-sentinel.post-down.d/00_example 4:4.0.2-3~
rm_conffile /etc/keydb/keydb-sentinel.post-up.d/00_example 4:4.0.2-3~
rm_conffile /etc/keydb/keydb-sentinel.pre-down.d/00_example 4:4.0.2-3~
rm_conffile /etc/keydb/keydb-sentinel.pre-up.d/00_example 4:4.0.2-3~
@@ -0,0 +1 @@
debian/keydb-sentinel.1
@@ -0,0 +1,24 @@
#!/bin/sh
set -eu
USER="keydb"
GROUP="$USER"
CONFFILE="/etc/keydb/sentinel.conf"
if [ "$1" = "configure" ]
then
if ! dpkg-statoverride --list ${CONFFILE} >/dev/null 2>&1
then
dpkg-statoverride --update --add ${USER} ${GROUP} 640 ${CONFFILE}
fi
fi
#DEBHELPER#
if [ "$1" = "configure" ]
then
find /etc/keydb -maxdepth 1 -type d -name 'keydb-sentinel.*.d' -empty -delete
fi
exit 0
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
set -eu
CONFFILE="/etc/keydb/sentinel.conf"
if [ "$1" = "purge" ]
then
dpkg-statoverride --remove ${CONFFILE} || test $? -eq 2
fi
#DEBHELPER#
exit 0
+23
View File
@@ -0,0 +1,23 @@
.TH KEYDB-SERVER 1 "August 17, 2019"
.SH NAME
keydb-server \- Persistent key-value database
.SH SYNOPSIS
.B keydb-server
.RI configfile
.SH DESCRIPTION
KeyDB is a key-value database. It is similar to memcached but the dataset is
not volatile and other datatypes (such as lists and sets) are natively
supported.
.PP
.SH OPTIONS
.IP "configfile"
Read options from specified configuration file.
.SH NOTES
On Debian GNU/Linux systems, \fBkeydb-server\fP is typically started via the
\fB/etc/init.d/keydb-server\fP initscript, not manually. This defaults to using
\fB/etc/keydb/keydb.conf\fP as a configuration file.
.SH AUTHOR
\fBkeydb-server\fP was written by John Sully, originating as a fork of Redis. Redis was written by Salvatore Sanfilippo.
.PP
This manual page was written by Chris Lamb <lamby@debian.org> for the Debian
project (but may be used by others). Modified by Ben Schermel <ben@eqalpha.com>
+7
View File
@@ -0,0 +1,7 @@
# keydb-server configure options
# ULIMIT: Call ulimit -n with this argument prior to invoking Redis itself.
# This may be required for high-concurrency environments. KeyDB itself cannot
# alter its limits as it is not being run as root. (default: 65536)
#
ULIMIT=65536
+1
View File
@@ -0,0 +1 @@
README.md
+89
View File
@@ -0,0 +1,89 @@
#! /bin/sh
### BEGIN INIT INFO
# Provides: keydb-server
# Required-Start: $syslog $remote_fs
# Required-Stop: $syslog $remote_fs
# Should-Start: $local_fs
# Should-Stop: $local_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: keydb-server - Persistent key-value db
# Description: keydb-server - Persistent key-value db
### END INIT INFO
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/bin/keydb-server
DAEMON_ARGS=/etc/keydb/keydb.conf
NAME=keydb-server
DESC=keydb-server
RUNDIR=/var/run/keydb
PIDFILE=$RUNDIR/keydb-server.pid
test -x $DAEMON || exit 0
if [ -r /etc/default/$NAME ]
then
. /etc/default/$NAME
fi
. /lib/lsb/init-functions
set -e
if [ "$(id -u)" != "0" ]
then
log_failure_msg "Must be run as root."
exit 1
fi
case "$1" in
start)
echo -n "Starting $DESC: "
mkdir -p $RUNDIR
touch $PIDFILE
chown keydb:keydb $RUNDIR $PIDFILE
chmod 755 $RUNDIR
if [ -n "$ULIMIT" ]
then
ulimit -n $ULIMIT || true
fi
if start-stop-daemon --start --quiet --oknodo --umask 007 --pidfile $PIDFILE --chuid keydb:keydb --exec $DAEMON -- $DAEMON_ARGS
then
echo "$NAME."
else
echo "failed"
fi
;;
stop)
echo -n "Stopping $DESC: "
if start-stop-daemon --stop --retry forever/TERM/1 --quiet --oknodo --pidfile $PIDFILE --exec $DAEMON
then
echo "$NAME."
else
echo "failed"
fi
rm -f $PIDFILE
sleep 1
;;
restart|force-reload)
${0} stop
${0} start
;;
status)
status_of_proc -p ${PIDFILE} ${DAEMON} ${NAME}
;;
*)
echo "Usage: /etc/init.d/$NAME {start|stop|restart|force-reload|status}" >&2
exit 1
;;
esac
exit 0
+2
View File
@@ -0,0 +1,2 @@
debian/keydb-server.service /lib/systemd/system
pkg/deb/conf/keydb.conf /etc/keydb
@@ -0,0 +1,7 @@
/var/log/keydb/keydb-server*.log {
weekly
missingok
rotate 12
compress
notifempty
}
@@ -0,0 +1,4 @@
rm_conffile /etc/keydb/keydb-server.post-down.d/00_example 4:4.0.2-3~
rm_conffile /etc/keydb/keydb-server.post-up.d/00_example 4:4.0.2-3~
rm_conffile /etc/keydb/keydb-server.pre-down.d/00_example 4:4.0.2-3~
rm_conffile /etc/keydb/keydb-server.pre-up.d/00_example 4:4.0.2-3~
+1
View File
@@ -0,0 +1 @@
debian/keydb-server.1
+24
View File
@@ -0,0 +1,24 @@
#!/bin/sh
set -eu
USER="keydb"
GROUP="$USER"
CONFFILE="/etc/keydb/keydb.conf"
if [ "$1" = "configure" ]
then
if ! dpkg-statoverride --list ${CONFFILE} >/dev/null 2>&1
then
dpkg-statoverride --update --add ${USER} ${GROUP} 640 ${CONFFILE}
fi
fi
#DEBHELPER#
if [ "$1" = "configure" ]
then
find /etc/keydb -maxdepth 1 -type d -name 'keydb-server.*.d' -empty -delete
fi
exit 0
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
set -eu
CONFFILE="/etc/keydb/keydb.conf"
if [ "${1}" = "purge" ]
then
dpkg-statoverride --remove ${CONFFILE} || test $? -eq 2
fi
#DEBHELPER#
exit 0
+2
View File
@@ -0,0 +1,2 @@
src/redis-trib.rb
utils/lru
+7
View File
@@ -0,0 +1,7 @@
debian/bash_completion.d/* /usr/share/bash-completion/completions
src/keydb-server /usr/bin
src/keydb-benchmark /usr/bin
src/keydb-check-aof /usr/bin
src/keydb-check-rdb /usr/bin
src/keydb-cli /usr/bin
src/keydb-sentinel /usr/bin
+4
View File
@@ -0,0 +1,4 @@
debian/keydb-benchmark.1
debian/keydb-check-aof.1
debian/keydb-check-rdb.1
debian/keydb-cli.1
+47
View File
@@ -0,0 +1,47 @@
#!/bin/sh
set -eu
USER="keydb"
Setup_dir () {
DIR="${1}"
MODE="${2}"
GROUP="${3}"
mkdir -p ${DIR}
case "${DIR}" in
/var/log/keydb)
MODE="02750"
GROUP="adm"
;;
*)
MODE="750"
GROUP="${USER}"
;;
esac
if ! dpkg-statoverride --list ${DIR} >/dev/null 2>&1
then
chown ${USER}:${GROUP} ${DIR}
chmod ${MODE} ${DIR}
fi
}
if [ "$1" = "configure" ]
then
adduser \
--system \
--home /var/lib/keydb \
--quiet \
--group \
${USER} || true
Setup_dir /var/log/keydb ${USER}:adm 2750
Setup_dir /var/lib/keydb ${USER}:${USER} 750
fi
#DEBHELPER#
exit 0
+13
View File
@@ -0,0 +1,13 @@
#!/bin/sh
set -eu
if [ "${1}" = "purge" ]
then
userdel keydb || true
rm -rf /var/lib/keydb /var/log/keydb
fi
#DEBHELPER#
exit 0
@@ -0,0 +1,21 @@
From: Chris Lamb <lamby@debian.org>
Date: Fri, 30 Oct 2015 10:53:42 +0000
Subject: fix-ftbfs-on-kfreebsd
---
src/fmacros.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/fmacros.h b/src/fmacros.h
index 6e56c75..d490aec 100644
--- a/src/fmacros.h
+++ b/src/fmacros.h
@@ -41,7 +41,7 @@
#define _ALL_SOURCE
#endif
-#if defined(__linux__) || defined(__OpenBSD__)
+#if defined(__linux__) || defined(__OpenBSD__) || defined(__GLIBC__)
#define _XOPEN_SOURCE 700
/*
* On NetBSD, _XOPEN_SOURCE undefines _NETBSD_SOURCE and
@@ -0,0 +1,84 @@
From: Chris Lamb <lamby@debian.org>
Date: Wed, 3 Oct 2018 19:24:16 +0100
Subject: Add support for a USE_SYSTEM_HIREDIS flag.
---
deps/Makefile | 2 ++
src/Makefile | 19 ++++++++++++++-----
2 files changed, 16 insertions(+), 5 deletions(-)
diff --git a/deps/Makefile b/deps/Makefile
index 2ed7736..d6b8e3e 100644
--- a/deps/Makefile
+++ b/deps/Makefile
@@ -33,7 +33,9 @@ ifneq ($(shell sh -c '[ -f .make-ldflags ] && cat .make-ldflags || echo none'),
endif
distclean:
+ifneq ($(USE_SYSTEM_HIREDIS),yes)
-(cd hiredis && $(MAKE) clean) > /dev/null || true
+endif
-(cd linenoise && $(MAKE) clean) > /dev/null || true
ifneq ($(USE_SYSTEM_LUA),yes)
-(cd lua && $(MAKE) clean) > /dev/null || true
diff --git a/src/Makefile b/src/Makefile
index 49085f2..9e7e6f1 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -16,7 +16,7 @@ release_hdr := $(shell sh -c './mkreleasehdr.sh')
uname_S := $(shell sh -c 'uname -s 2>/dev/null || echo not')
uname_M := $(shell sh -c 'uname -m 2>/dev/null || echo not')
OPTIMIZATION?=-O2
-DEPENDENCY_TARGETS=hiredis linenoise
+DEPENDENCY_TARGETS=linenoise
NODEPS:=clean distclean
# Default settings
@@ -107,7 +107,7 @@ endif
endif
endif
# Include paths to dependencies
-FINAL_CFLAGS+= -I../deps/hiredis -I../deps/linenoise
+FINAL_CFLAGS+= -I../deps/linenoise
ifeq ($(MALLOC),tcmalloc)
FINAL_CFLAGS+= -DUSE_TCMALLOC
@@ -139,6 +139,15 @@ else
FINAL_LIBS := ../deps/lua/src/liblua.a $(FINAL_LIBS)
endif
+ifeq ($(USE_SYSTEM_HIREDIS),yes)
+ FINAL_LIBS+= -lhiredis
+ FINAL_CFLAGS+= -I/usr/include/hiredis
+else
+ DEPENDENCY_TARGETS += hiredis
+ FINAL_LIBS+= ../deps/hiredis/libhiredis.a
+ FINAL_CFLAGS+= -I../deps/hiredis
+endif
+
REDIS_CC=$(QUIET_CC)$(CC) $(FINAL_CFLAGS) $(CPPFLAGS)
REDIS_LD=$(QUIET_LINK)$(CC) $(FINAL_LDFLAGS)
REDIS_INSTALL=$(QUIET_INSTALL)$(INSTALL)
@@ -210,7 +219,7 @@ endif
# redis-server
$(REDIS_SERVER_NAME): $(REDIS_SERVER_OBJ)
- $(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a $(FINAL_LIBS)
+ $(REDIS_LD) -o $@ $^ $(FINAL_LIBS)
# redis-sentinel
$(REDIS_SENTINEL_NAME): $(REDIS_SERVER_NAME)
@@ -226,11 +235,11 @@ $(REDIS_CHECK_AOF_NAME): $(REDIS_SERVER_NAME)
# redis-cli
$(REDIS_CLI_NAME): $(REDIS_CLI_OBJ)
- $(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a ../deps/linenoise/linenoise.o $(FINAL_LIBS)
+ $(REDIS_LD) -o $@ $^ ../deps/linenoise/linenoise.o $(FINAL_LIBS)
# redis-benchmark
$(REDIS_BENCHMARK_NAME): $(REDIS_BENCHMARK_OBJ)
- $(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a $(FINAL_LIBS)
+ $(REDIS_LD) -o $@ $^ $(FINAL_LIBS)
dict-benchmark: dict.c zmalloc.c sds.c siphash.c
$(REDIS_CC) $(FINAL_CFLAGS) $^ -D DICT_BENCHMARK_MAIN -o $@ $(FINAL_LIBS)
@@ -0,0 +1,107 @@
From: Chris Lamb <lamby@debian.org>
Date: Sat, 25 Aug 2018 17:52:13 +0200
Subject: Add support for USE_SYSTEM_JEMALLOC flag.
https://github.com/antirez/redis/pull/5279
---
deps/Makefile | 2 ++
src/Makefile | 5 +++++
src/object.c | 5 +++++
src/zmalloc.c | 10 ++++++++++
src/zmalloc.h | 4 ++++
5 files changed, 26 insertions(+)
diff --git a/deps/Makefile b/deps/Makefile
index eb35c1e..1342fac 100644
--- a/deps/Makefile
+++ b/deps/Makefile
@@ -36,7 +36,9 @@ distclean:
-(cd hiredis && $(MAKE) clean) > /dev/null || true
-(cd linenoise && $(MAKE) clean) > /dev/null || true
-(cd lua && $(MAKE) clean) > /dev/null || true
+ifneq ($(USE_SYSTEM_JEMALLOC),yes)
-(cd jemalloc && [ -f Makefile ] && $(MAKE) distclean) > /dev/null || true
+endif
-(rm -f .make-*)
.PHONY: distclean
diff --git a/src/Makefile b/src/Makefile
index 0ff6e8b..51363fe 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -120,10 +120,15 @@ ifeq ($(MALLOC),tcmalloc_minimal)
endif
ifeq ($(MALLOC),jemalloc)
+ifeq ($(USE_SYSTEM_JEMALLOC),yes)
+ FINAL_CFLAGS+= -DUSE_JEMALLOC -I/usr/include/jemalloc/include
+ FINAL_LIBS := -ljemalloc $(FINAL_LIBS)
+else
DEPENDENCY_TARGETS+= jemalloc
FINAL_CFLAGS+= -DUSE_JEMALLOC -I../deps/jemalloc/include
FINAL_LIBS := ../deps/jemalloc/lib/libjemalloc.a $(FINAL_LIBS)
endif
+endif
REDIS_CC=$(QUIET_CC)$(CC) $(FINAL_CFLAGS) $(CPPFLAGS)
REDIS_LD=$(QUIET_LINK)$(CC) $(FINAL_LDFLAGS)
diff --git a/src/object.c b/src/object.c
index 6987e1e..e49c2c2 100644
--- a/src/object.c
+++ b/src/object.c
@@ -36,6 +36,11 @@
#define strtold(a,b) ((long double)strtod((a),(b)))
#endif
+#if defined(USE_JEMALLOC) && (USE_SYSTEM_JEMALLOC == yes)
+#define je_mallctl mallctl
+#define je_malloc_stats_print malloc_stats_print
+#endif
+
/* ===================== Creation and parsing of objects ==================== */
robj *createObject(int type, void *ptr) {
diff --git a/src/zmalloc.c b/src/zmalloc.c
index 2482f51..80e6571 100644
--- a/src/zmalloc.c
+++ b/src/zmalloc.c
@@ -63,6 +63,15 @@ void zlibc_free(void *ptr) {
#define realloc(ptr,size) tc_realloc(ptr,size)
#define free(ptr) tc_free(ptr)
#elif defined(USE_JEMALLOC)
+#if USE_SYSTEM_JEMALLOC == yes
+#define malloc(size) malloc(size)
+#define calloc(count,size) calloc(count,size)
+#define realloc(ptr,size) realloc(ptr,size)
+#define free(ptr) free(ptr)
+#define mallocx(size,flags) mallocx(size,flags)
+#define dallocx(ptr,flags) dallocx(ptr,flags)
+#define je_mallctl mallctl
+#else
#define malloc(size) je_malloc(size)
#define calloc(count,size) je_calloc(count,size)
#define realloc(ptr,size) je_realloc(ptr,size)
@@ -70,6 +79,7 @@ void zlibc_free(void *ptr) {
#define mallocx(size,flags) je_mallocx(size,flags)
#define dallocx(ptr,flags) je_dallocx(ptr,flags)
#endif
+#endif
#define update_zmalloc_stat_alloc(__n) do { \
size_t _n = (__n); \
diff --git a/src/zmalloc.h b/src/zmalloc.h
index 6fb19b0..62ccf29 100644
--- a/src/zmalloc.h
+++ b/src/zmalloc.h
@@ -50,7 +50,11 @@
#include <jemalloc/jemalloc.h>
#if (JEMALLOC_VERSION_MAJOR == 2 && JEMALLOC_VERSION_MINOR >= 1) || (JEMALLOC_VERSION_MAJOR > 2)
#define HAVE_MALLOC_SIZE 1
+#if USE_SYSTEM_JEMALLOC == yes
+#define zmalloc_size(p) malloc_usable_size(p)
+#else
#define zmalloc_size(p) je_malloc_usable_size(p)
+#endif
#else
#error "Newer version of jemalloc required"
#endif
@@ -0,0 +1,75 @@
From: Chris Lamb <lamby@debian.org>
Date: Wed, 24 Jan 2018 22:06:35 +1100
Subject: Use get_current_dir_name over PATHMAX, etc.
---
src/aof.c | 3 ++-
src/rdb.c | 7 ++++---
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/src/aof.c b/src/aof.c
index 9723fc3..cc89847 100644
--- a/src/aof.c
+++ b/src/aof.c
@@ -246,7 +246,7 @@ int startAppendOnly(void) {
newfd = open(server.aof_filename,O_WRONLY|O_APPEND|O_CREAT,0644);
serverAssert(server.aof_state == AOF_OFF);
if (newfd == -1) {
- char *cwdp = getcwd(cwd,MAXPATHLEN);
+ char *cwdp = get_current_dir_name();
serverLog(LL_WARNING,
"Redis needs to enable the AOF but can't open the "
@@ -254,6 +254,7 @@ int startAppendOnly(void) {
server.aof_filename,
cwdp ? cwdp : "unknown",
strerror(errno));
+ zfree(cwdp);
return C_ERR;
}
if (server.rdb_child_pid != -1) {
diff --git a/src/rdb.c b/src/rdb.c
index 3e43cb4..6058160 100644
--- a/src/rdb.c
+++ b/src/rdb.c
@@ -1218,7 +1218,6 @@ werr: /* Write error. */
/* Save the DB on disk. Return C_ERR on error, C_OK on success. */
int rdbSave(char *filename, rdbSaveInfo *rsi) {
char tmpfile[256];
- char cwd[MAXPATHLEN]; /* Current working dir path for error messages. */
FILE *fp;
rio rdb;
int error = 0;
@@ -1226,13 +1225,14 @@ int rdbSave(char *filename, rdbSaveInfo *rsi) {
snprintf(tmpfile,256,"temp-%d.rdb", (int) getpid());
fp = fopen(tmpfile,"w");
if (!fp) {
- char *cwdp = getcwd(cwd,MAXPATHLEN);
+ char *cwdp = get_current_dir_name();
serverLog(LL_WARNING,
"Failed opening the RDB file %s (in server root dir %s) "
"for saving: %s",
filename,
cwdp ? cwdp : "unknown",
strerror(errno));
+ zfree(cwdp);
return C_ERR;
}
@@ -1254,7 +1254,7 @@ int rdbSave(char *filename, rdbSaveInfo *rsi) {
/* Use RENAME to make sure the DB file is changed atomically only
* if the generate DB file is ok. */
if (rename(tmpfile,filename) == -1) {
- char *cwdp = getcwd(cwd,MAXPATHLEN);
+ char *cwdp = get_current_dir_name();
serverLog(LL_WARNING,
"Error moving temp DB file %s on the final "
"destination %s (in server root dir %s): %s",
@@ -1262,6 +1262,7 @@ int rdbSave(char *filename, rdbSaveInfo *rsi) {
filename,
cwdp ? cwdp : "unknown",
strerror(errno));
+ zfree(cwdp);
unlink(tmpfile);
return C_ERR;
}
Loaded 100 of 254 files, more files were not shown because too many files have changed in this diff. Show more