Compare commits

...
68 Commits
Author SHA1 Message Date
benschermel 89beaf2460 add script to compare all active config parameters of any 2 config files 2021-08-10 20:10:28 -04:00
benschermel 0c5d12b914 update packaging config files 2021-08-10 20:08:39 -04:00
Christian Legge d7f6bc1614 Add REPLPING command for use during replication (#329)
* added replping command for initiating replication

* backwards compatibility for replping (retry if not recognized)

* don't allow ping during loading (load balancer fix)

* changed replping warning to notice
2021-07-29 15:50:30 -04:00
MalavanEQAlpha 83d878f111 Merge pull request #328 from EQ-Alpha/backtrace_for_alpine
Backtrace for alpine using libunwind
2021-07-28 17:17:06 -04:00
malavan 0fc6441c96 add some comments 2021-07-28 20:32:18 +00:00
malavan 5be1620629 add stacktrace message 2021-07-19 23:17:18 +00:00
malavan 685b87e2ef dd comment and set eip unused 2021-07-19 17:32:38 +00:00
malavan 72429c2106 dd comment and set eip unused 2021-07-19 17:31:24 +00:00
malavan 7707fe15a9 dd comment and set eip unused 2021-07-19 17:29:43 +00:00
malavan c583ffff41 add backtrace for alpine using libunwind 2021-07-19 17:20:52 +00:00
Christian Legge 7c1ab8b8e2 Merge pull request #321 from EQ-Alpha/ci-flags-fix
Requested changes from 6.2.2 merge
2021-07-15 16:42:02 -04:00
christianEQ 570fbd0149 fix warnings for mac build 2021-07-15 20:14:14 +00:00
christianEQ da1f09e9b5 Merge remote-tracking branch 'origin/unstable' into ci-flags-fix 2021-07-14 22:56:15 +00:00
christianEQ b2fb4f4b9b added REDIS_CFLAGS to makefile for backwards compatibility 2021-07-14 21:51:44 +00:00
malavan 51277b17a7 cleanup based on 6.2.2 merge review 2021-07-14 15:22:27 -04:00
Christian Legge 32f5ef864e Merge pull request #322 from EQ-Alpha/diagnostic-tool
Basic first iteration of Diagnostic Tool
2021-07-14 14:19:44 -04:00
Christian Legge a8f28297ec Merge pull request #327 from EQ-Alpha/issue323_fix
fix for #323: seg fault on expiremember
2021-07-12 16:54:31 -04:00
John Sully 1895dbb768 Merge tag '6.2.3' into unstable 2021-07-09 04:40:31 +00:00
christianEQ a6cbbd0f31 added eq alpha copyright 2021-07-08 02:58:18 +00:00
christianEQ e3c619eca4 various branding and cleanup fixes 2021-07-08 02:46:42 +00:00
christianEQ 57e59d97db removed unused command line options 2021-06-22 17:13:35 +00:00
christianEQ d7cada0f38 changed --ms to --time 2021-06-21 19:57:14 +00:00
christianEQ fd2ac1fcfc added more verbose options 2021-06-21 18:40:37 +00:00
christianEQ 0181b0e7a1 -t threads option 2021-06-21 18:36:21 +00:00
christianEQ 2b5e5cfa4c added starting line so diagnostic tool doesnt look frozen at first 2021-06-18 20:21:47 +00:00
christianEQ bfb0d02622 Merge remote-tracking branch 'origin/unstable' into diagnostic-tool 2021-06-18 20:19:31 +00:00
christianEQ 534b70643b detect stagnating server load before 100% (diagnostic tool) 2021-06-18 19:01:51 +00:00
christianEQ 3d0f729572 added config option for time to spin up new client threads (diagnostic tool) 2021-06-18 16:58:16 +00:00
christianEQ fd4ed1e425 detect full load + server threads (diagnostic tool) 2021-06-18 16:49:25 +00:00
christianEQ 693450393c detect client cpu usage in diagnostic tool 2021-06-17 20:40:10 +00:00
christianEQ 8ee7584cff accurate cpu usage in diagnostic tool 2021-06-17 18:32:11 +00:00
christianEQ 0727393627 changed redis_flags names to keydb in makefile 2021-06-14 16:09:42 +00:00
christianEQ d2ef059471 several redis->keydb changes in sentinel.conf 2021-06-11 19:16:33 +00:00
christianEQ 2e894a512b renamed redis mentions in runtest files 2021-06-11 19:11:12 +00:00
christianEQ 34b0209f05 renamed redis mention in readme 2021-06-11 19:10:35 +00:00
christianEQ 046f575632 renamed workspace gitignore 2021-06-11 19:10:12 +00:00
christianEQ 1c77104b5e renamed redis test files 2021-06-11 19:09:40 +00:00
christianEQ 386be8990a removed unreliable musl test and left only accurate new one 2021-06-11 18:19:59 +00:00
christianEQ 76d9c7c82a changed redis.pid to keydb.pid where necessary 2021-06-11 18:19:02 +00:00
christianEQ 98f08034a1 removed outdated comment in server.cpp 2021-06-11 18:17:18 +00:00
christianEQ 308cf77437 updated pkg/deb/conf/keydb.conf to use keydb name throughout 2021-06-11 18:15:50 +00:00
christianEQ ce64f469f4 removed the question issue template - doesn't seem necessary for us 2021-06-11 18:08:19 +00:00
christianEQ e632c3db1b fixed flag var names in ci 2021-06-11 17:04:33 +00:00
christianEQ 239d22ed72 don't enforce upper limit for requests issued; tool will run indefinitely 2021-06-11 16:28:29 +00:00
christianEQ fefbe96816 working diag tool prototype 2021-06-01 20:54:48 +00:00
Oran Agra e90e5640e7 Redis 6.2.3 2021-05-03 22:57:00 +03:00
Oran Agra 2df6695f2b Resolve nonsense static analysis warnings
(cherry picked from commit fd7d51c353607f350c865155444bce9236f3d682)
2021-05-03 22:57:00 +03:00
Oran Agra 92e3b1802f Fix integer overflow in STRALGO LCS (CVE-2021-29477)
An integer overflow bug in Redis version 6.0 or newer could be exploited using
the STRALGO LCS command to corrupt the heap and potentially result with remote
code execution.

(cherry picked from commit f0c5f920d0f88bd8aa376a2c05af4902789d1ef9)
2021-05-03 22:57:00 +03:00
Oran Agra 0463520693 Fix integer overflow in intset (CVE-2021-29478)
An integer overflow bug in Redis 6.2 could be exploited to corrupt the heap and
potentially result with remote code execution.

The vulnerability involves changing the default set-max-intset-entries
configuration value, creating a large set key that consists of integer values
and using the COPY command to duplicate it.

The integer overflow bug exists in all versions of Redis starting with 2.6,
where it could result with a corrupted RDB or DUMP payload, but not exploited
through COPY (which did not exist before 6.2).

(cherry picked from commit 29900d4e6bccdf3691bedf0ea9a5d84863fa3592)
2021-05-03 22:57:00 +03:00
sundb 922e3bf59f Fix memory leak in moduleDefragGlobals (#8853)
(cherry picked from commit 5100ef9f8246dec6590f35f6b9f0b88c2dea0cfb)
2021-05-03 22:57:00 +03:00
Huang Zhw 42f2ad0516 Improve redis-cli help. When help command, we only match command (#8879)
prefix args not all args. So when we help commands with subcommands,
all subcommands will be output.

(cherry picked from commit 0b1b9edb2843730b03f78b6073cdd30873dbba95)
2021-05-03 22:57:00 +03:00
Binbin 1eab6202ac redis-benchmark: Add zfree(data) and fix lrange size / text mismatch (#8872)
missing zfree(data) in redis-benchmark.

And also correct the wrong size in lrange.
the text mentioned 500, but size was 450, changed to 500

(cherry picked from commit 1eff8564c78011f7257e485796990a0d4d607a5b)
2021-05-03 22:57:00 +03:00
Binbin d30ee6c44a redis-cli: Do not use hostsocket when we got redirected in cluster mode (#8870)
When redis-cli was used with both -c (cluster) and -s (unix socket),
it would have kept trying to use that unix socket, even if it got
redirected by the cluster (resulting in an infinite loop).

(cherry picked from commit 416f2773395ffcd72d8d8408e1558f49d59a0077)
2021-05-03 22:57:00 +03:00
filipe oliveira 4f2da00e94 redis-benchmark: Error/Warning handling updates. (#8869)
- Immediately exit on errors that are not related to topology updates.
- Deprecates the `-e` option ( retro compatible ) and warns that we now
  exit immediately on errors that are not related to topology updates.
- Fixed wrongfully failing on config fetch error (warning only). This only affects RE.

Bottom line:
- MOVED and ASK errors will not show any warning (unlike the throttled error with `-e` before).
- CLUSTERDOWN still prints an error unconditionally and sleeps for 1 second.
- other errors are fatal.

(cherry picked from commit ef6f902372d4646b1894ec5dbd5f857dea5688d6)
2021-05-03 22:57:00 +03:00
Huang Zhw 34b9a3fa2e Fix potential CONFIG SET bind test failure. (#8875)
Use an invalid IP address to trigger CONFIG SET bind failure, instead of DNS which is not guaranteed to always fail.

(cherry picked from commit 2b22fffc787e91df789dabf23ddcf19ecf34cf6f)
2021-05-03 22:57:00 +03:00
yoav-steinberg 5c7b869e61 Bump freebsd-vm version to fix CI failures (#8876)
Specifically we had issues with NTP sync failure which was resolved here: https://github.com/vmactions/freebsd-vm/commit/457af7345642e154a79d219971a2d4a7c7fe2118

(cherry picked from commit 2e88b0639689a3019e27f55dfa40578847443eeb)
2021-05-03 22:57:00 +03:00
Oran Agra 6cbea7d29b Prevent replicas from sending commands that interact with keyspace (#8868)
This solves an issue reported in #8712 in which a replica would bypass
the client write pause check and cause an assertion due to executing a
write command during failover.

The fact is that we don't expect replicas to execute any command other
than maybe REPLCONF and PING, etc. but matching against the ADMIN
command flag is insufficient, so instead i just block keyspace access
for now.

(cherry picked from commit 46f4ebbe842620f0976a36741a72482620aa4b48)
2021-05-03 22:57:00 +03:00
Yossi Gottlieb 8cfa37fc21 Remove redundant -latomic on arm64. (#8867)
(cherry picked from commit ebfbb091096b6f36cf82e9f6e6583b10fd5b5acb)
2021-05-03 22:57:00 +03:00
Yang Bodong 16c53085f7 When the password is wrong, redis-benchmark should exit (#8855)
(cherry picked from commit 8423b77f14c0d3a58e580c65a70b4f980f5cdcf6)
2021-05-03 22:57:00 +03:00
Istemi Ekin Akkus 2ccb926314 Modules: Fix RM_GetClusterNodeInfo() to correctly populate the master_id (#8846)
(cherry picked from commit af035c1e1d3bcaf662051cff4dc49f6051321c9c)
2021-05-03 22:57:00 +03:00
zyxwvu Shi 6384fe3414 Use monotonic clock to check for Lua script timeout. (#8812)
This prevents a case where NTP moves the system clock
forward resulting in a false detection of a busy script.

Signed-off-by: zyxwvu Shi <i@shiyc.cn>
(cherry picked from commit f61c37cec900ba391541f20f7655aad44a26bafc)
2021-05-03 22:57:00 +03:00
Wang Yuan ef64333e63 Expire key firstly and then notify keyspace event (#8830)
Modules event subscribers may get wrong things in notifyKeyspaceEvent callback,
such as wrong number of keys, or be able to lookup this key.
This commit changes the order to be like the one in evict.c.

Cleanup:
Since we know the key exists (it expires now), db*Delete is sure to return 1,
so there's no need to check it's output (misleading).

(cherry picked from commit 63acfe4b00b9d3e34a53559f965c0bc44c03db61)
2021-05-03 22:57:00 +03:00
bugwz 0851705304 Print the number of abnormal line in AOF (#8823)
When redis-check-aof finds an error, it prints the line number for faster troubleshooting. 

(cherry picked from commit 761d7d27711edfbf737def41ff28f5b325fb16c8)
2021-05-03 22:57:00 +03:00
Huang Zhw b97a4ad7f2 Fix migrateCommand may migrate wrong value. (#8815)
This scene is hard to happen. When first attempt some keys expired,
only kv position is updated not ov. Then socket err happens, second
attempt is taken. This time kv items may be mismatching with ov items.

(cherry picked from commit 080d4579db40d965f8392af5b1da7a99d1a817d5)
2021-05-03 22:57:00 +03:00
Madelyn Olson d01afe885c Fix memory leak when doing lazyfreeing client tracking table (#8822)
Interior rax pointers were not being freed

(cherry picked from commit c73b4ddfd96d00ed0d0fde17953ce63d78bc3777)
2021-05-03 22:57:00 +03:00
christianEQ 66547bd28a fixed diagnostic tool to use correct obj files 2021-04-28 16:02:44 +00:00
christianEQ e6ab823473 updated makefile vars to keydb names 2021-04-27 21:14:05 +00:00
christianEQ a2c0bce4cc added keydb-diagnostic-tool binary (copy of benchmark) 2021-04-27 20:56:37 +00:00
63 changed files with 2734 additions and 946 deletions

No files matched your search

-21
View File
@@ -1,21 +0,0 @@
---
name: Question
about: Ask the Redis developers
title: '[QUESTION]'
labels: ''
assignees: ''
---
Please keep in mind that this issue tracker should be used for reporting bugs or proposing improvements to the Redis server.
Generally, questions about using Redis should be directed to the [community](https://redis.io/community):
* [the mailing list](https://groups.google.com/forum/#!forum/redis-db)
* [the `redis` tag at StackOverflow](http://stackoverflow.com/questions/tagged/redis)
* [/r/redis subreddit](http://www.reddit.com/r/redis)
* [the irc channel #redis](http://webchat.freenode.net/?channels=redis) on freenode
It is also possible that your question was already asked here, so please do a quick issues search before submitting. Lastly, if your question is about one of Redis' [clients](https://redis.io/clients), you may to contact your client's developers for help.
That said, please feel free to replace all this with your question :)
+2 -1
View File
@@ -30,6 +30,7 @@ redis-check-rdb
keydb-check-rdb
redis-check-dump
keydb-check-dump
keydb-diagnostic-tool
redis-cli
redis-sentinel
redis-server
@@ -58,4 +59,4 @@ Makefile.dep
.ccls
.ccls-cache/*
compile_commands.json
redis.code-workspace
keydb.code-workspace
+34
View File
@@ -11,6 +11,40 @@ CRITICAL: There is a critical bug affecting MOST USERS. Upgrade ASAP.
SECURITY: There are security fixes in the release.
--------------------------------------------------------------------------------
================================================================================
Redis 6.2.3 Released Mon May 3 19:00:00 IST 2021
================================================================================
Upgrade urgency: SECURITY, Contains fixes to security issues that affect
authenticated client connections. LOW otherwise.
Integer overflow in STRALGO LCS command (CVE-2021-29477):
An integer overflow bug in Redis version 6.0 or newer could be exploited using
the STRALGO LCS command to corrupt the heap and potentially result in remote
code execution. The integer overflow bug exists in all versions of Redis
starting with 6.0.
Integer overflow in COPY command for large intsets (CVE-2021-29478):
An integer overflow bug in Redis 6.2 could be exploited to corrupt the heap and
potentially result with remote code execution. The vulnerability involves
changing the default set-max-intset-entries configuration value, creating a
large set key that consists of integer values and using the COPY command to
duplicate it. The integer overflow bug exists in all versions of Redis starting
with 2.6, where it could result with a corrupted RDB or DUMP payload, but not
exploited through COPY (which did not exist before 6.2).
Bug fixes that are only applicable to previous releases of Redis 6.2:
* Fix memory leak in moduleDefragGlobals (#8853)
* Fix memory leak when doing lazy freeing client tracking table (#8822)
* Block abusive replicas from sending command that could assert and crash redis (#8868)
Other bug fixes:
* Use a monotonic clock to check for Lua script timeout (#8812)
* redis-cli: Do not use unix socket when we got redirected in cluster mode (#8870)
Modules:
* Fix RM_GetClusterNodeInfo() to correctly populate master id (#8846)
================================================================================
Redis 6.2.2 Released Mon April 19 19:00:00 IST 2021
================================================================================
+1 -1
View File
@@ -179,7 +179,7 @@ To compile against jemalloc on Mac OS X systems, use:
Monotonic clock
---------------
By default, Redis will build using the POSIX clock_gettime function as the
By default, KeyDB will build using the POSIX clock_gettime function as the
monotonic clock source. On most modern systems, the internal processor clock
can be used to improve performance. Cautions can be found here:
http://oliveryang.net/2015/09/pitfalls-of-TSC-usage/
+2 -2
View File
@@ -28,8 +28,8 @@ To manually run a Redis server with TLS mode (assuming `gen-test-certs.sh` was
invoked so sample certificates/keys are available):
./src/keydb-server --tls-port 6379 --port 0 \
--tls-cert-file ./tests/tls/keydb.crt \
--tls-key-file ./tests/tls/keydb.key \
--tls-cert-file ./tests/tls/client.crt \
--tls-key-file ./tests/tls/client.key \
--tls-ca-cert-file ./tests/tls/ca.crt
To connect to this Redis server with `keydb-cli`:
@@ -241,7 +241,7 @@ iget_defrag_hint(tsdn_t *tsdn, void* ptr) {
int free_in_slab = extent_nfree_get(slab);
if (free_in_slab) {
const bin_info_t *bin_info = &bin_infos[binind];
int curslabs = binshard->stats.curslabs;
ssize_t curslabs = binshard->stats.curslabs;
size_t curregs = binshard->stats.curregs;
if (binshard->slabcur) {
/* remove slabcur from the overall utilization */
+3 -3
View File
@@ -141,12 +141,12 @@ tcp-keepalive 300
# server to connected clients, masters or cluster peers. These files should be
# PEM formatted.
#
# tls-cert-file redis.crt
# tls-key-file redis.key
# tls-cert-file keydb.crt
# tls-key-file keydb.key
# Configure a DH parameters file to enable Diffie-Hellman (DH) key exchange:
#
# tls-dh-params-file redis.dh
# tls-dh-params-file keydb.dh
# Configure a CA certificate(s) bundle or directory to authenticate TLS/SSL
# clients and peers. Redis requires an explicit configuration of at least one
+331 -361
View File
File diff suppressed because it is too large. Load diff
+112 -15
View File
@@ -20,12 +20,12 @@
# The port that this sentinel instance will run on
port 26379
# By default Redis Sentinel does not run as a daemon. Use 'yes' if you need it.
# Note that Redis will write a pid file in /var/run/keydb-sentinel.pid when
# By default KeyDB Sentinel does not run as a daemon. Use 'yes' if you need it.
# Note that KeyDB will write a pid file in /var/run/keydb-sentinel.pid when
# daemonized.
daemonize yes
# When running daemonized, Redis Sentinel writes a pid file in
# When running daemonized, KeyDB Sentinel writes a pid file in
# /var/run/keydb-sentinel.pid by default. You can specify a custom pid file
# location here.
pidfile /var/run/sentinel/keydb-sentinel.pid
@@ -59,7 +59,7 @@ logfile /var/log/keydb/keydb-sentinel.log
# dir <working-directory>
# Every long running process should have a well-defined working directory.
# For Redis Sentinel to chdir to /tmp at startup is the simplest thing
# For KeyDB Sentinel to chdir to /tmp at startup is the simplest thing
# for the process to don't interfere with administrative tasks such as
# unmounting filesystems.
dir /var/lib/keydb
@@ -86,22 +86,34 @@ sentinel monitor mymaster 127.0.0.1 6379 2
# sentinel auth-pass <master-name> <password>
#
# Set the password to use to authenticate with the master and replicas.
# Useful if there is a password set in the Redis instances to monitor.
# Useful if there is a password set in the KeyDB instances to monitor.
#
# Note that the master password is also used for replicas, so it is not
# possible to set a different password in masters and replicas instances
# if you want to be able to monitor these instances with Sentinel.
#
# However you can have Redis instances without the authentication enabled
# mixed with Redis instances requiring the authentication (as long as the
# However you can have KeyDB instances without the authentication enabled
# mixed with KeyDB instances requiring the authentication (as long as the
# password set is the same for all the instances requiring the password) as
# the AUTH command will have no effect in Redis instances with authentication
# the AUTH command will have no effect in KeyDB instances with authentication
# switched off.
#
# Example:
#
# sentinel auth-pass mymaster MySUPER--secret-0123passw0rd
# sentinel auth-user <master-name> <username>
#
# This is useful in order to authenticate to instances having ACL capabilities,
# that is, running KeyDB 6.0 or greater. When just auth-pass is provided the
# Sentinel instance will authenticate to KeyDB using the old "AUTH <pass>"
# method. When also an username is provided, it will use "AUTH <user> <pass>".
# In the KeyDB servers side, the ACL to provide just minimal access to
# Sentinel instances, should be configured along the following lines:
#
# user sentinel-user >somepassword +client +subscribe +publish \
# +ping +info +multi +slaveof +config +client +exec on
# sentinel down-after-milliseconds <master-name> <milliseconds>
#
# Number of milliseconds the master (or any attached replica or sentinel) should
@@ -112,6 +124,73 @@ sentinel monitor mymaster 127.0.0.1 6379 2
# Default is 30 seconds.
sentinel down-after-milliseconds mymaster 30000
# IMPORTANT NOTE: starting with KeyDB 6.2 ACL capability is supported for
# Sentinel mode, please refer to the Redis website https://redis.io/topics/acl
# for more details.
# Sentinel's ACL users are defined in the following format:
#
# user <username> ... acl rules ...
#
# For example:
#
# user worker +@admin +@connection ~* on >ffa9203c493aa99
#
# For more information about ACL configuration please refer to the Redis
# website at https://redis.io/topics/acl and KeyDB server configuration
# template keydb.conf.
# ACL LOG
#
# The ACL Log tracks failed commands and authentication events associated
# with ACLs. The ACL Log is useful to troubleshoot failed commands blocked
# by ACLs. The ACL Log is stored in memory. You can reclaim memory with
# ACL LOG RESET. Define the maximum entry length of the ACL Log below.
acllog-max-len 128
# Using an external ACL file
#
# Instead of configuring users here in this file, it is possible to use
# a stand-alone file just listing users. The two methods cannot be mixed:
# if you configure users here and at the same time you activate the external
# ACL file, the server will refuse to start.
#
# The format of the external ACL user file is exactly the same as the
# format that is used inside keydb.conf to describe users.
#
# aclfile /etc/keydb/sentinel-users.acl
# requirepass <password>
#
# You can configure Sentinel itself to require a password, however when doing
# so Sentinel will try to authenticate with the same password to all the
# other Sentinels. So you need to configure all your Sentinels in a given
# group with the same "requirepass" password. Check the following documentation
# for more info: https://redis.io/topics/sentinel
#
# IMPORTANT NOTE: starting with KeyDB 6.2 "requirepass" is a compatibility
# layer on top of the ACL system. The option effect will be just setting
# the password for the default user. Clients will still authenticate using
# AUTH <password> as usually, or more explicitly with AUTH default <password>
# if they follow the new protocol: both will work.
#
# New config files are advised to use separate authentication control for
# incoming connections (via ACL), and for outgoing connections (via
# sentinel-user and sentinel-pass)
#
# The requirepass is not compatable with aclfile option and the ACL LOAD
# command, these will cause requirepass to be ignored.
# sentinel sentinel-user <username>
#
# You can configure Sentinel to authenticate with other Sentinels with specific
# user name.
# sentinel sentinel-pass <password>
#
# The password for Sentinel to authenticate with other Sentinels. If sentinel-user
# is not configured, Sentinel will use 'default' user with sentinel-pass to authenticate.
# sentinel parallel-syncs <master-name> <numreplicas>
#
# How many replicas we can reconfigure to point to the new replica simultaneously
@@ -172,7 +251,7 @@ sentinel failover-timeout mymaster 180000
# generated in the WARNING level (for instance -sdown, -odown, and so forth).
# This script should notify the system administrator via email, SMS, or any
# other messaging system, that there is something wrong with the monitored
# Redis systems.
# KeyDB systems.
#
# The script is called with just two arguments: the first is the event type
# and the second the event description.
@@ -182,7 +261,7 @@ sentinel failover-timeout mymaster 180000
#
# Example:
#
# sentinel notification-script mymaster /var/redis/notify.sh
# sentinel notification-script mymaster /var/keydb/notify.sh
# CLIENTS RECONFIGURATION SCRIPT
#
@@ -207,7 +286,7 @@ sentinel failover-timeout mymaster 180000
#
# Example:
#
# sentinel client-reconfig-script mymaster /var/redis/reconfig.sh
# sentinel client-reconfig-script mymaster /var/keydb/reconfig.sh
# SECURITY
#
@@ -218,11 +297,11 @@ sentinel failover-timeout mymaster 180000
sentinel deny-scripts-reconfig yes
# REDIS COMMANDS RENAMING
# KEYDB COMMANDS RENAMING
#
# Sometimes the Redis server has certain commands, that are needed for Sentinel
# Sometimes the KeyDB server has certain commands, that are needed for Sentinel
# to work correctly, renamed to unguessable strings. This is often the case
# of CONFIG and SLAVEOF in the context of providers that provide Redis as
# of CONFIG and SLAVEOF in the context of providers that provide KeyDB as
# a service, and don't want the customers to reconfigure the instances outside
# of the administration console.
#
@@ -239,6 +318,24 @@ sentinel deny-scripts-reconfig yes
# SENTINEL SET can also be used in order to perform this configuration at runtime.
#
# In order to set a command back to its original name (undo the renaming), it
# is possible to just rename a command to itsef:
# is possible to just rename a command to itself:
#
# SENTINEL rename-command mymaster CONFIG CONFIG
# HOSTNAMES SUPPORT
#
# Normally Sentinel uses only IP addresses and requires SENTINEL MONITOR
# to specify an IP address. Also, it requires the KeyDB replica-announce-ip
# keyword to specify only IP addresses.
#
# You may enable hostnames support by enabling resolve-hostnames. Note
# that you must make sure your DNS is configured properly and that DNS
# resolution does not introduce very long delays.
#
SENTINEL resolve-hostnames no
# When resolve-hostnames is enabled, Sentinel still uses IP addresses
# when exposing instances to users, configuration files, etc. If you want
# to retain the hostnames when announced, enable announce-hostnames below.
#
SENTINEL announce-hostnames no
File diff suppressed because it is too large. Load diff
@@ -20,20 +20,20 @@
# The port that this sentinel instance will run on
port 26379
# By default Redis Sentinel does not run as a daemon. Use 'yes' if you need it.
# Note that Redis will write a pid file in /var/run/redis-sentinel.pid when
# By default KeyDB Sentinel does not run as a daemon. Use 'yes' if you need it.
# Note that KeyDB will write a pid file in /var/run/keydb-sentinel.pid when
# daemonized.
daemonize no
# When running daemonized, Redis Sentinel writes a pid file in
# /var/run/redis-sentinel.pid by default. You can specify a custom pid file
# When running daemonized, KeyDB Sentinel writes a pid file in
# /var/run/keydb-sentinel.pid by default. You can specify a custom pid file
# location here.
pidfile /var/run/redis-sentinel.pid
pidfile /var/run/sentinel/keydb-sentinel.pid
# Specify the log file name. Also the empty string can be used to force
# Sentinel to log on the standard output. Note that if you use standard
# output for logging but daemonize, logs will be sent to /dev/null
logfile /var/log/redis/sentinel.log
logfile /var/log/keydb/keydb-sentinel.log
# sentinel announce-ip <ip>
# sentinel announce-port <port>
@@ -59,12 +59,12 @@ logfile /var/log/redis/sentinel.log
# dir <working-directory>
# Every long running process should have a well-defined working directory.
# For Redis Sentinel to chdir to /tmp at startup is the simplest thing
# For KeyDB Sentinel to chdir to /tmp at startup is the simplest thing
# for the process to don't interfere with administrative tasks such as
# unmounting filesystems.
dir /tmp
# sentinel monitor <master-name> <ip> <redis-port> <quorum>
# sentinel monitor <master-name> <ip> <keydb-port> <quorum>
#
# Tells Sentinel to monitor this master, and to consider it in O_DOWN
# (Objectively Down) state only if at least <quorum> sentinels agree.
@@ -86,22 +86,34 @@ sentinel monitor mymaster 127.0.0.1 6379 2
# sentinel auth-pass <master-name> <password>
#
# Set the password to use to authenticate with the master and replicas.
# Useful if there is a password set in the Redis instances to monitor.
# Useful if there is a password set in the KeyDB instances to monitor.
#
# Note that the master password is also used for replicas, so it is not
# possible to set a different password in masters and replicas instances
# if you want to be able to monitor these instances with Sentinel.
#
# However you can have Redis instances without the authentication enabled
# mixed with Redis instances requiring the authentication (as long as the
# However you can have KeyDB instances without the authentication enabled
# mixed with KeyDB instances requiring the authentication (as long as the
# password set is the same for all the instances requiring the password) as
# the AUTH command will have no effect in Redis instances with authentication
# the AUTH command will have no effect in KeyDB instances with authentication
# switched off.
#
# Example:
#
# sentinel auth-pass mymaster MySUPER--secret-0123passw0rd
# sentinel auth-user <master-name> <username>
#
# This is useful in order to authenticate to instances having ACL capabilities,
# that is, running KeyDB 6.0 or greater. When just auth-pass is provided the
# Sentinel instance will authenticate to KeyDB using the old "AUTH <pass>"
# method. When also an username is provided, it will use "AUTH <user> <pass>".
# In the KeyDB servers side, the ACL to provide just minimal access to
# Sentinel instances, should be configured along the following lines:
#
# user sentinel-user >somepassword +client +subscribe +publish \
# +ping +info +multi +slaveof +config +client +exec on
# sentinel down-after-milliseconds <master-name> <milliseconds>
#
# Number of milliseconds the master (or any attached replica or sentinel) should
@@ -112,6 +124,73 @@ sentinel monitor mymaster 127.0.0.1 6379 2
# Default is 30 seconds.
sentinel down-after-milliseconds mymaster 30000
# IMPORTANT NOTE: starting with KeyDB 6.2 ACL capability is supported for
# Sentinel mode, please refer to the Redis website https://redis.io/topics/acl
# for more details.
# Sentinel's ACL users are defined in the following format:
#
# user <username> ... acl rules ...
#
# For example:
#
# user worker +@admin +@connection ~* on >ffa9203c493aa99
#
# For more information about ACL configuration please refer to the Redis
# website at https://redis.io/topics/acl and KeyDB server configuration
# template keydb.conf.
# ACL LOG
#
# The ACL Log tracks failed commands and authentication events associated
# with ACLs. The ACL Log is useful to troubleshoot failed commands blocked
# by ACLs. The ACL Log is stored in memory. You can reclaim memory with
# ACL LOG RESET. Define the maximum entry length of the ACL Log below.
acllog-max-len 128
# Using an external ACL file
#
# Instead of configuring users here in this file, it is possible to use
# a stand-alone file just listing users. The two methods cannot be mixed:
# if you configure users here and at the same time you activate the external
# ACL file, the server will refuse to start.
#
# The format of the external ACL user file is exactly the same as the
# format that is used inside keydb.conf to describe users.
#
# aclfile /etc/keydb/sentinel-users.acl
# requirepass <password>
#
# You can configure Sentinel itself to require a password, however when doing
# so Sentinel will try to authenticate with the same password to all the
# other Sentinels. So you need to configure all your Sentinels in a given
# group with the same "requirepass" password. Check the following documentation
# for more info: https://redis.io/topics/sentinel
#
# IMPORTANT NOTE: starting with KeyDB 6.2 "requirepass" is a compatibility
# layer on top of the ACL system. The option effect will be just setting
# the password for the default user. Clients will still authenticate using
# AUTH <password> as usually, or more explicitly with AUTH default <password>
# if they follow the new protocol: both will work.
#
# New config files are advised to use separate authentication control for
# incoming connections (via ACL), and for outgoing connections (via
# sentinel-user and sentinel-pass)
#
# The requirepass is not compatable with aclfile option and the ACL LOAD
# command, these will cause requirepass to be ignored.
# sentinel sentinel-user <username>
#
# You can configure Sentinel to authenticate with other Sentinels with specific
# user name.
# sentinel sentinel-pass <password>
#
# The password for Sentinel to authenticate with other Sentinels. If sentinel-user
# is not configured, Sentinel will use 'default' user with sentinel-pass to authenticate.
# sentinel parallel-syncs <master-name> <numreplicas>
#
# How many replicas we can reconfigure to point to the new replica simultaneously
@@ -172,7 +251,7 @@ sentinel failover-timeout mymaster 180000
# generated in the WARNING level (for instance -sdown, -odown, and so forth).
# This script should notify the system administrator via email, SMS, or any
# other messaging system, that there is something wrong with the monitored
# Redis systems.
# KeyDB systems.
#
# The script is called with just two arguments: the first is the event type
# and the second the event description.
@@ -182,7 +261,7 @@ sentinel failover-timeout mymaster 180000
#
# Example:
#
# sentinel notification-script mymaster /var/redis/notify.sh
# sentinel notification-script mymaster /var/keydb/notify.sh
# CLIENTS RECONFIGURATION SCRIPT
#
@@ -207,7 +286,7 @@ sentinel failover-timeout mymaster 180000
#
# Example:
#
# sentinel client-reconfig-script mymaster /var/redis/reconfig.sh
# sentinel client-reconfig-script mymaster /var/keydb/reconfig.sh
# SECURITY
#
@@ -218,11 +297,11 @@ sentinel failover-timeout mymaster 180000
sentinel deny-scripts-reconfig yes
# REDIS COMMANDS RENAMING
# KEYDB COMMANDS RENAMING
#
# Sometimes the Redis server has certain commands, that are needed for Sentinel
# Sometimes the KeyDB server has certain commands, that are needed for Sentinel
# to work correctly, renamed to unguessable strings. This is often the case
# of CONFIG and SLAVEOF in the context of providers that provide Redis as
# of CONFIG and SLAVEOF in the context of providers that provide KeyDB as
# a service, and don't want the customers to reconfigure the instances outside
# of the administration console.
#
@@ -239,6 +318,24 @@ sentinel deny-scripts-reconfig yes
# SENTINEL SET can also be used in order to perform this configuration at runtime.
#
# In order to set a command back to its original name (undo the renaming), it
# is possible to just rename a command to itsef:
# is possible to just rename a command to itself:
#
# SENTINEL rename-command mymaster CONFIG CONFIG
# HOSTNAMES SUPPORT
#
# Normally Sentinel uses only IP addresses and requires SENTINEL MONITOR
# to specify an IP address. Also, it requires the KeyDB replica-announce-ip
# keyword to specify only IP addresses.
#
# You may enable hostnames support by enabling resolve-hostnames. Note
# that you must make sure your DNS is configured properly and that DNS
# resolution does not introduce very long delays.
#
SENTINEL resolve-hostnames no
# When resolve-hostnames is enabled, Sentinel still uses IP addresses
# when exposing instances to users, configuration files, etc. If you want
# to retain the hostnames when announced, enable announce-hostnames below.
#
SENTINEL announce-hostnames no
+1 -1
View File
@@ -10,7 +10,7 @@ done
if [ -z $TCLSH ]
then
echo "You need tcl 8.5 or newer in order to run the Redis test"
echo "You need tcl 8.5 or newer in order to run the KeyDB test"
exit 1
fi
$TCLSH tests/test_helper.tcl "${@}"
+1 -1
View File
@@ -8,7 +8,7 @@ done
if [ -z $TCLSH ]
then
echo "You need tcl 8.5 or newer in order to run the Redis Cluster test"
echo "You need tcl 8.5 or newer in order to run the KeyDB Cluster test"
exit 1
fi
$TCLSH tests/cluster/run.tcl $*
+1 -1
View File
@@ -9,7 +9,7 @@ done
if [ -z $TCLSH ]
then
echo "You need tcl 8.5 or newer in order to run the Redis ModuleApi test"
echo "You need tcl 8.5 or newer in order to run the KeyDB ModuleApi test"
exit 1
fi
+1 -1
View File
@@ -8,7 +8,7 @@ done
if [ -z $TCLSH ]
then
echo "You need tcl 8.5 or newer in order to run the Redis Sentinel test"
echo "You need tcl 8.5 or newer in order to run the KeyDB Sentinel test"
exit 1
fi
$TCLSH tests/sentinel/run.tcl $*
+24 -24
View File
@@ -20,12 +20,12 @@
# The port that this sentinel instance will run on
port 26379
# By default Redis Sentinel does not run as a daemon. Use 'yes' if you need it.
# Note that Redis will write a pid file in /var/run/keydb-sentinel.pid when
# By default KeyDB Sentinel does not run as a daemon. Use 'yes' if you need it.
# Note that KeyDB will write a pid file in /var/run/keydb-sentinel.pid when
# daemonized.
daemonize no
# When running daemonized, Redis Sentinel writes a pid file in
# When running daemonized, KeyDB Sentinel writes a pid file in
# /var/run/keydb-sentinel.pid by default. You can specify a custom pid file
# location here.
pidfile /var/run/keydb-sentinel.pid
@@ -59,7 +59,7 @@ logfile ""
# dir <working-directory>
# Every long running process should have a well-defined working directory.
# For Redis Sentinel to chdir to /tmp at startup is the simplest thing
# For KeyDB Sentinel to chdir to /tmp at startup is the simplest thing
# for the process to don't interfere with administrative tasks such as
# unmounting filesystems.
dir /tmp
@@ -86,16 +86,16 @@ sentinel monitor mymaster 127.0.0.1 6379 2
# sentinel auth-pass <master-name> <password>
#
# Set the password to use to authenticate with the master and replicas.
# Useful if there is a password set in the Redis instances to monitor.
# Useful if there is a password set in the KeyDB instances to monitor.
#
# Note that the master password is also used for replicas, so it is not
# possible to set a different password in masters and replicas instances
# if you want to be able to monitor these instances with Sentinel.
#
# However you can have Redis instances without the authentication enabled
# mixed with Redis instances requiring the authentication (as long as the
# However you can have KeyDB instances without the authentication enabled
# mixed with KeyDB instances requiring the authentication (as long as the
# password set is the same for all the instances requiring the password) as
# the AUTH command will have no effect in Redis instances with authentication
# the AUTH command will have no effect in KeyDB instances with authentication
# switched off.
#
# Example:
@@ -105,10 +105,10 @@ sentinel monitor mymaster 127.0.0.1 6379 2
# sentinel auth-user <master-name> <username>
#
# This is useful in order to authenticate to instances having ACL capabilities,
# that is, running Redis 6.0 or greater. When just auth-pass is provided the
# Sentinel instance will authenticate to Redis using the old "AUTH <pass>"
# that is, running KeyDB 6.0 or greater. When just auth-pass is provided the
# Sentinel instance will authenticate to KeyDB using the old "AUTH <pass>"
# method. When also an username is provided, it will use "AUTH <user> <pass>".
# In the Redis servers side, the ACL to provide just minimal access to
# In the KeyDB servers side, the ACL to provide just minimal access to
# Sentinel instances, should be configured along the following lines:
#
# user sentinel-user >somepassword +client +subscribe +publish \
@@ -125,7 +125,7 @@ sentinel monitor mymaster 127.0.0.1 6379 2
sentinel down-after-milliseconds mymaster 30000
# IMPORTANT NOTE: starting with KeyDB 6.2 ACL capability is supported for
# Sentinel mode, please refer to the KeyDB website https://redis.io/topics/acl
# Sentinel mode, please refer to the Redis website https://redis.io/topics/acl
# for more details.
# Sentinel's ACL users are defined in the following format:
@@ -137,8 +137,8 @@ sentinel down-after-milliseconds mymaster 30000
# user worker +@admin +@connection ~* on >ffa9203c493aa99
#
# For more information about ACL configuration please refer to the Redis
# website at https://redis.io/topics/acl and redis server configuration
# template redis.conf.
# website at https://redis.io/topics/acl and KeyDB server configuration
# template keydb.conf.
# ACL LOG
#
@@ -156,9 +156,9 @@ acllog-max-len 128
# ACL file, the server will refuse to start.
#
# The format of the external ACL user file is exactly the same as the
# format that is used inside redis.conf to describe users.
# format that is used inside keydb.conf to describe users.
#
# aclfile /etc/redis/sentinel-users.acl
# aclfile /etc/keydb/sentinel-users.acl
# requirepass <password>
#
@@ -168,7 +168,7 @@ acllog-max-len 128
# group with the same "requirepass" password. Check the following documentation
# for more info: https://redis.io/topics/sentinel
#
# IMPORTANT NOTE: starting with Redis 6.2 "requirepass" is a compatibility
# IMPORTANT NOTE: starting with KeyDB 6.2 "requirepass" is a compatibility
# layer on top of the ACL system. The option effect will be just setting
# the password for the default user. Clients will still authenticate using
# AUTH <password> as usually, or more explicitly with AUTH default <password>
@@ -251,7 +251,7 @@ sentinel failover-timeout mymaster 180000
# generated in the WARNING level (for instance -sdown, -odown, and so forth).
# This script should notify the system administrator via email, SMS, or any
# other messaging system, that there is something wrong with the monitored
# Redis systems.
# KeyDB systems.
#
# The script is called with just two arguments: the first is the event type
# and the second the event description.
@@ -261,7 +261,7 @@ sentinel failover-timeout mymaster 180000
#
# Example:
#
# sentinel notification-script mymaster /var/redis/notify.sh
# sentinel notification-script mymaster /var/keydb/notify.sh
# CLIENTS RECONFIGURATION SCRIPT
#
@@ -286,7 +286,7 @@ sentinel failover-timeout mymaster 180000
#
# Example:
#
# sentinel client-reconfig-script mymaster /var/redis/reconfig.sh
# sentinel client-reconfig-script mymaster /var/keydb/reconfig.sh
# SECURITY
#
@@ -297,11 +297,11 @@ sentinel failover-timeout mymaster 180000
sentinel deny-scripts-reconfig yes
# REDIS COMMANDS RENAMING
# KEYDB COMMANDS RENAMING
#
# Sometimes the Redis server has certain commands, that are needed for Sentinel
# Sometimes the KeyDB server has certain commands, that are needed for Sentinel
# to work correctly, renamed to unguessable strings. This is often the case
# of CONFIG and SLAVEOF in the context of providers that provide Redis as
# of CONFIG and SLAVEOF in the context of providers that provide KeyDB as
# a service, and don't want the customers to reconfigure the instances outside
# of the administration console.
#
@@ -325,7 +325,7 @@ sentinel deny-scripts-reconfig yes
# HOSTNAMES SUPPORT
#
# Normally Sentinel uses only IP addresses and requires SENTINEL MONITOR
# to specify an IP address. Also, it requires the Redis replica-announce-ip
# to specify an IP address. Also, it requires the KeyDB replica-announce-ip
# keyword to specify only IP addresses.
#
# You may enable hostnames support by enabling resolve-hostnames. Note
+36 -18
View File
@@ -3,11 +3,11 @@
# This file is released under the BSD license, see the COPYING file
#
# The Makefile composes the final FINAL_CFLAGS and FINAL_LDFLAGS using
# what is needed for Redis plus the standard CFLAGS and LDFLAGS passed.
# what is needed for KeyDB plus the standard CFLAGS and LDFLAGS passed.
# However when building the dependencies (Jemalloc, Lua, Hiredis, ...)
# CFLAGS and LDFLAGS are propagated to the dependencies, so to pass
# flags only to be used when compiling / linking Redis itself REDIS_CFLAGS
# and REDIS_LDFLAGS are used instead (this is the case of 'make gcov').
# flags only to be used when compiling / linking KeyDB itself KEYDB_CFLAGS
# and KEYDB_LDFLAGS are used instead (this is the case of 'make gcov').
#
# Dependencies are stored in the Makefile.dep file. To rebuild this file
# Just use 'make dep', but this is only needed by developers.
@@ -29,7 +29,7 @@ ifneq (,$(findstring FreeBSD,$(uname_S)))
STD+=-Wno-c11-extensions
endif
endif
WARN=-Wall -W -Wno-missing-field-initializers
WARN=-Wall -W -Wno-missing-field-initializers -Wno-address-of-packed-member -Wno-atomic-alignment
OPT=$(OPTIMIZATION)
# Detect if the compiler supports C11 _Atomic
@@ -85,7 +85,7 @@ ifeq ($(COMPILER_NAME),clang)
LDFLAGS+= -latomic
endif
# To get ARM stack traces if Redis crashes we need a special C flag.
# To get ARM stack traces if KeyDB crashes we need a special C flag.
ifneq (,$(filter aarch64 armv,$(uname_M)))
CFLAGS+=-funwind-tables
CXXFLAGS+=-funwind-tables
@@ -116,15 +116,19 @@ endif
# Override default settings if possible
-include .make-settings
FINAL_CFLAGS=$(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS) $(REDIS_CFLAGS)
FINAL_CXXFLAGS=$(CXX_STD) $(WARN) $(OPT) $(DEBUG) $(CXXFLAGS) $(REDIS_CFLAGS)
FINAL_LDFLAGS=$(LDFLAGS) $(REDIS_LDFLAGS) $(DEBUG)
FINAL_CFLAGS=$(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS) $(KEYDB_CFLAGS) $(REDIS_CFLAGS)
FINAL_CXXFLAGS=$(CXX_STD) $(WARN) $(OPT) $(DEBUG) $(CXXFLAGS) $(KEYDB_CFLAGS) $(REDIS_CFLAGS)
FINAL_LDFLAGS=$(LDFLAGS) $(KEYDB_LDFLAGS) $(DEBUG)
FINAL_LIBS+=-lm
DEBUG=-g -ggdb
ifneq ($(uname_S),Darwin)
FINAL_LIBS+=-latomic
endif
# Linux ARM32 needs -latomic at linking time
ifneq (,$(findstring armv,$(uname_M)))
FINAL_LIBS+=-latomic
endif
ifeq ($(uname_S),SunOS)
@@ -173,7 +177,7 @@ ifeq ($(uname_S),OpenBSD)
FINAL_CXXFLAGS+= -DUSE_BACKTRACE -I/usr/local/include
FINAL_LDFLAGS+= -L/usr/local/lib
FINAL_LIBS+= -lexecinfo
endif
endif
else
ifeq ($(uname_S),NetBSD)
@@ -183,7 +187,7 @@ ifeq ($(uname_S),NetBSD)
FINAL_CFLAGS+= -DUSE_BACKTRACE -I/usr/pkg/include
FINAL_LDFLAGS+= -L/usr/pkg/lib
FINAL_LIBS+= -lexecinfo
endif
endif
else
ifeq ($(uname_S),FreeBSD)
# FreeBSD
@@ -312,6 +316,14 @@ else
endef
endif
# Alpine OS doesn't have support for the execinfo backtrace library we use for debug, so we provide an alternate implementation using libwunwind.
OS := $(shell cat /etc/os-release | grep ID= | head -n 1 | cut -d'=' -f2)
ifeq ($(OS),alpine)
FINAL_CXXFLAGS+=-DUNW_LOCAL_ONLY
FINAL_LIBS += -lunwind
endif
REDIS_CC=$(QUIET_CC)$(CC) $(FINAL_CFLAGS)
REDIS_CXX=$(QUIET_CC)$(CXX) $(FINAL_CXXFLAGS)
KEYDB_AS=$(QUIET_CC) as --64 -g
@@ -340,8 +352,10 @@ REDIS_BENCHMARK_NAME=keydb-benchmark$(PROG_SUFFIX)
REDIS_BENCHMARK_OBJ=ae.o anet.o redis-benchmark.o adlist.o dict.o zmalloc.o release.o crcspeed.o crc64.o siphash.o redis-benchmark.o storage-lite.o fastlock.o new.o monotonic.o cli_common.o mt19937-64.o $(ASM_OBJ)
REDIS_CHECK_RDB_NAME=keydb-check-rdb$(PROG_SUFFIX)
REDIS_CHECK_AOF_NAME=keydb-check-aof$(PROG_SUFFIX)
KEYDB_DIAGNOSTIC_NAME=keydb-diagnostic-tool$(PROG_SUFFIX)
KEYDB_DIAGNOSTIC_OBJ=ae.o anet.o keydb-diagnostic-tool.o adlist.o dict.o zmalloc.o release.o crcspeed.o crc64.o siphash.o keydb-diagnostic-tool.o storage-lite.o fastlock.o new.o monotonic.o cli_common.o mt19937-64.o $(ASM_OBJ)
all: $(REDIS_SERVER_NAME) $(REDIS_SENTINEL_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCHMARK_NAME) $(REDIS_CHECK_RDB_NAME) $(REDIS_CHECK_AOF_NAME)
all: $(REDIS_SERVER_NAME) $(REDIS_SENTINEL_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCHMARK_NAME) $(REDIS_CHECK_RDB_NAME) $(REDIS_CHECK_AOF_NAME) $(KEYDB_DIAGNOSTIC_NAME)
@echo ""
@echo "Hint: It's a good idea to run 'make test' ;)"
@echo ""
@@ -365,9 +379,9 @@ persist-settings: distclean
echo CFLAGS=$(CFLAGS) >> .make-settings
echo CXXFLAGS=$(CXXFLAGS) >> .make-settings
echo LDFLAGS=$(LDFLAGS) >> .make-settings
echo REDIS_CFLAGS=$(REDIS_CFLAGS) >> .make-settings
echo REDIS_CXXFLAGS=$(REDIS_CXXFLAGS) >> .make-settings
echo REDIS_LDFLAGS=$(REDIS_LDFLAGS) >> .make-settings
echo KEYDB_CFLAGS=$(KEYDB_CFLAGS) >> .make-settings
echo KEYDB_CXXFLAGS=$(KEYDB_CXXFLAGS) >> .make-settings
echo KEYDB_LDFLAGS=$(KEYDB_LDFLAGS) >> .make-settings
echo PREV_FINAL_CFLAGS=$(FINAL_CFLAGS) >> .make-settings
echo PREV_FINAL_CXXFLAGS=$(FINAL_CXXFLAGS) >> .make-settings
echo PREV_FINAL_LDFLAGS=$(FINAL_LDFLAGS) >> .make-settings
@@ -413,6 +427,10 @@ $(REDIS_CLI_NAME): $(REDIS_CLI_OBJ)
$(REDIS_BENCHMARK_NAME): $(REDIS_BENCHMARK_OBJ)
$(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a ../deps/hdr_histogram/hdr_histogram.o $(FINAL_LIBS)
# keydb-diagnostic-tool
$(KEYDB_DIAGNOSTIC_NAME): $(KEYDB_DIAGNOSTIC_OBJ)
$(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a $(FINAL_LIBS)
DEP = $(REDIS_SERVER_OBJ:%.o=%.d) $(REDIS_CLI_OBJ:%.o=%.d) $(REDIS_BENCHMARK_OBJ:%.o=%.d)
-include $(DEP)
@@ -429,7 +447,7 @@ DEP = $(REDIS_SERVER_OBJ:%.o=%.d) $(REDIS_CLI_OBJ:%.o=%.d) $(REDIS_BENCHMARK_OBJ
$(KEYDB_AS) $< -o $@
clean:
rm -rf $(REDIS_SERVER_NAME) $(REDIS_SENTINEL_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCHMARK_NAME) $(REDIS_CHECK_RDB_NAME) $(REDIS_CHECK_AOF_NAME) *.o *.gcda *.gcno *.gcov KeyDB.info lcov-html Makefile.dep
rm -rf $(REDIS_SERVER_NAME) $(REDIS_SENTINEL_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCHMARK_NAME) $(REDIS_CHECK_RDB_NAME) $(REDIS_CHECK_AOF_NAME) $(KEYDB_DIAGNOSTIC_NAME) *.o *.gcda *.gcno *.gcov KeyDB.info lcov-html Makefile.dep
rm -f $(DEP)
.PHONY: clean
@@ -468,7 +486,7 @@ bench: $(REDIS_BENCHMARK_NAME)
$(MAKE) CXXFLAGS="-m32" CFLAGS="-m32" LDFLAGS="-m32"
gcov:
$(MAKE) REDIS_CXXFLAGS="-fprofile-arcs -ftest-coverage -DCOVERAGE_TEST" REDIS_CFLAGS="-fprofile-arcs -ftest-coverage -DCOVERAGE_TEST" REDIS_LDFLAGS="-fprofile-arcs -ftest-coverage"
$(MAKE) KEYDB_CXXFLAGS="-fprofile-arcs -ftest-coverage -DCOVERAGE_TEST" KEYDB_CFLAGS="-fprofile-arcs -ftest-coverage -DCOVERAGE_TEST" KEYDB_LDFLAGS="-fprofile-arcs -ftest-coverage"
noopt:
$(MAKE) OPTIMIZATION="-O0"
@@ -477,7 +495,7 @@ valgrind:
$(MAKE) OPTIMIZATION="-O0" USEASM="false" MALLOC="libc" CFLAGS="-DSANITIZE" CXXFLAGS="-DSANITIZE"
helgrind:
$(MAKE) OPTIMIZATION="-O0" MALLOC="libc" CFLAGS="-D__ATOMIC_VAR_FORCE_SYNC_MACROS" REDIS_CFLAGS="-I/usr/local/include" REDIS_LDFLAGS="-L/usr/local/lib"
$(MAKE) OPTIMIZATION="-O0" MALLOC="libc" CFLAGS="-D__ATOMIC_VAR_FORCE_SYNC_MACROS" KEYDB_CFLAGS="-I/usr/local/include" KEYDB_LDFLAGS="-L/usr/local/lib"
src/help.h:
@../utils/generate-command-help.rb > help.h
@@ -492,4 +510,4 @@ install: all
@ln -sf $(REDIS_SERVER_NAME) $(INSTALL_BIN)/$(REDIS_SENTINEL_NAME)
uninstall:
rm -f $(INSTALL_BIN)/{$(REDIS_SERVER_NAME),$(REDIS_BENCHMARK_NAME),$(REDIS_CLI_NAME),$(REDIS_CHECK_RDB_NAME),$(REDIS_CHECK_AOF_NAME),$(REDIS_SENTINEL_NAME)}
rm -f $(INSTALL_BIN)/{$(REDIS_SERVER_NAME),$(REDIS_BENCHMARK_NAME),$(REDIS_CLI_NAME),$(REDIS_CHECK_RDB_NAME),$(REDIS_CHECK_AOF_NAME),$(REDIS_SENTINEL_NAME),$(KEYDB_DIAGNOSTIC_NAME)}
+2 -3
View File
@@ -173,12 +173,11 @@ void queueClientForReprocessing(client *c) {
/* The client may already be into the unblocked list because of a previous
* blocking operation, don't add back it into the list multiple times. */
serverAssert(GlobalLocksAcquired());
fastlock_lock(&c->lock);
std::unique_lock<fastlock> ul(c->lock);
if (!(c->flags & CLIENT_UNBLOCKED)) {
c->flags |= CLIENT_UNBLOCKED;
listAddNodeTail(g_pserver->rgthreadvar[c->iel].unblocked_clients,c);
}
fastlock_unlock(&c->lock);
}
/* Unblock a client calling the right function depending on the kind
@@ -792,4 +791,4 @@ void signalKeyAsReady(redisDb *db, sds key, int type) {
redisObjectStack o;
initStaticStringObject(o, key);
signalKeyAsReady(db, &o, type);
}
}
+3 -2
View File
@@ -561,7 +561,7 @@ void clusterInit(void) {
serverAssert(serverTL == &g_pserver->rgthreadvar[IDX_EVENT_LOOP_MAIN]);
if (createSocketAcceptHandler(&g_pserver->cfd, clusterAcceptHandler) != C_OK) {
serverPanic("Unrecoverable error creating Redis Cluster socket accept handler.");
serverPanic("Unrecoverable error creating KeyDB Cluster socket accept handler.");
}
/* The slots -> keys map is a radix tree. Initialize it here. */
@@ -5567,9 +5567,10 @@ try_again:
if (ttl < 1) ttl = 1;
}
/* Relocate valid (non expired) keys into the array in successive
/* Relocate valid (non expired) keys and values into the array in successive
* positions to remove holes created by the keys that were present
* in the first lookup but are now expired after the second lookup. */
ov[non_expired] = ov[j];
kv[non_expired++] = kv[j];
serverAssertWithInfo(c,NULL,
+3
View File
@@ -437,6 +437,9 @@ void connSetThreadAffinity(connection *conn, int cpu) {
{
serverLog(LL_WARNING, "Failed to set socket affinity");
}
#else
(void)conn;
(void)cpu;
#endif
}
+10 -8
View File
@@ -1403,9 +1403,8 @@ void copyCommand(client *c) {
}
dbAdd(dst,newkey,newobj);
if (expire != nullptr) {
if (expire != nullptr) setExpire(c, dst, newkey, expire->duplicate());
}
if (expire != nullptr)
setExpire(c, dst, newkey, expire->duplicate());
/* OK! key copied */
signalModifiedKey(c,dst,c->argv[2]);
@@ -1771,7 +1770,7 @@ int keyIsExpired(redisDb *db, robj *key) {
* script execution, making propagation to slaves / AOF consistent.
* See issue #1525 on Github for more information. */
if (g_pserver->lua_caller) {
now = g_pserver->lua_time_start;
now = g_pserver->lua_time_snapshot;
}
/* If we are in the middle of a command execution, we still want to use
* a reference time that does not change: in that case we just use the
@@ -1832,14 +1831,17 @@ int expireIfNeeded(redisDb *db, robj *key) {
if (checkClientPauseTimeoutAndReturnIfPaused()) return 1;
/* Delete the key */
if (g_pserver->lazyfree_lazy_expire) {
dbAsyncDelete(db,key);
} else {
dbSyncDelete(db,key);
}
g_pserver->stat_expiredkeys++;
propagateExpire(db,key,g_pserver->lazyfree_lazy_expire);
notifyKeyspaceEvent(NOTIFY_EXPIRED,
"expired",key,db->id);
int retval = g_pserver->lazyfree_lazy_expire ? dbAsyncDelete(db,key) :
dbSyncDelete(db,key);
if (retval) signalModifiedKey(NULL,db,key);
return retval;
signalModifiedKey(NULL,db,key);
return 1;
}
/* -----------------------------------------------------------------------------
+72 -5
View File
@@ -51,6 +51,12 @@ typedef ucontext_t sigcontext_t;
#include <cxxabi.h>
#endif /* HAVE_BACKTRACE */
//UNW_LOCAL_ONLY being set means we use libunwind for backtraces instead of execinfo
#ifdef UNW_LOCAL_ONLY
#include <libunwind.h>
#include <cxxabi.h>
#endif
#ifdef __CYGWIN__
#ifndef SA_ONSTACK
#define SA_ONSTACK 0x08000000
@@ -947,7 +953,7 @@ void _serverAssert(const char *estr, const char *file, int line) {
serverLog(LL_WARNING,"==> %s:%d '%s' is not true",file,line,estr);
if (g_pserver->crashlog_enabled) {
#ifdef HAVE_BACKTRACE
#if defined HAVE_BACKTRACE || defined UNW_LOCAL_ONLY
logStackTrace(NULL, 1);
#endif
printCrashReport();
@@ -1038,14 +1044,13 @@ void _serverPanic(const char *file, int line, const char *msg, ...) {
vsnprintf(fmtmsg,sizeof(fmtmsg),msg,ap);
va_end(ap);
g_fInCrash = true;
bugReportStart();
serverLog(LL_WARNING,"------------------------------------------------");
serverLog(LL_WARNING,"!!! Software Failure. Press left mouse button to continue");
serverLog(LL_WARNING,"Guru Meditation: %s #%s:%d",fmtmsg,file,line);
if (g_pserver->crashlog_enabled) {
#ifdef HAVE_BACKTRACE
#if defined HAVE_BACKTRACE || defined UNW_LOCAL_ONLY
logStackTrace(NULL, 1);
#endif
printCrashReport();
@@ -1600,6 +1605,65 @@ void safe_write(int fd, const void *pv, ssize_t cb)
} while (offset < cb);
}
#ifdef UNW_LOCAL_ONLY
/* Logs the stack trace using the libunwind call.
* The eip argument is unused as libunwind only gets local context.
* The uplevel argument indicates how many of the calling functions to skip.
*/
void logStackTrace(void * eip, int uplevel) {
(void)eip;//UNUSED
const char *msg;
int fd = openDirectLogFiledes();
if (fd == -1) return; /* If we can't log there is anything to do. */
msg = "\n------ STACK TRACE ------\n";
if (write(fd,msg,strlen(msg)) == -1) {/* Avoid warning. */};
unw_cursor_t cursor;
unw_context_t context;
unw_getcontext(&context);
unw_init_local(&cursor, &context);
/* Write symbols to log file */
msg = "\nBacktrace:\n";
if (write(fd,msg,strlen(msg)) == -1) {/* Avoid warning. */};
for (int i = 0; i < uplevel; i++) {
unw_step(&cursor);
}
while ( unw_step(&cursor) ) {
unw_word_t ip, sp, off;
unw_get_reg(&cursor, UNW_REG_IP, &ip);
unw_get_reg(&cursor, UNW_REG_SP, &sp);
char symbol[256] = {"<unknown>"};
char *name = symbol;
if ( !unw_get_proc_name(&cursor, symbol, sizeof(symbol), &off) ) {
int status;
if ( (name = abi::__cxa_demangle(symbol, NULL, NULL, &status)) == 0 )
name = symbol;
}
dprintf(fd, "%s(+0x%" PRIxPTR ") [0x%016" PRIxPTR "] sp=0x%016" PRIxPTR "\n",
name,
static_cast<uintptr_t>(off),
static_cast<uintptr_t>(ip),
static_cast<uintptr_t>(sp));
if ( name != symbol )
free(name);
}
}
#endif /* UNW_LOCAL_ONLY */
#ifdef HAVE_BACKTRACE
void backtrace_symbols_demangle_fd(void **trace, size_t csym, int fd)
{
char **syms = backtrace_symbols(trace, csym);
@@ -1643,8 +1707,6 @@ void backtrace_symbols_demangle_fd(void **trace, size_t csym, int fd)
free(syms);
}
#ifdef HAVE_BACKTRACE
/* Logs the stack trace using the backtrace() call. This function is designed
* to be called from signal handlers safely.
* The eip argument is optional (can take NULL).
@@ -1935,6 +1997,9 @@ void sigsegvHandler(int sig, siginfo_t *info, void *secret) {
logRegisters(uc);
#endif
#ifdef UNW_LOCAL_ONLY
logStackTrace(NULL, 1);
#endif
printCrashReport();
@@ -2029,6 +2094,8 @@ void watchdogSignalHandler(int sig, siginfo_t *info, void *secret) {
serverLogFromHandler(LL_WARNING,"\n--- WATCHDOG TIMER EXPIRED ---");
#ifdef HAVE_BACKTRACE
logStackTrace(getMcontextEip(uc), 1);
#elif defined UNW_LOCAL_ONLY
logStackTrace(NULL, 1);
#else
serverLogFromHandler(LL_WARNING,"Sorry: no support for backtrace().");
#endif
+1 -1
View File
@@ -281,7 +281,7 @@ uint32_t intsetLen(const intset *is) {
/* Return intset blob size in bytes. */
size_t intsetBlobLen(intset *is) {
return sizeof(intset)+intrev32ifbe(is->length)*intrev32ifbe(is->encoding);
return sizeof(intset)+(size_t)intrev32ifbe(is->length)*intrev32ifbe(is->encoding);
}
/* Validate the integrity of the data structure.
+967
View File
@@ -0,0 +1,967 @@
/* KeyDB diagnostic utility.
*
* Copyright (c) 2009-2021, Salvatore Sanfilippo <antirez at gmail dot com>
* Copyright (c) 2021, EQ Alpha Technology Ltd. <john at eqalpha dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "fmacros.h"
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <time.h>
#include <sys/resource.h>
#include <sys/time.h>
#include <signal.h>
#include <assert.h>
#include <math.h>
#include <pthread.h>
#include <deque>
extern "C" {
#include <sds.h> /* Use hiredis sds. */
#include <sdscompat.h>
#include "hiredis.h"
}
#include "ae.h"
#include "adlist.h"
#include "dict.h"
#include "zmalloc.h"
#include "storage.h"
#include "atomicvar.h"
#include "crc16_slottable.h"
#define UNUSED(V) ((void) V)
#define RANDPTR_INITIAL_SIZE 8
#define MAX_LATENCY_PRECISION 3
#define MAX_THREADS 500
#define CLUSTER_SLOTS 16384
#define CLIENT_GET_EVENTLOOP(c) \
(c->thread_id >= 0 ? config.threads[c->thread_id]->el : config.el)
struct benchmarkThread;
struct clusterNode;
struct redisConfig;
int g_fTestMode = false;
static struct config {
aeEventLoop *el;
const char *hostip;
int hostport;
const char *hostsocket;
int numclients;
int liveclients;
int period_ms;
int requests;
int requests_issued;
int requests_finished;
int keysize;
int datasize;
int randomkeys;
int randomkeys_keyspacelen;
int keepalive;
int pipeline;
int showerrors;
long long start;
long long totlatency;
long long *latency;
const char *title;
list *clients;
int quiet;
int csv;
int loop;
int idlemode;
int dbnum;
sds dbnumstr;
char *tests;
char *auth;
const char *user;
int precision;
int max_threads;
struct benchmarkThread **threads;
int cluster_mode;
int cluster_node_count;
struct clusterNode **cluster_nodes;
struct redisConfig *redis_config;
int is_fetching_slots;
int is_updating_slots;
int slots_last_update;
int enable_tracking;
/* Thread mutexes to be used as fallbacks by atomicvar.h */
pthread_mutex_t requests_issued_mutex;
pthread_mutex_t requests_finished_mutex;
pthread_mutex_t liveclients_mutex;
pthread_mutex_t is_fetching_slots_mutex;
pthread_mutex_t is_updating_slots_mutex;
pthread_mutex_t updating_slots_mutex;
pthread_mutex_t slots_last_update_mutex;
} config;
typedef struct _client {
redisContext *context;
sds obuf;
char **randptr; /* Pointers to :rand: strings inside the command buf */
size_t randlen; /* Number of pointers in client->randptr */
size_t randfree; /* Number of unused pointers in client->randptr */
char **stagptr; /* Pointers to slot hashtags (cluster mode only) */
size_t staglen; /* Number of pointers in client->stagptr */
size_t stagfree; /* Number of unused pointers in client->stagptr */
size_t written; /* Bytes of 'obuf' already written */
long long start; /* Start time of a request */
long long latency; /* Request latency */
int pending; /* Number of pending requests (replies to consume) */
int prefix_pending; /* If non-zero, number of pending prefix commands. Commands
such as auth and select are prefixed to the pipeline of
benchmark commands and discarded after the first send. */
int prefixlen; /* Size in bytes of the pending prefix commands */
int thread_id;
struct clusterNode *cluster_node;
int slots_last_update;
redisReply *lastReply;
} *client;
/* Threads. */
typedef struct benchmarkThread {
int index;
pthread_t thread;
aeEventLoop *el;
} benchmarkThread;
/* Cluster. */
typedef struct clusterNode {
char *ip;
int port;
sds name;
int flags;
sds replicate; /* Master ID if node is a replica */
int *slots;
int slots_count;
int current_slot_index;
int *updated_slots; /* Used by updateClusterSlotsConfiguration */
int updated_slots_count; /* Used by updateClusterSlotsConfiguration */
int replicas_count;
sds *migrating; /* An array of sds where even strings are slots and odd
* strings are the destination node IDs. */
sds *importing; /* An array of sds where even strings are slots and odd
* strings are the source node IDs. */
int migrating_count; /* Length of the migrating array (migrating slots*2) */
int importing_count; /* Length of the importing array (importing slots*2) */
struct redisConfig *redis_config;
} clusterNode;
typedef struct redisConfig {
sds save;
sds appendonly;
} redisConfig;
int g_fInCrash = false;
/* Prototypes */
static void writeHandler(aeEventLoop *el, int fd, void *privdata, int mask);
static benchmarkThread *createBenchmarkThread(int index);
static void freeBenchmarkThread(benchmarkThread *thread);
static void freeBenchmarkThreads();
static redisContext *getRedisContext(const char *ip, int port,
const char *hostsocket);
/* Implementation */
static long long ustime(void) {
struct timeval tv;
long long ust;
gettimeofday(&tv, NULL);
ust = ((long)tv.tv_sec)*1000000;
ust += tv.tv_usec;
return ust;
}
/* _serverAssert is needed by dict */
extern "C" void _serverAssert(const char *estr, const char *file, int line) {
fprintf(stderr, "=== ASSERTION FAILED ===");
fprintf(stderr, "==> %s:%d '%s' is not true",file,line,estr);
*((char*)-1) = 'x';
}
static redisContext *getRedisContext(const char *ip, int port,
const char *hostsocket)
{
redisContext *ctx = NULL;
redisReply *reply = NULL;
if (hostsocket == NULL)
ctx = redisConnect(ip, port);
else
ctx = redisConnectUnix(hostsocket);
if (ctx == NULL || ctx->err) {
fprintf(stderr,"Could not connect to Redis at ");
const char *err = (ctx != NULL ? ctx->errstr : "");
if (hostsocket == NULL)
fprintf(stderr,"%s:%d: %s\n",ip,port,err);
else
fprintf(stderr,"%s: %s\n",hostsocket,err);
goto cleanup;
}
if (config.auth == NULL)
return ctx;
if (config.user == NULL)
reply = (redisReply*)redisCommand(ctx,"AUTH %s", config.auth);
else
reply = (redisReply*)redisCommand(ctx,"AUTH %s %s", config.user, config.auth);
if (reply != NULL) {
if (reply->type == REDIS_REPLY_ERROR) {
if (hostsocket == NULL)
fprintf(stderr, "Node %s:%d replied with error:\n%s\n", ip, port, reply->str);
else
fprintf(stderr, "Node %s replied with error:\n%s\n", hostsocket, reply->str);
goto cleanup;
}
freeReplyObject(reply);
return ctx;
}
fprintf(stderr, "ERROR: failed to fetch reply from ");
if (hostsocket == NULL)
fprintf(stderr, "%s:%d\n", ip, port);
else
fprintf(stderr, "%s\n", hostsocket);
cleanup:
freeReplyObject(reply);
redisFree(ctx);
return NULL;
}
static void freeClient(client c) {
aeEventLoop *el = CLIENT_GET_EVENTLOOP(c);
listNode *ln;
aeDeleteFileEvent(el,c->context->fd,AE_WRITABLE);
aeDeleteFileEvent(el,c->context->fd,AE_READABLE);
if (c->thread_id >= 0) {
int requests_finished = 0;
atomicGet(config.requests_finished, requests_finished);
if (requests_finished >= config.requests) {
aeStop(el);
}
}
redisFree(c->context);
sdsfree(c->obuf);
zfree(c->randptr);
zfree(c->stagptr);
zfree(c);
if (config.max_threads) pthread_mutex_lock(&(config.liveclients_mutex));
config.liveclients--;
ln = listSearchKey(config.clients,c);
assert(ln != NULL);
listDelNode(config.clients,ln);
if (config.max_threads) pthread_mutex_unlock(&(config.liveclients_mutex));
}
static void freeAllClients(void) {
listNode *ln = config.clients->head, *next;
while(ln) {
next = ln->next;
freeClient((client)ln->value);
ln = next;
}
}
static void resetClient(client c) {
aeEventLoop *el = CLIENT_GET_EVENTLOOP(c);
aeDeleteFileEvent(el,c->context->fd,AE_WRITABLE);
aeDeleteFileEvent(el,c->context->fd,AE_READABLE);
aeCreateFileEvent(el,c->context->fd,AE_WRITABLE,writeHandler,c);
c->written = 0;
c->pending = config.pipeline;
}
static void randomizeClientKey(client c) {
size_t i;
for (i = 0; i < c->randlen; i++) {
char *p = c->randptr[i]+11;
size_t r = 0;
if (config.randomkeys_keyspacelen != 0)
r = random() % config.randomkeys_keyspacelen;
size_t j;
for (j = 0; j < 12; j++) {
*p = '0'+r%10;
r/=10;
p--;
}
}
}
static void readHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
client c = (client)privdata;
void *reply = NULL;
UNUSED(el);
UNUSED(fd);
UNUSED(mask);
/* Calculate latency only for the first read event. This means that the
* server already sent the reply and we need to parse it. Parsing overhead
* is not part of the latency, so calculate it only once, here. */
if (c->latency < 0) c->latency = ustime()-(c->start);
if (redisBufferRead(c->context) != REDIS_OK) {
fprintf(stderr,"Error: %s\n",c->context->errstr);
exit(1);
} else {
while(c->pending) {
if (redisGetReply(c->context,&reply) != REDIS_OK) {
fprintf(stderr,"Error: %s\n",c->context->errstr);
exit(1);
}
if (reply != NULL) {
if (reply == (void*)REDIS_REPLY_ERROR) {
fprintf(stderr,"Unexpected error reply, exiting...\n");
exit(1);
}
redisReply *r = (redisReply*)reply;
int is_err = (r->type == REDIS_REPLY_ERROR);
if (is_err && config.showerrors) {
/* TODO: static lasterr_time not thread-safe */
static time_t lasterr_time = 0;
time_t now = time(NULL);
if (lasterr_time != now) {
lasterr_time = now;
if (c->cluster_node) {
printf("Error from server %s:%d: %s\n",
c->cluster_node->ip,
c->cluster_node->port,
r->str);
} else printf("Error from server: %s\n", r->str);
}
}
freeReplyObject(reply);
/* This is an OK for prefix commands such as auth and select.*/
if (c->prefix_pending > 0) {
c->prefix_pending--;
c->pending--;
/* Discard prefix commands on first response.*/
if (c->prefixlen > 0) {
size_t j;
sdsrange(c->obuf, c->prefixlen, -1);
/* We also need to fix the pointers to the strings
* we need to randomize. */
for (j = 0; j < c->randlen; j++)
c->randptr[j] -= c->prefixlen;
c->prefixlen = 0;
}
continue;
}
int requests_finished = 0;
atomicGetIncr(config.requests_finished, requests_finished, 1);
if (requests_finished < config.requests)
config.latency[requests_finished] = c->latency;
c->pending--;
if (c->pending == 0) {
resetClient(c);
break;
}
} else {
break;
}
}
}
}
static void writeHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
client c = (client)privdata;
UNUSED(el);
UNUSED(fd);
UNUSED(mask);
/* Initialize request when nothing was written. */
if (c->written == 0) {
/* Really initialize: randomize keys and set start time. */
if (config.randomkeys) randomizeClientKey(c);
atomicGet(config.slots_last_update, c->slots_last_update);
c->start = ustime();
c->latency = -1;
}
if (sdslen(c->obuf) > c->written) {
void *ptr = c->obuf+c->written;
ssize_t nwritten = write(c->context->fd,ptr,sdslen(c->obuf)-c->written);
if (nwritten == -1) {
if (errno != EPIPE)
fprintf(stderr, "Writing to socket: %s\n", strerror(errno));
freeClient(c);
return;
}
c->written += nwritten;
if (sdslen(c->obuf) == c->written) {
aeDeleteFileEvent(el,c->context->fd,AE_WRITABLE);
aeCreateFileEvent(el,c->context->fd,AE_READABLE,readHandler,c);
}
}
}
/* Create a benchmark client, configured to send the command passed as 'cmd' of
* 'len' bytes.
*
* The command is copied N times in the client output buffer (that is reused
* again and again to send the request to the server) accordingly to the configured
* pipeline size.
*
* Also an initial SELECT command is prepended in order to make sure the right
* database is selected, if needed. The initial SELECT will be discarded as soon
* as the first reply is received.
*
* To create a client from scratch, the 'from' pointer is set to NULL. If instead
* we want to create a client using another client as reference, the 'from' pointer
* points to the client to use as reference. In such a case the following
* information is take from the 'from' client:
*
* 1) The command line to use.
* 2) The offsets of the __rand_int__ elements inside the command line, used
* for arguments randomization.
*
* Even when cloning another client, prefix commands are applied if needed.*/
static client createClient(const char *cmd, size_t len, client from, int thread_id) {
int j;
int is_cluster_client = (config.cluster_mode && thread_id >= 0);
client c = (client)zmalloc(sizeof(struct _client), MALLOC_LOCAL);
const char *ip = NULL;
int port = 0;
c->cluster_node = NULL;
if (config.hostsocket == NULL || is_cluster_client) {
if (!is_cluster_client) {
ip = config.hostip;
port = config.hostport;
} else {
int node_idx = 0;
if (config.max_threads < config.cluster_node_count)
node_idx = config.liveclients % config.cluster_node_count;
else
node_idx = thread_id % config.cluster_node_count;
clusterNode *node = config.cluster_nodes[node_idx];
assert(node != NULL);
ip = (const char *) node->ip;
port = node->port;
c->cluster_node = node;
}
c->context = redisConnectNonBlock(ip,port);
} else {
c->context = redisConnectUnixNonBlock(config.hostsocket);
}
if (c->context->err) {
fprintf(stderr,"Could not connect to Redis at ");
if (config.hostsocket == NULL || is_cluster_client)
fprintf(stderr,"%s:%d: %s\n",ip,port,c->context->errstr);
else
fprintf(stderr,"%s: %s\n",config.hostsocket,c->context->errstr);
exit(1);
}
c->thread_id = thread_id;
/* Suppress hiredis cleanup of unused buffers for max speed. */
c->context->reader->maxbuf = 0;
/* Build the request buffer:
* Queue N requests accordingly to the pipeline size, or simply clone
* the example client buffer. */
c->obuf = sdsempty();
/* Prefix the request buffer with AUTH and/or SELECT commands, if applicable.
* These commands are discarded after the first response, so if the client is
* reused the commands will not be used again. */
c->prefix_pending = 0;
if (config.auth) {
char *buf = NULL;
int len;
if (config.user == NULL)
len = redisFormatCommand(&buf, "AUTH %s", config.auth);
else
len = redisFormatCommand(&buf, "AUTH %s %s",
config.user, config.auth);
c->obuf = sdscatlen(c->obuf, buf, len);
free(buf);
c->prefix_pending++;
}
if (config.enable_tracking) {
char *buf = NULL;
int len = redisFormatCommand(&buf, "CLIENT TRACKING on");
c->obuf = sdscatlen(c->obuf, buf, len);
free(buf);
c->prefix_pending++;
}
/* If a DB number different than zero is selected, prefix our request
* buffer with the SELECT command, that will be discarded the first
* time the replies are received, so if the client is reused the
* SELECT command will not be used again. */
if (config.dbnum != 0 && !is_cluster_client) {
c->obuf = sdscatprintf(c->obuf,"*2\r\n$6\r\nSELECT\r\n$%d\r\n%s\r\n",
(int)sdslen(config.dbnumstr),config.dbnumstr);
c->prefix_pending++;
}
c->prefixlen = sdslen(c->obuf);
/* Append the request itself. */
if (from) {
c->obuf = sdscatlen(c->obuf,
from->obuf+from->prefixlen,
sdslen(from->obuf)-from->prefixlen);
} else {
for (j = 0; j < config.pipeline; j++)
c->obuf = sdscatlen(c->obuf,cmd,len);
}
c->written = 0;
c->pending = config.pipeline+c->prefix_pending;
c->randptr = NULL;
c->randlen = 0;
c->stagptr = NULL;
c->staglen = 0;
/* Find substrings in the output buffer that need to be randomized. */
if (config.randomkeys) {
if (from) {
c->randlen = from->randlen;
c->randfree = 0;
c->randptr = (char**)zmalloc(sizeof(char*)*c->randlen, MALLOC_LOCAL);
/* copy the offsets. */
for (j = 0; j < (int)c->randlen; j++) {
c->randptr[j] = c->obuf + (from->randptr[j]-from->obuf);
/* Adjust for the different select prefix length. */
c->randptr[j] += c->prefixlen - from->prefixlen;
}
} else {
char *p = c->obuf;
c->randlen = 0;
c->randfree = RANDPTR_INITIAL_SIZE;
c->randptr = (char**)zmalloc(sizeof(char*)*c->randfree, MALLOC_LOCAL);
while ((p = strstr(p,"__rand_int__")) != NULL) {
if (c->randfree == 0) {
c->randptr = (char**)zrealloc(c->randptr,sizeof(char*)*c->randlen*2, MALLOC_LOCAL);
c->randfree += c->randlen;
}
c->randptr[c->randlen++] = p;
c->randfree--;
p += 12; /* 12 is strlen("__rand_int__). */
}
}
}
/* If cluster mode is enabled, set slot hashtags pointers. */
if (config.cluster_mode) {
if (from) {
c->staglen = from->staglen;
c->stagfree = 0;
c->stagptr = (char**)zmalloc(sizeof(char*)*c->staglen, MALLOC_LOCAL);
/* copy the offsets. */
for (j = 0; j < (int)c->staglen; j++) {
c->stagptr[j] = c->obuf + (from->stagptr[j]-from->obuf);
/* Adjust for the different select prefix length. */
c->stagptr[j] += c->prefixlen - from->prefixlen;
}
} else {
char *p = c->obuf;
c->staglen = 0;
c->stagfree = RANDPTR_INITIAL_SIZE;
c->stagptr = (char**)zmalloc(sizeof(char*)*c->stagfree, MALLOC_LOCAL);
while ((p = strstr(p,"{tag}")) != NULL) {
if (c->stagfree == 0) {
c->stagptr = (char**)zrealloc(c->stagptr,
sizeof(char*) * c->staglen*2, MALLOC_LOCAL);
c->stagfree += c->staglen;
}
c->stagptr[c->staglen++] = p;
c->stagfree--;
p += 5; /* 5 is strlen("{tag}"). */
}
}
}
aeEventLoop *el = NULL;
if (thread_id < 0) el = config.el;
else {
benchmarkThread *thread = config.threads[thread_id];
el = thread->el;
}
if (config.idlemode == 0)
aeCreateFileEvent(el,c->context->fd,AE_WRITABLE,writeHandler,c);
listAddNodeTail(config.clients,c);
atomicIncr(config.liveclients, 1);
atomicGet(config.slots_last_update, c->slots_last_update);
return c;
}
static void initBenchmarkThreads() {
int i;
if (config.threads) freeBenchmarkThreads();
config.threads = (benchmarkThread**)zmalloc(config.max_threads * sizeof(benchmarkThread*), MALLOC_LOCAL);
for (i = 0; i < config.max_threads; i++) {
benchmarkThread *thread = createBenchmarkThread(i);
config.threads[i] = thread;
}
}
/* Thread functions. */
static benchmarkThread *createBenchmarkThread(int index) {
benchmarkThread *thread = (benchmarkThread*)zmalloc(sizeof(*thread), MALLOC_LOCAL);
if (thread == NULL) return NULL;
thread->index = index;
thread->el = aeCreateEventLoop(1024*10);
return thread;
}
static void freeBenchmarkThread(benchmarkThread *thread) {
if (thread->el) aeDeleteEventLoop(thread->el);
zfree(thread);
}
static void freeBenchmarkThreads() {
int i = 0;
for (; i < config.max_threads; i++) {
benchmarkThread *thread = config.threads[i];
if (thread) freeBenchmarkThread(thread);
}
zfree(config.threads);
config.threads = NULL;
}
static void *execBenchmarkThread(void *ptr) {
benchmarkThread *thread = (benchmarkThread *) ptr;
aeMain(thread->el);
return NULL;
}
void initConfigDefaults() {
config.numclients = 50;
config.requests = 100000;
config.liveclients = 0;
config.el = aeCreateEventLoop(1024*10);
config.keepalive = 1;
config.datasize = 3;
config.pipeline = 1;
config.period_ms = 5000;
config.showerrors = 0;
config.randomkeys = 0;
config.randomkeys_keyspacelen = 0;
config.quiet = 0;
config.csv = 0;
config.loop = 0;
config.idlemode = 0;
config.latency = NULL;
config.clients = listCreate();
config.hostip = "127.0.0.1";
config.hostport = 6379;
config.hostsocket = NULL;
config.tests = NULL;
config.dbnum = 0;
config.auth = NULL;
config.precision = 1;
config.max_threads = MAX_THREADS;
config.threads = NULL;
config.cluster_mode = 0;
config.cluster_node_count = 0;
config.cluster_nodes = NULL;
config.redis_config = NULL;
config.is_fetching_slots = 0;
config.is_updating_slots = 0;
config.slots_last_update = 0;
config.enable_tracking = 0;
}
/* Returns number of consumed options. */
int parseOptions(int argc, const char **argv) {
int i;
int lastarg;
int exit_status = 1;
for (i = 1; i < argc; i++) {
lastarg = (i == (argc-1));
if (!strcmp(argv[i],"-c") || !strcmp(argv[i],"--clients")) {
if (lastarg) goto invalid;
config.numclients = atoi(argv[++i]);
} else if (!strcmp(argv[i],"--time")) {
if (lastarg) goto invalid;
config.period_ms = atoi(argv[++i]);
if (config.period_ms <= 0) {
printf("Warning: Invalid value for thread time. Defaulting to 5000ms.\n");
config.period_ms = 5000;
}
} else if (!strcmp(argv[i],"-h") || !strcmp(argv[i],"--host")) {
if (lastarg) goto invalid;
config.hostip = strdup(argv[++i]);
} else if (!strcmp(argv[i],"-p") || !strcmp(argv[i],"--port")) {
if (lastarg) goto invalid;
config.hostport = atoi(argv[++i]);
} else if (!strcmp(argv[i],"-s")) {
if (lastarg) goto invalid;
config.hostsocket = strdup(argv[++i]);
} else if (!strcmp(argv[i],"--password") ) {
if (lastarg) goto invalid;
config.auth = strdup(argv[++i]);
} else if (!strcmp(argv[i],"--user")) {
if (lastarg) goto invalid;
config.user = argv[++i];
} else if (!strcmp(argv[i],"--dbnum")) {
if (lastarg) goto invalid;
config.dbnum = atoi(argv[++i]);
config.dbnumstr = sdsfromlonglong(config.dbnum);
} else if (!strcmp(argv[i],"-t") || !strcmp(argv[i],"--threads")) {
if (lastarg) goto invalid;
config.max_threads = atoi(argv[++i]);
if (config.max_threads > MAX_THREADS) {
printf("Warning: Too many threads, limiting threads to %d.\n", MAX_THREADS);
config.max_threads = MAX_THREADS;
} else if (config.max_threads <= 0) {
printf("Warning: Invalid value for max threads. Defaulting to %d.\n", MAX_THREADS);
config.max_threads = MAX_THREADS;
}
} else if (!strcmp(argv[i],"--help")) {
exit_status = 0;
goto usage;
} else {
/* Assume the user meant to provide an option when the arg starts
* with a dash. We're done otherwise and should use the remainder
* as the command and arguments for running the benchmark. */
if (argv[i][0] == '-') goto invalid;
return i;
}
}
return i;
invalid:
printf("Invalid option \"%s\" or option argument missing\n\n",argv[i]);
usage:
printf(
"Usage: keydb-benchmark [-h <host>] [-p <port>] [-c <clients>] [-n <requests>] [-k <boolean>]\n\n"
" -h, --host <hostname> Server hostname (default 127.0.0.1)\n"
" -p, --port <port> Server port (default 6379)\n"
" -c <clients> Number of parallel connections (default 50)\n"
" -t, --threads <threads> Maximum number of threads to start before ending\n"
" --time <time> Time between spinning up new client threads, in milliseconds\n"
" --dbnum <db> Select the specified DB number (default 0)\n"
" --user <username> Used to send ACL style 'AUTH username pass'. Needs -a.\n"
" --password <password> Password for Redis Auth\n\n"
);
exit(exit_status);
}
int extractPropertyFromInfo(const char *info, const char *key, double &val) {
char *line = strstr((char*)info, key);
if (line == nullptr) return 1;
line += strlen(key) + 1; // Skip past key name and following colon
char *newline = strchr(line, '\n');
*newline = 0; // Terminate string after relevant line
val = strtod(line, nullptr);
return 0;
}
int extractPropertyFromInfo(const char *info, const char *key, unsigned int &val) {
char *line = strstr((char*)info, key);
if (line == nullptr) return 1;
line += strlen(key) + 1; // Skip past key name and following colon
char *newline = strchr(line, '\n');
*newline = 0; // Terminate string after relevant line
val = atoi(line);
return 0;
}
double getSelfCpuTime(struct rusage *self_ru) {
getrusage(RUSAGE_SELF, self_ru);
double user_time = self_ru->ru_utime.tv_sec + (self_ru->ru_utime.tv_usec / (double)1000000);
double system_time = self_ru->ru_stime.tv_sec + (self_ru->ru_stime.tv_usec / (double)1000000);
return user_time + system_time;
}
double getServerCpuTime(redisContext *ctx) {
redisReply *reply = (redisReply*)redisCommand(ctx, "INFO CPU");
if (reply->type != REDIS_REPLY_STRING) {
freeReplyObject(reply);
printf("Error executing INFO command. Exiting.\n");
return -1;
}
double used_cpu_user, used_cpu_sys;
if (extractPropertyFromInfo(reply->str, "used_cpu_user", used_cpu_user)) {
printf("Error reading user CPU usage from INFO command. Exiting.\n");
return -1;
}
if (extractPropertyFromInfo(reply->str, "used_cpu_sys", used_cpu_sys)) {
printf("Error reading system CPU usage from INFO command. Exiting.\n");
return -1;
}
freeReplyObject(reply);
return used_cpu_user + used_cpu_sys;
}
double getMean(std::deque<double> *q) {
double sum = 0;
for (long unsigned int i = 0; i < q->size(); i++) {
sum += (*q)[i];
}
return sum / q->size();
}
bool isAtFullLoad(double cpuPercent, unsigned int threads) {
return cpuPercent / threads >= 96;
}
int main(int argc, const char **argv) {
int i;
storage_init(NULL, 0);
srandom(time(NULL));
signal(SIGHUP, SIG_IGN);
signal(SIGPIPE, SIG_IGN);
initConfigDefaults();
i = parseOptions(argc,argv);
argc -= i;
argv += i;
config.latency = (long long*)zmalloc(sizeof(long long)*config.requests, MALLOC_LOCAL);
if (config.max_threads > 0) {
int err = 0;
err |= pthread_mutex_init(&(config.requests_issued_mutex), NULL);
err |= pthread_mutex_init(&(config.requests_finished_mutex), NULL);
err |= pthread_mutex_init(&(config.liveclients_mutex), NULL);
err |= pthread_mutex_init(&(config.is_fetching_slots_mutex), NULL);
err |= pthread_mutex_init(&(config.is_updating_slots_mutex), NULL);
err |= pthread_mutex_init(&(config.updating_slots_mutex), NULL);
err |= pthread_mutex_init(&(config.slots_last_update_mutex), NULL);
if (err != 0)
{
perror("Failed to initialize mutex");
exit(EXIT_FAILURE);
}
}
const char *set_value = "abcdefghijklmnopqrstuvwxyz";
int self_threads = 0;
char command[63];
initBenchmarkThreads();
redisContext *ctx = getRedisContext(config.hostip, config.hostport, config.hostsocket);
double server_cpu_time, last_server_cpu_time = getServerCpuTime(ctx);
struct rusage self_ru;
double self_cpu_time, last_self_cpu_time = getSelfCpuTime(&self_ru);
double server_cpu_load, last_server_cpu_load = 0, self_cpu_load, server_cpu_gain;
std::deque<double> load_gain_history = {};
double current_gain_avg, peak_gain_avg = 0;
redisReply *reply = (redisReply*)redisCommand(ctx, "INFO CPU");
if (reply->type != REDIS_REPLY_STRING) {
freeReplyObject(reply);
printf("Error executing INFO command. Exiting.\r\n");
return 1;
}
unsigned int server_threads;
if (extractPropertyFromInfo(reply->str, "server_threads", server_threads)) {
printf("Error reading server threads from INFO command. Exiting.\r\n");
return 1;
}
freeReplyObject(reply);
printf("Server has %d threads.\nStarting...\n", server_threads);
fflush(stdout);
while (self_threads < config.max_threads) {
for (int i = 0; i < config.numclients; i++) {
sprintf(command, "SET %d %s\r\n", self_threads * config.numclients + i, set_value);
createClient(command, strlen(command), NULL,self_threads);
}
benchmarkThread *t = config.threads[self_threads];
if (pthread_create(&(t->thread), NULL, execBenchmarkThread, t)){
fprintf(stderr, "FATAL: Failed to start thread %d. Exiting.\n", self_threads);
exit(1);
}
self_threads++;
usleep(config.period_ms * 1000);
server_cpu_time = getServerCpuTime(ctx);
self_cpu_time = getSelfCpuTime(&self_ru);
server_cpu_load = (server_cpu_time - last_server_cpu_time) * 100000 / config.period_ms;
self_cpu_load = (self_cpu_time - last_self_cpu_time) * 100000 / config.period_ms;
if (server_cpu_time < 0) {
break;
}
printf("%d threads, %d total clients. CPU Usage Self: %.1f%% (%.1f%% per thread), Server: %.1f%% (%.1f%% per thread)\r",
self_threads,
self_threads * config.numclients,
self_cpu_load,
self_cpu_load / self_threads,
server_cpu_load,
server_cpu_load / server_threads);
fflush(stdout);
server_cpu_gain = server_cpu_load - last_server_cpu_load;
load_gain_history.push_back(server_cpu_gain);
if (load_gain_history.size() > 5) {
load_gain_history.pop_front();
}
current_gain_avg = getMean(&load_gain_history);
if (current_gain_avg > peak_gain_avg) {
peak_gain_avg = current_gain_avg;
}
last_server_cpu_time = server_cpu_time;
last_self_cpu_time = self_cpu_time;
last_server_cpu_load = server_cpu_load;
if (isAtFullLoad(server_cpu_load, server_threads)) {
printf("\nServer is at full CPU load. If higher performance is expected, check server configuration.\n");
break;
}
if (current_gain_avg <= 0.05 * peak_gain_avg) {
printf("\nServer CPU load appears to have stagnated with increasing clients.\n"
"Server does not appear to be at full load. Check network for throughput.\n");
break;
}
if (self_threads * config.numclients > 2000) {
printf("\nClient limit of 2000 reached. Server is not at full load and appears to be increasing.\n"
"2000 clients should be more than enough to reach a bottleneck. Check all configuration.\n");
}
}
printf("Done.\n");
freeAllClients();
freeBenchmarkThreads();
return 0;
}
+1 -1
View File
@@ -426,7 +426,7 @@ sds createLatencyReport(void) {
}
if (advise_slowlog_inspect) {
report = sdscat(report,"- Check your Slow Log to understand what are the commands you are running which are too slow to execute. Please check https://redis.io/commands/slowlog for more information.\n");
report = sdscat(report,"- Check your Slow Log to understand what are the commands you are running which are too slow to execute. Please check https://docs.keydb.dev/docs/commands#slowlog for more information.\n");
}
/* Intrinsic latency. */
+2 -3
View File
@@ -39,12 +39,11 @@ void lazyfreeFreeSlotsMap(void *args[]) {
atomicIncr(lazyfreed_objects,len);
}
/* Release the rax mapping Redis Cluster keys to slots in the
* lazyfree thread. */
/* Release the key tracking table. */
void lazyFreeTrackingTable(void *args[]) {
rax *rt = (rax*)args[0];
size_t len = rt->numele;
raxFree(rt);
freeTrackingRadixTree(rt);
atomicDecr(lazyfree_objects,len);
atomicIncr(lazyfreed_objects,len);
}
+2 -2
View File
@@ -94,8 +94,8 @@ lwCanvas *lwCreateCanvas(int width, int height, int bgcolor) {
lwCanvas *canvas = zmalloc(sizeof(*canvas));
canvas->width = width;
canvas->height = height;
canvas->pixels = zmalloc(width*height);
memset(canvas->pixels,bgcolor,width*height);
canvas->pixels = zmalloc((size_t)width*height);
memset(canvas->pixels,bgcolor,(size_t)width*height);
return canvas;
}
+1 -1
View File
@@ -71,7 +71,7 @@ void memtest_progress_start(char *title, int pass) {
printf("\x1b[H\x1b[2K"); /* Cursor home, clear current line. */
printf("%s [%d]\n", title, pass); /* Print title. */
progress_printed = 0;
progress_full = ws.ws_col*(ws.ws_row-3);
progress_full = (size_t)ws.ws_col*(ws.ws_row-3);
fflush(stdout);
}
+2 -1
View File
@@ -6366,7 +6366,7 @@ int RM_GetClusterNodeInfo(RedisModuleCtx *ctx, const char *id, char *ip, char *m
/* If the information is not available, the function will set the
* field to zero bytes, so that when the field can't be populated the
* function kinda remains predictable. */
if (node->flags & CLUSTER_NODE_MASTER && node->slaveof)
if (node->flags & CLUSTER_NODE_SLAVE && node->slaveof)
memcpy(master_id,node->slaveof->name,REDISMODULE_NODE_ID_LEN);
else
memset(master_id,0,REDISMODULE_NODE_ID_LEN);
@@ -9425,6 +9425,7 @@ long moduleDefragGlobals(void) {
module->defrag_cb(&defrag_ctx);
defragged += defrag_ctx.defragged;
}
dictReleaseIterator(di);
return defragged;
}
+1 -1
View File
@@ -902,7 +902,7 @@ size_t objectComputeSize(robj *o, size_t sample_size) {
if (samples) asize += (double)elesize/samples*dictSize(d);
} else if (o->encoding == OBJ_ENCODING_INTSET) {
intset *is = (intset*)ptrFromObj(o);
asize = sizeof(*o)+sizeof(*is)+is->encoding*is->length;
asize = sizeof(*o)+sizeof(*is)+(size_t)is->encoding*is->length;
} else {
serverPanic("Unknown set encoding");
}
+45 -44
View File
@@ -103,7 +103,6 @@ static struct config {
int randomkeys_keyspacelen;
int keepalive;
int pipeline;
int showerrors;
long long start;
long long totlatency;
const char *title;
@@ -313,7 +312,9 @@ static redisContext *getRedisContext(const char *ip, int port,
fprintf(stderr, "Node %s:%d replied with error:\n%s\n", ip, port, reply->str);
else
fprintf(stderr, "Node %s replied with error:\n%s\n", hostsocket, reply->str);
goto cleanup;
freeReplyObject(reply);
redisFree(ctx);
exit(1);
}
freeReplyObject(reply);
return ctx;
@@ -370,9 +371,15 @@ fail:
fprintf(stderr, "ERROR: failed to fetch CONFIG from ");
if (hostsocket == NULL) fprintf(stderr, "%s:%d\n", ip, port);
else fprintf(stderr, "%s\n", hostsocket);
int abort_test = 0;
if (!strncmp(reply->str,"NOAUTH",5) ||
!strncmp(reply->str,"WRONGPASS",9) ||
!strncmp(reply->str,"NOPERM",5))
abort_test = 1;
freeReplyObject(reply);
redisFree(c);
freeRedisConfig(cfg);
if (abort_test) exit(1);
return NULL;
}
static void freeRedisConfig(redisConfig *cfg) {
@@ -517,44 +524,39 @@ static void readHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
exit(1);
}
redisReply *r = (redisReply*)reply;
int is_err = (r->type == REDIS_REPLY_ERROR);
if (is_err && config.showerrors) {
/* TODO: static lasterr_time not thread-safe */
static time_t lasterr_time = 0;
time_t now = time(NULL);
if (lasterr_time != now) {
lasterr_time = now;
if (c->cluster_node) {
printf("Error from server %s:%d: %s\n",
if (r->type == REDIS_REPLY_ERROR) {
/* Try to update slots configuration if reply error is
* MOVED/ASK/CLUSTERDOWN and the key(s) used by the command
* contain(s) the slot hash tag.
* If the error is not topology-update related then we
* immediately exit to avoid false results. */
if (c->cluster_node && c->staglen) {
int fetch_slots = 0, do_wait = 0;
if (!strncmp(r->str,"MOVED",5) || !strncmp(r->str,"ASK",3))
fetch_slots = 1;
else if (!strncmp(r->str,"CLUSTERDOWN",11)) {
/* Usually the cluster is able to recover itself after
* a CLUSTERDOWN error, so try to sleep one second
* before requesting the new configuration. */
fetch_slots = 1;
do_wait = 1;
printf("Error from server %s:%d: %s.\n",
c->cluster_node->ip,
c->cluster_node->port,
r->str);
}
if (do_wait) sleep(1);
if (fetch_slots && !fetchClusterSlotsConfiguration(c))
exit(1);
} else {
if (c->cluster_node) {
printf("Error from server %s:%d: %s\n",
c->cluster_node->ip,
c->cluster_node->port,
r->str);
} else printf("Error from server: %s\n", r->str);
}
}
/* Try to update slots configuration if reply error is
* MOVED/ASK/CLUSTERDOWN and the key(s) used by the command
* contain(s) the slot hash tag. */
if (is_err && c->cluster_node && c->staglen) {
int fetch_slots = 0, do_wait = 0;
if (!strncmp(r->str,"MOVED",5) || !strncmp(r->str,"ASK",3))
fetch_slots = 1;
else if (!strncmp(r->str,"CLUSTERDOWN",11)) {
/* Usually the cluster is able to recover itself after
* a CLUSTERDOWN error, so try to sleep one second
* before requesting the new configuration. */
fetch_slots = 1;
do_wait = 1;
printf("Error from server %s:%d: %s\n",
c->cluster_node->ip,
c->cluster_node->port,
r->str);
}
if (do_wait) sleep(1);
if (fetch_slots && !fetchClusterSlotsConfiguration(c))
exit(1);
}
}
freeReplyObject(reply);
@@ -1298,8 +1300,7 @@ static int fetchClusterSlotsConfiguration(client c) {
atomicGetIncr(config.is_fetching_slots, is_fetching_slots, 1);
if (is_fetching_slots) return -1; //TODO: use other codes || errno ?
atomicSet(config.is_fetching_slots, 1);
if (config.showerrors)
printf("Cluster slots configuration changed, fetching new one...\n");
printf("WARNING: Cluster slots configuration changed, fetching new one...\n");
const char *errmsg = "Failed to update cluster slots configuration";
static dictType dtype = {
dictSdsHash, /* hash function */
@@ -1475,7 +1476,8 @@ int parseOptions(int argc, const char **argv) {
} else if (!strcmp(argv[i],"-I")) {
config.idlemode = 1;
} else if (!strcmp(argv[i],"-e")) {
config.showerrors = 1;
printf("WARNING: -e option has been deprecated. "
"We now immediatly exit on error to avoid false results.\n");
} else if (!strcmp(argv[i],"-t")) {
if (lastarg) goto invalid;
/* We get the list of tests to run as a string in the form
@@ -1578,8 +1580,6 @@ usage:
" is executed. Default tests use this to hit random keys in the\n"
" specified range.\n"
" -P <numreq> Pipeline <numreq> requests. Default 1 (no pipeline).\n"
" -e If server replies with errors, show them on stdout.\n"
" (no more than 1 error per second is displayed)\n"
" -q Quiet. Just show query/sec values\n"
" --precision Number of decimal places to display in latency output (default 0)\n"
" --csv Output in CSV format\n"
@@ -1707,7 +1707,6 @@ int main(int argc, const char **argv) {
config.keepalive = 1;
config.datasize = 3;
config.pipeline = 1;
config.showerrors = 0;
config.randomkeys = 0;
config.randomkeys_keyspacelen = 0;
config.quiet = 0;
@@ -1790,8 +1789,9 @@ int main(int argc, const char **argv) {
} else {
config.redis_config =
getRedisConfig(config.hostip, config.hostport, config.hostsocket);
if (config.redis_config == NULL)
if (config.redis_config == NULL) {
fprintf(stderr, "WARN: could not fetch server CONFIG\n");
}
}
if (config.num_threads > 0) {
pthread_mutex_init(&(config.liveclients_mutex), NULL);
@@ -1954,8 +1954,8 @@ int main(int argc, const char **argv) {
}
if (test_is_selected("lrange") || test_is_selected("lrange_500")) {
len = redisFormatCommand(&cmd,"LRANGE mylist%s 0 449",tag);
benchmark("LRANGE_500 (first 450 elements)",cmd,len);
len = redisFormatCommand(&cmd,"LRANGE mylist%s 0 499",tag);
benchmark("LRANGE_500 (first 500 elements)",cmd,len);
free(cmd);
}
@@ -1983,6 +1983,7 @@ int main(int argc, const char **argv) {
} while(config.loop);
zfree(data);
zfree(data);
if (config.redis_config != NULL) freeRedisConfig(config.redis_config);
return 0;
+4 -2
View File
@@ -39,12 +39,14 @@
static char error[1044];
static off_t epos;
static long long line = 1;
int consumeNewline(char *buf) {
if (strncmp(buf,"\r\n",2) != 0) {
ERROR("Expected \\r\\n, got: %02x%02x",buf[0],buf[1]);
return 0;
}
line += 1;
return 1;
}
@@ -201,8 +203,8 @@ int redis_check_aof_main(int argc, char **argv) {
off_t pos = process(fp);
off_t diff = size-pos;
printf("AOF analyzed: size=%lld, ok_up_to=%lld, diff=%lld\n",
(long long) size, (long long) pos, (long long) diff);
printf("AOF analyzed: size=%lld, ok_up_to=%lld, ok_up_to_line=%lld, diff=%lld\n",
(long long) size, (long long) pos, line, (long long) diff);
if (diff > 0) {
if (fix) {
char buf[2];
+1 -1
View File
@@ -250,7 +250,7 @@ int redis_check_rdb(const char *rdbfilename, FILE *fp) {
rdbstate.doing = RDB_CHECK_DOING_READ_LEN;
if ((dbid = rdbLoadLen(&rdb,NULL)) == RDB_LENERR)
goto eoferr;
rdbCheckInfo("Selecting DB ID %d", dbid);
rdbCheckInfo("Selecting DB ID %llu", (unsigned long long)dbid);
continue; /* Read type again. */
} else if (type == RDB_OPCODE_RESIZEDB) {
/* RESIZEDB: Hint about the size of the keys in the currently
+7 -5
View File
@@ -467,7 +467,7 @@ static void cliOutputHelp(int argc, char **argv) {
help = entry->org;
if (group == -1) {
/* Compare all arguments */
if (argc == entry->argc) {
if (argc <= entry->argc) {
for (j = 0; j < argc; j++) {
if (strcasecmp(argv[j],entry->argv[j]) != 0) break;
}
@@ -648,7 +648,9 @@ static int cliConnect(int flags) {
cliRefreshPrompt();
}
if (config.hostsocket == NULL) {
/* Do not use hostsocket when we got redirected in cluster mode */
if (config.hostsocket == NULL ||
(config.cluster_mode && config.cluster_reissue_command)) {
context = redisConnect(config.hostip,config.hostport);
} else {
context = redisConnectUnix(config.hostsocket);
@@ -4554,7 +4556,7 @@ static void clusterManagerNodeArrayReset(clusterManagerNodeArray *array) {
static void clusterManagerNodeArrayShift(clusterManagerNodeArray *array,
clusterManagerNode **nodeptr)
{
assert(array->nodes < (array->nodes + array->len));
assert(array->len > 0);
/* If the first node to be shifted is not NULL, decrement count. */
if (*array->nodes != NULL) array->count--;
/* Store the first node to be shifted into 'nodeptr'. */
@@ -4567,7 +4569,7 @@ static void clusterManagerNodeArrayShift(clusterManagerNodeArray *array,
static void clusterManagerNodeArrayAdd(clusterManagerNodeArray *array,
clusterManagerNode *node)
{
assert(array->nodes < (array->nodes + array->len));
assert(array->len > 0);
assert(node != NULL);
assert(array->count < array->len);
array->nodes[array->count++] = node;
@@ -5944,7 +5946,7 @@ void showLatencyDistSamples(struct distsamples *samples, long long tot) {
printf("\033[38;5;0m"); /* Set foreground color to black. */
for (j = 0; ; j++) {
int coloridx =
ceil((float) samples[j].count / tot * (spectrum_palette_size-1));
ceil((double) samples[j].count / tot * (spectrum_palette_size-1));
int color = spectrum_palette[coloridx];
printf("\033[48;5;%dm%c", (int)color, samples[j].character);
samples[j].count = 0;
+12 -5
View File
@@ -2759,18 +2759,19 @@ void syncWithMaster(connection *conn) {
goto error;
}
retry_connect:
/* Send a PING to check the master is able to reply without errors. */
if (mi->repl_state == REPL_STATE_CONNECTING) {
if (mi->repl_state == REPL_STATE_CONNECTING || mi->repl_state == REPL_STATE_RETRY_NOREPLPING) {
serverLog(LL_NOTICE,"Non blocking connect for SYNC fired the event.");
/* Delete the writable event so that the readable event remains
* registered and we can wait for the PONG reply. */
connSetReadHandler(conn, syncWithMaster);
connSetWriteHandler(conn, NULL);
mi->repl_state = REPL_STATE_RECEIVE_PING_REPLY;
/* Send the PING, don't check for errors at all, we have the timeout
* that will take care about this. */
err = sendCommand(conn,"PING",NULL);
err = sendCommand(conn,mi->repl_state == REPL_STATE_RETRY_NOREPLPING ? "PING" : "REPLPING",NULL);
if (err) goto write_error;
mi->repl_state = REPL_STATE_RECEIVE_PING_REPLY;
return;
}
@@ -2783,7 +2784,13 @@ void syncWithMaster(connection *conn) {
* Note that older versions of Redis replied with "operation not
* permitted" instead of using a proper error code, so we test
* both. */
if (err[0] != '+' &&
if (strncmp(err,"-ERR unknown command",20) == 0) {
serverLog(LL_NOTICE,"Master does not support REPLPING, sending PING instead...");
mi->repl_state = REPL_STATE_RETRY_NOREPLPING;
sdsfree(err);
err = NULL;
goto retry_connect;
} else if (err[0] != '+' &&
strncmp(err,"-NOAUTH",7) != 0 &&
strncmp(err,"-NOPERM",7) != 0 &&
strncmp(err,"-ERR operation not permitted",28) != 0)
@@ -4803,7 +4810,7 @@ void replicationNotifyLoadedKey(redisDb *db, robj_roptr key, robj_roptr val, lon
redisObjectStack objTtl;
initStaticStringObject(objTtl, sdscatprintf(sdsempty(), "%lld", expire));
redisObjectStack objMvcc;
initStaticStringObject(objMvcc, sdscatprintf(sdsempty(), "%lu", mvccFromObj(val)));
initStaticStringObject(objMvcc, sdscatprintf(sdsempty(), "%" PRIu64, mvccFromObj(val)));
redisObject *argv[5] = {shared.mvccrestore, key.unsafe_robjcast(), &objMvcc, &objTtl, &objPayload};
replicationFeedSlaves(g_pserver->slaves, db - g_pserver->db, argv, 5);
+7 -4
View File
@@ -31,6 +31,7 @@
#include "sha1.h"
#include "rand.h"
#include "cluster.h"
#include "monotonic.h"
extern "C" {
#include <lua.h>
@@ -1437,7 +1438,7 @@ sds luaCreateFunction(client *c, lua_State *lua, robj *body) {
/* This is the Lua script "count" hook that we use to detect scripts timeout. */
void luaMaskCountHook(lua_State *lua, lua_Debug *ar) {
long long elapsed = mstime() - g_pserver->lua_time_start;
long long elapsed = elapsedMs(g_pserver->lua_time_start);
UNUSED(ar);
UNUSED(lua);
@@ -1588,7 +1589,8 @@ void evalGenericCommand(client *c, int evalsha) {
serverTL->in_eval = 1;
g_pserver->lua_caller = c;
g_pserver->lua_cur_script = funcname + 2;
g_pserver->lua_time_start = mstime();
g_pserver->lua_time_start = getMonotonicUs();
g_pserver->lua_time_snapshot = mstime();
g_pserver->lua_kill = 0;
if (g_pserver->lua_time_limit > 0 && ldb.active == 0) {
lua_sethook(lua,luaMaskCountHook,LUA_MASKCOUNT,100000);
@@ -2747,7 +2749,7 @@ void luaLdbLineHook(lua_State *lua, lua_Debug *ar) {
/* Check if a timeout occurred. */
if (ar->event == LUA_HOOKCOUNT && ldb.step == 0 && bp == 0) {
mstime_t elapsed = mstime() - g_pserver->lua_time_start;
mstime_t elapsed = elapsedMs(g_pserver->lua_time_start);
mstime_t timelimit = g_pserver->lua_time_limit ?
g_pserver->lua_time_limit : 5000;
if (elapsed >= timelimit) {
@@ -2777,6 +2779,7 @@ void luaLdbLineHook(lua_State *lua, lua_Debug *ar) {
lua_pushstring(lua, "timeout during Lua debugging with client closing connection");
lua_error(lua);
}
g_pserver->lua_time_start = mstime();
g_pserver->lua_time_start = getMonotonicUs();
g_pserver->lua_time_snapshot = mstime();
}
}
+6 -6
View File
@@ -4131,16 +4131,16 @@ void sentinelSetCommand(client *c) {
int numargs = j-old_j+1;
switch(numargs) {
case 2:
sentinelEvent(LL_WARNING,"+set",ri,"%@ %s %s",ptrFromObj(c->argv[old_j]),
ptrFromObj(c->argv[old_j+1]));
sentinelEvent(LL_WARNING,"+set",ri,"%@ %s %s",szFromObj(c->argv[old_j]),
szFromObj(c->argv[old_j+1]));
break;
case 3:
sentinelEvent(LL_WARNING,"+set",ri,"%@ %s %s %s",ptrFromObj(c->argv[old_j]),
ptrFromObj(c->argv[old_j+1]),
ptrFromObj(c->argv[old_j+2]));
sentinelEvent(LL_WARNING,"+set",ri,"%@ %s %s %s",szFromObj(c->argv[old_j]),
szFromObj(c->argv[old_j+1]),
szFromObj(c->argv[old_j+2]));
break;
default:
sentinelEvent(LL_WARNING,"+set",ri,"%@ %s",ptrFromObj(c->argv[old_j]));
sentinelEvent(LL_WARNING,"+set",ri,"%@ %s",szFromObj(c->argv[old_j]));
break;
}
}
+17 -8
View File
@@ -57,7 +57,6 @@
#include <limits.h>
#include <float.h>
#include <math.h>
#include <sys/resource.h>
#include <sys/utsname.h>
#include <locale.h>
#include <sys/socket.h>
@@ -67,7 +66,6 @@
#include "aelocker.h"
#include "motd.h"
#include "t_nhash.h"
#include <sys/resource.h>
#ifdef __linux__
#include <sys/prctl.h>
#include <sys/mman.h>
@@ -755,6 +753,10 @@ struct redisCommand redisCommandTable[] = {
"ok-stale ok-loading fast @connection @replication",
0,NULL,0,0,0,0,0,0},
{"replping",pingCommand,-1,
"ok-stale fast @connection @replication",
0,NULL,0,0,0,0,0,0},
{"echo",echoCommand,2,
"fast @connection",
0,NULL,0,0,0,0,0,0},
@@ -2770,6 +2772,7 @@ void createSharedObjects(void) {
shared.lastid = makeObjectShared("LASTID",6);
shared.default_username = makeObjectShared("default",7);
shared.ping = makeObjectShared("ping",4);
shared.replping = makeObjectShared("replping", 8);
shared.setid = makeObjectShared("SETID",5);
shared.keepttl = makeObjectShared("KEEPTTL",7);
shared.load = makeObjectShared("LOAD",4);
@@ -4301,6 +4304,8 @@ int processCommand(client *c, int callFlags) {
return C_OK;
}
int is_read_command = (c->cmd->flags & CMD_READONLY) ||
(c->cmd->proc == execCommand && (c->mstate.cmd_flags & CMD_READONLY));
int is_write_command = (c->cmd->flags & CMD_WRITE) ||
(c->cmd->proc == execCommand && (c->mstate.cmd_flags & CMD_WRITE));
int is_denyoom_command = (c->cmd->flags & CMD_DENYOOM) ||
@@ -4515,7 +4520,7 @@ int processCommand(client *c, int callFlags) {
c->cmd->proc != discardCommand &&
c->cmd->proc != watchCommand &&
c->cmd->proc != unwatchCommand &&
c->cmd->proc != resetCommand &&
c->cmd->proc != resetCommand &&
!(c->cmd->proc == shutdownCommand &&
c->argc == 2 &&
tolower(((char*)ptrFromObj(c->argv[1]))[0]) == 'n') &&
@@ -4527,6 +4532,14 @@ int processCommand(client *c, int callFlags) {
return C_OK;
}
/* Prevent a replica from sending commands that access the keyspace.
* The main objective here is to prevent abuse of client pause check
* from which replicas are exempt. */
if ((c->flags & CLIENT_SLAVE) && (is_may_replicate_command || is_write_command || is_read_command)) {
rejectCommandFormat(c, "Replica can't interract with the keyspace");
return C_OK;
}
/* If the server is paused, block the client until
* the pause has ended. Replicas are never paused. */
if (!(c->flags & CLIENT_SLAVE) &&
@@ -4635,10 +4648,6 @@ int prepareForShutdown(int flags) {
overwrite the synchronous saving did by SHUTDOWN. */
if (g_pserver->child_type == CHILD_TYPE_RDB) {
serverLog(LL_WARNING,"There is a child saving an .rdb. Killing it!");
/* Note that, in killRDBChild, we call rdbRemoveTempFile that will
* do close fd(in order to unlink file actully) in background thread.
* The temp rdb file fd may won't be closed when redis exits quickly,
* but OS will close this fd when process exits. */
killRDBChild();
/* Note that, in killRDBChild normally has backgroundSaveDoneHandler
* doing it's cleanup, but in this case this code will not be reached,
@@ -6834,7 +6843,7 @@ int main(int argc, char **argv) {
serverLog(LL_WARNING, "Failed to test the kernel for a bug that could lead to data corruption during background save. "
"Your system could be affected, please report this error.");
if (!checkIgnoreWarning("ARM64-COW-BUG")) {
serverLog(LL_WARNING,"Redis will now exit to prevent data corruption. "
serverLog(LL_WARNING,"KeyDB will now exit to prevent data corruption. "
"Note that it is possible to suppress this warning by setting the following config: ignore-warnings ARM64-COW-BUG");
exit(1);
}
+5 -2
View File
@@ -505,6 +505,7 @@ typedef enum {
REPL_STATE_NONE = 0, /* No active replication */
REPL_STATE_CONNECT, /* Must connect to master */
REPL_STATE_CONNECTING, /* Connecting to master */
REPL_STATE_RETRY_NOREPLPING, /* Master does not support REPLPING, retry with PING */
/* --- Handshake states, must be ordered --- */
REPL_STATE_RECEIVE_PING_REPLY, /* Wait for PING reply */
REPL_STATE_SEND_HANDSHAKE, /* Send handshake sequance to master */
@@ -1283,7 +1284,7 @@ struct sharedObjectsStruct {
*emptyscan, *multi, *exec, *left, *right, *hset, *srem, *xgroup, *xclaim,
*script, *replconf, *eval, *persist, *set, *pexpireat, *pexpire,
*time, *pxat, *px, *retrycount, *force, *justid,
*lastid, *ping, *setid, *keepttl, *load, *createconsumer,
*lastid, *ping, *replping, *setid, *keepttl, *load, *createconsumer,
*getack, *special_asterick, *special_equals, *default_username,
*hdel, *zrem, *mvccrestore, *pexpirememberat,
*select[PROTO_SHARED_SELECT_CMDS],
@@ -1934,7 +1935,8 @@ struct redisServer {
::dict *lua_scripts; /* A dictionary of SHA1 -> Lua scripts */
unsigned long long lua_scripts_mem; /* Cached scripts' memory + oh */
mstime_t lua_time_limit; /* Script timeout in milliseconds */
mstime_t lua_time_start; /* Start time of script, milliseconds time */
monotime lua_time_start; /* monotonic timer to detect timed-out script */
mstime_t lua_time_snapshot; /* Snapshot of mstime when script is started */
int lua_write_dirty; /* True if a write command was called during the
execution of the current script. */
int lua_random_dirty; /* True if a random command was called during the
@@ -2296,6 +2298,7 @@ void disableTracking(client *c);
void trackingRememberKeys(client *c);
void trackingInvalidateKey(client *c, robj *keyobj);
void trackingInvalidateKeysOnFlush(int async);
void freeTrackingRadixTree(rax *rt);
void freeTrackingRadixTreeAsync(rax *rt);
void trackingLimitUsedSlots(void);
uint64_t trackingGetTotalItems(void);
+3 -3
View File
@@ -512,7 +512,7 @@ void spopWithCountCommand(client *c) {
const char *sdsele;
robj *objele;
int encoding;
int64_t llele;
int64_t llele = 0;
unsigned long remaining = size-count; /* Elements left after SPOP. */
/* If we are here, the number of requested elements is less than the
@@ -664,7 +664,7 @@ void srandmemberWithCountCommand(client *c) {
int uniq = 1;
robj_roptr set;
const char *ele;
int64_t llele;
int64_t llele = 0;
int encoding;
dict *d;
@@ -813,7 +813,7 @@ void srandmemberWithCountCommand(client *c) {
void srandmemberCommand(client *c) {
robj_roptr set;
const char *ele;
int64_t llele;
int64_t llele = 0;
int encoding;
if (c->argc == 3) {
+3 -3
View File
@@ -813,7 +813,7 @@ int64_t streamTrim(stream *s, streamAddTrimArgs *args) {
}
deleted += deleted_from_lp;
/* Now we the entries/deleted counters. */
/* Now we update the entries/deleted counters. */
p = lpFirst(lp);
lp = lpReplaceInteger(lp,&p,entries-deleted_from_lp);
p = lpNext(lp,p); /* Skip deleted field. */
@@ -842,7 +842,7 @@ int64_t streamTrim(stream *s, streamAddTrimArgs *args) {
/* Trims a stream by length. Returns the number of deleted items. */
int64_t streamTrimByLength(stream *s, long long maxlen, int approx) {
streamAddTrimArgs args = {0};
streamAddTrimArgs args = {{0}};
args.trim_strategy = TRIM_STRATEGY_MAXLEN;
args.approx_trim = approx;
args.limit = approx ? 100 * g_pserver->stream_node_max_entries : 0;
@@ -852,7 +852,7 @@ int64_t streamTrimByLength(stream *s, long long maxlen, int approx) {
/* Trims a stream by minimum ID. Returns the number of deleted items. */
int64_t streamTrimByID(stream *s, streamID minid, int approx) {
streamAddTrimArgs args = {0};
streamAddTrimArgs args = {{0}};
args.trim_strategy = TRIM_STRATEGY_MINID;
args.approx_trim = approx;
args.limit = approx ? 100 * g_pserver->stream_node_max_entries : 0;
+1 -1
View File
@@ -810,7 +810,7 @@ void stralgoLCS(client *c) {
/* Setup an uint32_t array to store at LCS[i,j] the length of the
* LCS A0..i-1, B0..j-1. Note that we have a linear array here, so
* we index it as LCS[j+(blen+1)*j] */
uint32_t *lcs = (uint32_t*)zmalloc((alen+1)*(blen+1)*sizeof(uint32_t));
uint32_t *lcs = (uint32_t*)zmalloc((size_t)(alen+1)*(blen+1)*sizeof(uint32_t));
#define LCS(A,B) lcs[(B)+((A)*(blen+1))]
/* Start building the LCS table. */
+2 -2
View File
@@ -1,9 +1,9 @@
# Redis configuration for testing.
# KeyDB configuration for testing.
always-show-logo yes
notify-keyspace-events KEA
daemonize no
pidfile /var/run/redis.pid
pidfile /var/run/keydb.pid
port 6379
timeout 0
bind 127.0.0.1
+1 -1
View File
@@ -1,5 +1,5 @@
# Minimal configuration for testing.
always-show-logo yes
daemonize no
pidfile /var/run/redis.pid
pidfile /var/run/keydb.pid
loglevel verbose
+1 -1
View File
@@ -1,4 +1,4 @@
source tests/support/redis.tcl
source tests/support/keydb.tcl
source tests/support/util.tcl
set ::tlsdir "tests/tls"
+1 -1
View File
@@ -1,4 +1,4 @@
source tests/support/redis.tcl
source tests/support/keydb.tcl
source tests/support/util.tcl
set ::tlsdir "tests/tls"
+1 -1
View File
@@ -1,4 +1,4 @@
source tests/support/redis.tcl
source tests/support/keydb.tcl
set ::tlsdir "tests/tls"
+8 -8
View File
@@ -10,7 +10,7 @@
package require Tcl 8.5
set tcl_precision 17
source ../support/redis.tcl
source ../support/keydb.tcl
source ../support/util.tcl
source ../support/server.tcl
source ../support/test.tcl
@@ -36,7 +36,7 @@ set ::run_matching {} ; # If non empty, only tests matching pattern are run.
if {[catch {cd tmp}]} {
puts "tmp directory not found."
puts "Please run this test from the Redis source root."
puts "Please run this test from the KeyDB source root."
exit 1
}
@@ -92,7 +92,7 @@ proc spawn_instance {type base_port count {conf {}} {base_conf_file ""}} {
puts $cfg [format "tls-key-file %s/../../tls/server.key" [pwd]]
puts $cfg [format "tls-client-cert-file %s/../../tls/client.crt" [pwd]]
puts $cfg [format "tls-client-key-file %s/../../tls/client.key" [pwd]]
puts $cfg [format "tls-dh-params-file %s/../../tls/redis.dh" [pwd]]
puts $cfg [format "tls-dh-params-file %s/../../tls/keydb.dh" [pwd]]
puts $cfg [format "tls-ca-cert-file %s/../../tls/ca.crt" [pwd]]
puts $cfg "loglevel debug"
} else {
@@ -303,7 +303,7 @@ proc pause_on_error {} {
set count 10
if {[lindex $argv 1] ne {}} {set count [lindex $argv 1]}
foreach_redis_id id {
puts "=== REDIS $id ===="
puts "=== KeyDB $id ===="
puts [exec tail -$count redis_$id/log.txt]
puts "---------------------\n"
}
@@ -317,7 +317,7 @@ proc pause_on_error {} {
}
} elseif {$cmd eq {ls}} {
foreach_redis_id id {
puts -nonewline "Redis $id"
puts -nonewline "KeyDB $id"
set errcode [catch {
set str {}
append str "@[RI $id tcp_port]: "
@@ -348,13 +348,13 @@ proc pause_on_error {} {
}
}
} elseif {$cmd eq {help}} {
puts "ls List Sentinel and Redis instances."
puts "ls List Sentinel and KeyDB instances."
puts "show-sentinel-logs \[N\] Show latest N lines of logs."
puts "show-keydb-logs \[N\] Show latest N lines of logs."
puts "S <id> cmd ... arg Call command in Sentinel <id>."
puts "R <id> cmd ... arg Call command in Redis <id>."
puts "R <id> cmd ... arg Call command in KeyDB <id>."
puts "SI <id> <field> Show Sentinel <id> INFO <field>."
puts "RI <id> <field> Show Redis <id> INFO <field>."
puts "RI <id> <field> Show KeyDB <id> INFO <field>."
puts "continue Resume test."
} else {
set errcode [catch {eval $line} retval]
+12
View File
@@ -158,6 +158,18 @@ tags {"aof"} {
assert_match "*not valid*" $result
}
test "Short read: Utility should show the abnormal line num in AOF" {
create_aof {
append_to_aof [formatCommand set foo hello]
append_to_aof "!!!"
}
catch {
exec src/keydb-check-aof $aof_path
} result
assert_match "*ok_up_to_line=8*" $result
}
test "Short read: Utility should be able to fix the AOF" {
set result [exec src/keydb-check-aof --fix $aof_path << "y\n"]
assert_match "*Successfully truncated AOF*" $result
File renamed without changes.
File renamed without changes.
+1 -3
View File
@@ -1,12 +1,10 @@
set system_name [string tolower [exec uname -s]]
# ldd --version returns 1 under musl for unknown reasons. If this check stops working, that may be why
set is_musl [catch {exec ldd --version}]
set system_supported 0
# We only support darwin or Linux with glibc
if {$system_name eq {darwin}} {
set system_supported 1
} elseif {$system_name eq {linux} && $is_musl eq 0} {
} elseif {$system_name eq {linux}} {
# Avoid the test on libmusl, which does not support backtrace
set ldd [exec ldd src/keydb-server]
if {![string match {*libc.musl*} $ldd]} {
+1 -1
View File
@@ -4,7 +4,7 @@ proc show_cluster_status {} {
# The following is the regexp we use to match the log line
# time info. Logs are in the following form:
#
# 11296:M 25 May 2020 17:37:14.652 # Server initialized
# 11296:11296:M 25 May 2020 17:37:14.652 # Server initialized
set log_regexp {^[0-9]+:^[0-9]+:[A-Z] [0-9]+ [A-z]+ [0-9]+ ([0-9:.]+) .*}
set repl_regexp {(master|repl|sync|backlog|meaningful|offset)}
File renamed without changes.
+1 -1
View File
@@ -355,7 +355,7 @@ proc start_server {options {code undefined}} {
dict set config "tls-key-file" [format "%s/tests/tls/server.key" [pwd]]
dict set config "tls-client-cert-file" [format "%s/tests/tls/client.crt" [pwd]]
dict set config "tls-client-key-file" [format "%s/tests/tls/client.key" [pwd]]
dict set config "tls-dh-params-file" [format "%s/tests/tls/redis.dh" [pwd]]
dict set config "tls-dh-params-file" [format "%s/tests/tls/keydb.dh" [pwd]]
dict set config "tls-ca-cert-file" [format "%s/tests/tls/ca.crt" [pwd]]
dict set config "loglevel" "debug"
}
+3 -3
View File
@@ -5,7 +5,7 @@
package require Tcl 8.5
set tcl_precision 17
source tests/support/redis.tcl
source tests/support/keydb.tcl
source tests/support/server.tcl
source tests/support/tmpfile.tcl
source tests/support/test.tcl
@@ -57,8 +57,8 @@ set ::all_tests {
integration/psync2-reg
integration/psync2-pingoff
integration/failover
integration/redis-cli
integration/redis-benchmark
integration/keydb-cli
integration/keydb-benchmark
unit/pubsub
unit/slowlog
unit/scripting
+2 -2
View File
@@ -25,7 +25,7 @@ test {CONFIG SET port number} {
test {CONFIG SET bind address} {
start_server {} {
# non-valid address
catch {r CONFIG SET bind "some.wrong.bind.address"} e
catch {r CONFIG SET bind "999.999.999.999"} e
assert_match {*Failed to bind to specified addresses*} $e
# make sure server still bound to the previous address
@@ -33,4 +33,4 @@ test {CONFIG SET bind address} {
$rd PING
$rd close
}
}
}
+44 -46
View File
@@ -62,57 +62,55 @@ start_server {overrides {save ""} tags {"other"}} {
} {*index is out of range*}
tags {consistency} {
if {true} {
if {$::accurate} {set numops 10000} else {set numops 1000}
test {Check consistency of different data types after a reload} {
r flushdb
createComplexDataset r $numops
set dump [csvdump r]
set sha1 [r debug digest]
r debug reload
set sha1_after [r debug digest]
if {$sha1 eq $sha1_after} {
set _ 1
} else {
set newdump [csvdump r]
puts "Consistency test failed!"
puts "You can inspect the two dumps in /tmp/repldump*.txt"
if {$::accurate} {set numops 10000} else {set numops 1000}
test {Check consistency of different data types after a reload} {
r flushdb
createComplexDataset r $numops
set dump [csvdump r]
set sha1 [r debug digest]
r debug reload
set sha1_after [r debug digest]
if {$sha1 eq $sha1_after} {
set _ 1
} else {
set newdump [csvdump r]
puts "Consistency test failed!"
puts "You can inspect the two dumps in /tmp/repldump*.txt"
set fd [open /tmp/repldump1.txt w]
puts $fd $dump
close $fd
set fd [open /tmp/repldump2.txt w]
puts $fd $newdump
close $fd
set fd [open /tmp/repldump1.txt w]
puts $fd $dump
close $fd
set fd [open /tmp/repldump2.txt w]
puts $fd $newdump
close $fd
set _ 0
}
} {1}
set _ 0
}
} {1}
test {Same dataset digest if saving/reloading as AOF?} {
r config set aof-use-rdb-preamble no
r bgrewriteaof
waitForBgrewriteaof r
r debug loadaof
set sha1_after [r debug digest]
if {$sha1 eq $sha1_after} {
set _ 1
} else {
set newdump [csvdump r]
puts "Consistency test failed!"
puts "You can inspect the two dumps in /tmp/aofdump*.txt"
test {Same dataset digest if saving/reloading as AOF?} {
r config set aof-use-rdb-preamble no
r bgrewriteaof
waitForBgrewriteaof r
r debug loadaof
set sha1_after [r debug digest]
if {$sha1 eq $sha1_after} {
set _ 1
} else {
set newdump [csvdump r]
puts "Consistency test failed!"
puts "You can inspect the two dumps in /tmp/aofdump*.txt"
set fd [open /tmp/aofdump1.txt w]
puts $fd $dump
close $fd
set fd [open /tmp/aofdump2.txt w]
puts $fd $newdump
close $fd
set fd [open /tmp/aofdump1.txt w]
puts $fd $dump
close $fd
set fd [open /tmp/aofdump2.txt w]
puts $fd $newdump
close $fd
set _ 0
}
} {1}
}
set _ 0
}
} {1}
}
test {EXPIRES after a reload (snapshot + append only file rewrite)} {
+2 -2
View File
@@ -100,8 +100,8 @@ start_server {tags {"tls"}} {
set master_port [srv 0 port]
# Use a non-restricted client/server cert for the replica
set redis_crt [format "%s/tests/tls/redis.crt" [pwd]]
set redis_key [format "%s/tests/tls/redis.key" [pwd]]
set redis_crt [format "%s/tests/tls/keydb.crt" [pwd]]
set redis_key [format "%s/tests/tls/keydb.key" [pwd]]
start_server [list overrides [list tls-cert-file $redis_crt tls-key-file $redis_key] \
omit [list tls-client-cert-file tls-client-key-file]] {
+11
View File
@@ -395,6 +395,17 @@ start_server {tags {"tracking network"}} {
assert {[lindex msg 2] eq {} }
}
test {Test ASYNC flushall} {
clean_all
r CLIENT TRACKING on REDIRECT $redir_id
r GET key1
r GET key2
assert_equal [s 0 tracking_total_keys] 2
$rd_sg FLUSHALL ASYNC
assert_equal [s 0 tracking_total_keys] 0
assert_equal [lindex [$rd_redirection read] 2] {}
}
# Keys are defined to be evicted 100 at a time by default.
# If after eviction the number of keys still surpasses the limit
# defined in tracking-table-max-keys, we increases eviction
+40
View File
@@ -0,0 +1,40 @@
#! /bin/bash
if [[ "$1" == "--help" ]] || [[ "$1" == "-h" ]] || [[ "$#" -ne 2 ]] ; then
echo "This script is used to compare different KeyDB configuration files."
echo ""
echo " Usage: compare_config.sh [keydb1.conf] [keydb2.conf]"
echo ""
echo "Output: a side by side sorted list of all active parameters, followed by a summary of the differences."
exit 0
fi
conf_1=$(mktemp)
conf_2=$(mktemp)
echo "----------------------------------------------------"
echo "--- display all active parameters in config files---"
echo "----------------------------------------------------"
echo ""
echo "--- $1 ---" > $conf_1
echo "" >> $conf_1
grep -ve "^#" -ve "^$" $1 | sort >> $conf_1
echo "--- $2 ---" >> $conf_2
echo "" >> $conf_2
grep -ve "^#" -ve "^$" $2 | sort >> $conf_2
pr -T --merge $conf_1 $conf_2
echo ""
echo ""
echo "--------------------------------------------"
echo "--- display config file differences only ---"
echo "--------------------------------------------"
echo ""
sdiff --suppress-common-lines $conf_1 $conf_2
rm $conf_1
rm $conf_2
exit 0
+6 -6
View File
@@ -3,10 +3,10 @@
# Generate some test certificates which are used by the regression test suite:
#
# tests/tls/ca.{crt,key} Self signed CA certificate.
# tests/tls/redis.{crt,key} A certificate with no key usage/policy restrictions.
# tests/tls/keydb.{crt,key} A certificate with no key usage/policy restrictions.
# tests/tls/client.{crt,key} A certificate restricted for SSL client usage.
# tests/tls/server.{crt,key} A certificate restricted fro SSL server usage.
# tests/tls/redis.dh DH Params file.
# tests/tls/keydb.dh DH Params file.
generate_cert() {
local name=$1
@@ -19,7 +19,7 @@ generate_cert() {
[ -f $keyfile ] || openssl genrsa -out $keyfile 2048
openssl req \
-new -sha256 \
-subj "/O=Redis Test/CN=$cn" \
-subj "/O=KeyDB Test/CN=$cn" \
-key $keyfile | \
openssl x509 \
-req -sha256 \
@@ -38,7 +38,7 @@ openssl req \
-x509 -new -nodes -sha256 \
-key tests/tls/ca.key \
-days 3650 \
-subj '/O=Redis Test/CN=Certificate Authority' \
-subj '/O=KeyDB Test/CN=Certificate Authority' \
-out tests/tls/ca.crt
cat > tests/tls/openssl.cnf <<_END_
@@ -53,6 +53,6 @@ _END_
generate_cert server "Server-only" "-extfile tests/tls/openssl.cnf -extensions server_cert"
generate_cert client "Client-only" "-extfile tests/tls/openssl.cnf -extensions client_cert"
generate_cert redis "Generic-cert"
generate_cert keydb "Generic-cert"
[ -f tests/tls/redis.dh ] || openssl dhparam -out tests/tls/redis.dh 2048
[ -f tests/tls/keydb.dh ] || openssl dhparam -out tests/tls/keydb.dh 2048
+1 -1
View File
@@ -2,7 +2,7 @@
# Copyright (C) 2011 Salvatore Sanfilippo
# Released under the BSD license like Redis itself
source ../tests/support/redis.tcl
source ../tests/support/keydb.tcl
set ::port 12123
set ::tests {PING,SET,GET,INCR,LPUSH,LPOP,SADD,SPOP,LRANGE_100,LRANGE_600,MSET}
set ::datasize 16