upstream/KeyDB
2
0 Fork 1
mirror of https://github.com/Snapchat/KeyDB.git synced 2026-11-06 17:42:32 -05:00
Code Releases 1 Activity
14,204 Commits 106 Branches 29 Tags
fix_compile_errors
Commit Graph
1119 Commits
Author SHA1 Message Date
John Sully d207bc6612 Ensure clean shutdown with multiple threads 2022-05-24 05:10:18 +00:00
John Sully fef21b9c04 Implements the soft shutdown feature to allow clients to cooperatively disconnect preventing disruption during shutdown 2022-05-24 03:04:57 +00:00
John Sully fb7aacec46 Merge OSS back into pro 2022-05-18 01:29:15 +00:00
John Sully a2f3682e87 Fix PSYNC test crashes 2022-05-04 01:37:40 +00:00
John Sully 930b375309 PSYNC production fixes 2022-04-26 02:07:28 +00:00
John Sully d864e299b3 Make active client balancing a configurable option 2022-04-26 01:55:22 +00:00
Vivek Saini 51aefbb677 Removed mergeReplicationId 2022-04-26 01:55:22 +00:00
Vivek Saini cebf0fcf64 Inclusive language 2022-04-26 01:55:22 +00:00
Christian Legge d18477c9e3 Save and restore master info in rdb to allow active replica partial sync (#371) 
* save replid for all masters in rdb

* expanded rdbSaveInfo to hold multiple master structs

* parse repl-masters from rdb

* recover replid info from rdb in active replica mode, attempt partial sync

* save offset from rdb into correct variable

* don't change replid based on master in active rep

* save and load psync info from correct fields
 2022-04-26 01:55:22 +00:00
Malavan Sotheeswaran 812a671e0d hide forklock object in ae 2022-04-26 01:55:22 +00:00
John SullyandJohn Sully 3439bd47a7 Make multithread load configurable and disabled by default (#57) 
Co-authored-by: John Sully <john@csquare.ca>
 2022-04-07 13:20:38 -04:00
John Sully 9dd19e4e60 Log the connected masters in the INFO command 2022-04-02 01:20:45 -04:00
John Sully 370785244b Support dynamic runtime modification of tls-allowlist, and add config rewrite support 2022-03-31 17:48:37 -04:00
Vivek Saini 300a945ecd Changed term whitelist to allowlist (#54) 2022-03-31 14:15:03 -04:00
John Sully fb2f289e89 Fix lock inversion in freeClientAsync - make it not dependent on the AE Lock 2022-03-29 00:19:49 -04:00
John Sully 4354bd4fe8 Fix mac build warnings 2022-03-07 19:28:39 -05:00
John Sully 9320e10396 Fix test instability caused by MVCC slip with lfence command when async commands are enabled 2022-03-07 16:40:56 -05:00
John Sully 0329e62d31 Optimize async command snapshot creation, and make slip configurable 2022-03-07 16:00:48 -05:00
John Sully adc66136d9 Fix mac build breaks and remove license checks (won't work on mac) 2022-03-07 14:50:31 -05:00
Vivek Saini 9a0c765875 TLS Name Validation (#48) 
Added TLS Name Validation
 2022-03-03 13:59:07 -05:00
Vivek Saini 97bff49db4 Added TLS rotation support 2022-02-02 14:20:19 -05:00
christianEQ 898191750b Merge remote-tracking branch 'opensource/RELEASE_6' into keydbpro 
Former-commit-id: 439c655a543f4d3224d90bcdeb21ba43c2ab8ab7
 2022-01-14 22:19:42 +00:00
MalavanEQAlpha dfcbba494a Merge pull request #386 from EQ-Alpha/fix_rdb_hang 
add readwrite lock for forking

Former-commit-id: 95c2a7e970ce17dd3e35eeaec0b63f7dd96d1cf5
 2022-01-14 13:19:33 -08:00
John Sully cbedcda44a Fix partial sync corruption with FLASH 
Former-commit-id: 532f58c0539b775c040c0dd9a2ad3dc349faf87a
 2021-12-23 00:04:28 -05:00
VivekSainiEQ 7702c97cdf Merge remote-tracking branch 'mainpro/PRO_RELEASE_6' into keydbpro 
Former-commit-id: 5a32d66ee382b6d227a67073afc81ca058d605ed
 2021-12-06 20:43:23 +00:00
John Sully 273bc25b2b Implent the force-backlog-disk-reserve flag 
Former-commit-id: d39f7f4407f8935b1540dd302be3e24ac02c5700
 2021-12-04 02:07:03 +00:00
malavan 5555589247 add async to hget and fix bug where a command can hold a key from main db without lock 
Former-commit-id: b0b8f426fda7cdabd3898a2d0b38d0fae47d1e69
 2021-12-02 17:11:26 +00:00
Malavan Sotheeswaran a9647ed275 Merge branch 'async_commands' of https://gitlab.eqalpha.com/external-collab/keydb-pro-6 into async_commands 
Former-commit-id: 1afa51c4d21d695c052dbec690bf3880b243dbec
 2021-11-30 12:17:46 -05:00
Malavan Sotheeswaran 5233969299 Merge branch 'keydbpro' into async_commands 
Former-commit-id: 9eaddb8ca1424ff3225dac9c144d23848228c7d2
 2021-11-30 11:47:51 -05:00
Malavan Sotheeswaran fff3c87882 Merge branch 'fastsync_collab' into 'keydbpro' 
Fastsync collab

See merge request external-collab/keydb-pro-6!9

Former-commit-id: 8f410e4b814ea6ac506ab540ee1394834fd1d8a8
 2021-11-26 20:53:00 +00:00
John Sully 10f5eb8908 Implement the disk backed backlog functionality 
Former-commit-id: 759cc01c6ba05f9a865c11580cc4975b5f1bd1d6
 2021-11-26 03:05:23 +00:00
VivekSainiEQ 21ca83f64a Removed trial code entirely 
Former-commit-id: 540a60eb12d01c2839228f9fdaa6233aed99dc38
 2021-11-15 23:56:30 +00:00
VivekSainiEQ fd29199a98 Removed KeyDB Enterprise Trial 
Former-commit-id: 0899ede8db54db80a2cc101412e5dbcaedb706b1
 2021-11-15 20:33:54 +00:00
VivekSainiEQ 1ff6e37d2f Merge remote-tracking branch 'opensource/unstable' into PRO_RELEASE_6 
Former-commit-id: 60da86471f68b13e2456e113ecd4aa117d95b134
 2021-11-12 21:12:53 +00:00
christianEQ 0c5052a2b8 added createMetadataDb to IStorageFactory + implementations 
Former-commit-id: a2acf75484d2af93aad9d03a20bd402893044860
 2021-11-11 13:41:00 +00:00
John Sully d1a7f5ba2c Initial implementation of snapshot fast replication. There are still a few TODOs in progress 
Former-commit-id: 0febdcdab8693af443f350968ed3d8c80106675d
 2021-11-09 19:36:07 +00:00
VivekSainiEQ 43d055f68d Backported time thread code from enterprise 
Former-commit-id: 3f27b35af74f5bf5e80837d6ce12abc977f1434b
 2021-11-05 00:30:34 +00:00
malavan eca3cecafc Merge branch 'keydbpro' of https://gitlab.eqalpha.com/external-collab/keydb-pro-6 into keydbpro 
Former-commit-id: 5b88af1067d344f121b2d65e34fb40e229722575
 2021-11-02 19:14:47 +00:00
malavan d629880543 don't delete db or snapshots on shutdown, still delete storage provider 
Former-commit-id: edb840ce10ea77ce654ba27c9eadbf98bbc13403
 2021-11-02 19:01:37 +00:00
John Sully 9d9779c2a7 Make the replica weighting configurable 
Former-commit-id: be6a8a7e68acb5cfbe950f13b903e6f7b98c5a39
 2021-10-29 17:59:46 +00:00
John Sully 205c90bca4 Fix slower performance during replication by better balancing client load on threads servicing a replica 
Former-commit-id: 496f91d3f169fcfe6d94c2ea69cee402f8eb60ca
 2021-10-22 03:16:33 +00:00
VivekSainiEQ 817d2c55e4 Resolved merge conflicts in prior commit 
Former-commit-id: 1594ef5e3e9f6413d8a68d91cad8eec0dc163b4e
 2021-10-21 22:35:15 +00:00
VivekSainiEQ 443f8d8f9b Merge tag '6.2.6' into Redis_626_Merge 
Former-commit-id: a4220f27d308863d77feab1713a920add23062ff
 2021-10-21 22:33:55 +00:00
jsully 34f36bfe38 Merge branch 'multithread_load' into 'keydbpro' 
Multithread load

See merge request external-collab/keydb-pro-6!5

Former-commit-id: 20e712244071028b0f75ccad477308efd139261f
 2021-10-08 17:55:55 +00:00
DarrenJiang13 94bf9b175e [BUGFIX] Add some missed error statistics (#9328) 
add error counting for some missed behaviors.

(cherry picked from commit 43eb0ce3bf76a5d287b93a767bead9ad6230a1ad)
 2021-10-04 13:59:40 +03:00
Oran Agra 1334180f82 Improvements to corrupt payload sanitization (#9321) 
Recently we found two issues in the fuzzer tester: #9302 #9285
After fixing them, more problems surfaced and this PR (as well as #9297) aims to fix them.

Here's a list of the fixes
- Prevent an overflow when allocating a dict hashtable
- Prevent OOM when attempting to allocate a huge string
- Prevent a few invalid accesses in listpack
- Improve sanitization of listpack first entry
- Validate integrity of stream consumer groups PEL
- Validate integrity of stream listpack entry IDs
- Validate ziplist tail followed by extra data which start with 0xff

Co-authored-by: sundb <sundbcn@gmail.com>
(cherry picked from commit 0c90370e6d71cc68e4d9cc79a0d8b1e768712a5b)
 2021-10-04 13:59:40 +03:00
menwen 44b3a6df77 Add latency monitor sample when key is deleted via lazy expire (#9317) 
Fix that there is no sample latency after the key expires via expireIfNeeded().
Some refactoring for shared code.

(cherry picked from commit ca559819f7dcd97ba9ef667bf38360a9527d62f6)
 2021-10-04 13:59:40 +03:00
Oran Agra 24977cdff6 Fix ziplist and listpack overflows and truncations (CVE-2021-32627, CVE-2021-32628) 
- fix possible heap corruption in ziplist and listpack resulting by trying to
  allocate more than the maximum size of 4GB.
- prevent ziplist (hash and zset) from reaching size of above 1GB, will be
  converted to HT encoding, that's not a useful size.
- prevent listpack (stream) from reaching size of above 1GB.
- XADD will start a new listpack if the new record may cause the previous
  listpack to grow over 1GB.
- XADD will respond with an error if a single stream record is over 1GB
- List type (ziplist in quicklist) was truncating strings that were over 4GB,
  now it'll respond with an error.
 2021-10-04 13:59:40 +03:00
Oran Agra c34f970be5 Prevent unauthenticated client from easily consuming lots of memory (CVE-2021-32675) 
This change sets a low limit for multibulk and bulk length in the
protocol for unauthenticated connections, so that they can't easily
cause redis to allocate massive amounts of memory by sending just a few
characters on the network.
The new limits are 10 arguments of 16kb each (instead of 1m of 512mb)
 2021-10-04 13:59:40 +03:00
John Sully e27091d82e Handle the case where the key cache exceeds maxmemory 
Former-commit-id: 01febf902267fec7fe87e6437b0b81fd08b50963
 2021-10-04 07:34:05 +00:00