upstream/KeyDB
2
0 Fork 1
mirror of https://github.com/Snapchat/KeyDB.git synced 2026-11-06 01:22:42 -05:00
Code Releases 1 Activity
27,573 Commits 106 Branches 29 Tags
async_flash
Commit Graph
27573 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
John Sully f886fce2d1 Prevent erroneous socket closure 2024-04-05 21:48:04 +00:00
John Sully 66be482ef5 Naming fix 2024-04-05 19:15:24 +00:00
John Sully c50e2d49d6 Fix fastsync test failures caused by closing due to repl buffer size too soon 2024-04-05 19:12:19 +00:00
John Sully e580a8ebbe Handle pending IO fastsync 2024-04-04 21:20:47 +00:00
John Sully bb1780a3f3 Fix warning 2024-03-20 19:24:36 +00:00
John Sully 41733d733f Add data on actual disk used that is compared to maxstorage 2024-03-20 19:21:13 +00:00
John Sully f5e367d889 Fix hash count mismatch asserts for getKeysInSlot 2024-03-19 23:10:03 +00:00
John Sully 0267bc5ef7 Add FLASH tests to CI 2024-03-19 03:32:44 +00:00
John Sully a430473a39 Fix asserts after RDB load with FLASH 2024-03-19 03:28:48 +00:00
John Sully 5fbe9915c3 Fixup ifdef for easier switching 2024-03-18 21:50:51 +00:00
John Sully 559588b995 Disable swapdb tests with flash 2024-03-08 22:27:55 +00:00
John Sully d84fa525b5 Fix crash in FLASH eviction 2024-03-08 20:21:30 +00:00
John Sully 088a1dbf82 Ensure cluster tests run in flash mode in CI 2024-03-08 17:56:57 +00:00
John Sully ad2a2824b5 Disable swapdb with flash since it doesn't work 2024-03-08 08:43:09 +00:00
John Sully c83c55f181 Fix expire tracking assert failure 2024-03-08 08:12:22 +00:00
John Sully fe5503a640 Emit maxstorage metric 2024-03-07 20:27:29 +00:00
John Sully 6c15c438c6 Build with flash 2024-03-05 19:19:53 +00:00
John Sully bff8b73d0d Fix ARM CI 2024-03-05 03:42:34 +00:00
John Sully 8a2016aaea Enable key cache for async reads 2024-03-05 03:22:24 +00:00
John Sully 7a488a83e9 Serialize writes due to races causing test failures 2024-03-05 03:04:10 +00:00
John Sully 8c4fa5d991 Fix typo in test path 2024-03-05 03:03:08 +00:00
John Sully c5c4779ca4 Fix hung clients 2024-03-05 02:54:15 +00:00
John Sully 8f2dbc3b53 Fix crash 2024-03-05 02:49:05 +00:00
John Sully e7ab275276 Fix linker issues with newer linkers 2024-03-04 20:50:22 +00:00
John Sully e5f8e77a52 Fix test failure due to slowness with many DBs 2024-03-01 23:40:43 +00:00
John Sully 260f47bbb5 Do not create threadpools if we won't need them 2024-03-01 22:19:25 +00:00
John Sully c08683e920 Return set to vector for clients to be processed ensuring we run them in order 2024-03-01 22:18:56 +00:00
John Sully 490ee844e4 Merge branch 'main' into async_flash 2024-02-29 23:36:17 +00:00
Alex Cope 752f204454 Fix for hit rate metric (#257) 
Tweak hit rate metric to get delta instead of aggregate
 2024-02-22 11:55:58 -08:00
Alex Cope cba39986c1 Limit repl buffer writes per event loop (#256) 
We discovered that full sync loops were happening because after the replica loads the RDB file, the repl backlog which the primary sends over can sometimes exceed int32 max, causing an SSL error due to integer overflow:

```
Error writing to client: error:140D010F:SSL routines:SSL_write:bad length
```

After fixing that error by capping the  and testing in staging, we still observed errors because the primary attempts to send a massive (multi-GB) payload in one event loop, which on the staging instances takes over one minute causing the primary to fail health checks and restart. To address this we cap the writes sent per event loop to a configurable amount, similar to `NET_MAX_WRITES_PER_EVENT` in the normal case.
 2024-02-20 18:09:40 -08:00
Alex Cope 195828ed4c Log hitrate metric in modstatsd (#255) 
https://jira.sc-corp.net/browse/CACHE-1543

Log hit rate metric separately because M3DB caps values at 360 million so dividing monotonically increasing `keyspace_hit/miss` values is meaningless with any significant amount of traffic. Multiplying by one million since M3DB only uses integer values.
 2024-02-15 12:58:20 -08:00
John Sully 7b50232757 Merge branch 'main' into async_flash 2024-01-31 23:07:35 +00:00
John Sully 989421ea55 Prevent crash on client disconnecting while I/O in flight 2024-01-31 22:02:26 +00:00
John Sully 65b9165031 Fix crash in replica replaying storage op 2024-01-31 22:02:02 +00:00
Malavan Sotheeswaran 77eaaa2247 skip memefficiency tests in CI as they are inconsistent (#250) 2024-01-30 17:12:00 -05:00
Malavan Sotheeswaran 5681d067a6 Fix replication rdb load timeout (#251) 
* don't timeout replication when loading rdb

* fix bracket placement
 2024-01-30 14:22:51 -05:00
Malavan Sotheeswaran d1484aa780 Add redis bloom module (#252) 
* build redisbloom in dockerfile

* add separate step for redisbloom

* end RUN properly

* can't verify with github?

* just needed ca-cert
 2024-01-30 13:54:18 -05:00
Malavan Sotheeswaran 7a32cf9955 reset replica offsets after resetting replication buffer (#247) 2024-01-25 19:03:38 -05:00
Malavan SotheeswaranandSergey Kolosov 2b01c6c710 handle nodes in handshake state in statsd module cluster nodes metrics (#246) 
* handle nodes in handshake state in statsd module cluster nodes metrics

* handle all possible outputs of cluster nodes

* Update src/modules/keydb_modstatsd/modmain.cpp

Co-authored-by: Sergey Kolosov <skolosov@snapchat.com>

* make error plural

Co-authored-by: Sergey Kolosov <skolosov@snapchat.com>
 2024-01-25 15:54:00 -05:00
Malavan Sotheeswaran b74652a180 change overload protection to sending -OVERLOAD errors instead of disconnecting clients (#245) 2024-01-23 14:29:43 -05:00
John Sully 3d4c5a0af0 Fix compile warning 2024-01-11 19:16:39 +00:00
Yu Zhao (Yuri) e1935a7cfd refactor asyncwrite and asyncread work queue 2024-01-10 15:31:26 -05:00
Jacob Bohac bb7c5a4c77 TLS cname loadshedding ignore list (#242) 
* TLS cname loadshedding ignore list
 2023-12-08 14:15:50 -07:00
Malavan Sotheeswaran bb115ca99a add overload-protect-tenacity to configure how many clients to evict (#243) 
* add overload-protect-tenacity to configure how many clients to evict

* need to divide tenacity config by 100 to get percentage
 2023-12-08 15:02:44 -05:00
Malavan Sotheeswaran 49afd8ff2f update overload-ignore-list to accept subnets as well (#239) 
* update overload-ignore-list to be use subnets as well

* get ip directly from conn

* add getIPV4 impl

* ntohl

* don't shift at the end of the loop

* update for more efficient impl and add ipv6 support

* array of arrays

* one combined array

* have to reassign output of sdscat

* include mask in operator<

* replace raw numbers with constants

* add log when we ignore client for loadshedding
 2023-12-04 16:20:40 -05:00
Zhaozhen Liang deb139eb08 Merge pull request #241 from Snapchat/improve-insufficient-replica-alarm 
update insufficient replicas metrics to separate severity
 2023-11-27 18:43:41 -07:00
zliang 18d6241d75 Merge branch 'main' into improve-insufficient-replica-alarm 2023-11-27 18:10:01 -07:00
zliang 443e1afe89 add cool 2023-11-27 17:50:06 -07:00
zliang 8fec9ba466 emit number of active replicas metrics instead 2023-11-27 17:01:04 -07:00
Jacob Bohac 8a5857ddb7 Overload CPU reading metric (#240) 
* Overload CPU reading metric

* fix

* fix
 2023-11-27 15:31:29 -07:00