91 Commits
Author SHA1 Message Date
38169682f0 Merge latest internal to OSS (#720)
* add docker build

* fix the working dir in Dockerfile

* add release publish docker image

* address intentation and use default release

* migrate keydb_modstatsd to keydb-internal

* rm

* add submodule cpp-statsd-client

* include trigger keydb_modstatsd Makefile in modules Makefile

* update

* have clean also trigger keydb_modstatsd clean

* move cpp-statsd-client to deps

* checkout to a06a5b9359f31d946fe163b9038586982971ae49

* update relative path in compilation

* remove submodule instead use the source

* include building keydb statsd module

* fix check in Dockerfile docker-entrypoint.sh

* fix

* fix the comment caused stuck docker build

* use commit hash as tag template

* fix

* test tag

* Revert "test tag"

This reverts commit 9cbc57137d.

* make docker build independent

* add new build to ci

* emit system free metrics with '/proc/meminfo'

* have emit system free memory within metrics_time_taken_us and also add metric time taken for it

* Remove Expireset (#217)

Major refactor to place expiry information directly in the object struct.

* update MemFree to MemAvailable in keydb statsd

* add metric emit for non-empty primary with less than 2 connected replicas

* address comments

* Multiply CPU percent metric by 100

* Fix memory leaks

* Fix slow to free when low lock contention

* fix nodename metricsname

* fix unnecessary replace

* Make propagating before freeing module context optional (#225)

* don't propogate on module context free for rdb load

* default in wrong place

* Flash expiration (#197)

Design Doc: https://docs.google.com/document/d/1NmnYGnHLdZp-KOUCUatX5iXpF-L3YK4VUc9Lm3Tqxpo/edit?usp=sharing

* Emit more aggregate metrics in modstatsd (#223)

* Permit keys of differing slots as long as they are served by this cluster and we are not migrating

* Fix over pessimistic checks that prevent replicas from serving mget

* Fix logic bug

* async rehash is preventing rehashing during RDB load after a db flush.  Ensure it can't interefere after a flush

* make async rehash configurable

* only use internal locks when multithreaded (#205)

* Fix crossslot error migrating batches of keys

* Fix bug where we erroneously answer queries belonging to another shard

* fix mac compile

* enable level_compaction_dynamic_level_bytes after flush, and flush expires for FLASH (#229)

* enable level_compaction_dynamic_level_bytes after flush, and flush expires

* update debug reload for flash

* update debug reload for flash complete

* missing forward declare

* commit existing changes then track changes for debug reload

* missing args

* commitChanges is conditional

Co-authored-by: John Sully <jsully@snapchat.com>

---------

Co-authored-by: zliang <zliang@snapchat.com>
Co-authored-by: John Sully <jsully@snapchat.com>
Co-authored-by: Alex Cope <acope@snapchat.com>
Co-authored-by: John Sully <john@csquare.ca>
2023-09-28 18:13:27 -04:00
Ikko Eltociear Ashimine 870b0db23d Fix typo in module.cpp (#686)
comparision -> comparison
2023-07-25 21:33:30 -07:00
Malavan Sotheeswaran b258322f9e Fix propagate in transaction crash (#200)
* ignore propagate_in_transaction in processCommand if command is async

* make propogate_in_transaction thread local

* forgot module.cpp

* move client_pause_in_transaction to thread local

* don't do async commands while in eval or exec

* don't do keys or scan async if in transaction
2023-06-21 16:28:57 -07:00
malavan 0ee3a5ace1 fork lock for all threads, use fastlock for readwritelock 2022-04-13 13:52:41 -04:00
John Sully d77bbee238 Fix mac build breaks and remove license checks (won't work on mac) 2022-03-07 14:50:31 -05:00
christianEQ b785b03068 Merge remote-tracking branch 'opensource/RELEASE_6' into keydbpro
Former-commit-id: 439c655a543f4d3224d90bcdeb21ba43c2ab8ab7
2022-01-14 22:19:42 +00:00
MalavanEQAlpha 0f4268579e Merge pull request #386 from EQ-Alpha/fix_rdb_hang
add readwrite lock for forking

Former-commit-id: dabb81960f
2022-01-14 13:19:33 -08:00
John Sully 968d81dcf9 Fix module test break
Former-commit-id: 50dfee2039
2021-12-23 13:12:34 -05:00
John Sully 129f27f751 Fix module test failures with blocked clients
Former-commit-id: 18ec4b8600ea3462843b58a0f0e917824dfd7ff3
2021-12-23 02:10:20 -05:00
VivekSainiEQ bfcea943ea Merge remote-tracking branch 'mainpro/PRO_RELEASE_6' into keydbpro
Former-commit-id: 5a32d66ee382b6d227a67073afc81ca058d605ed
2021-12-06 20:43:23 +00:00
VivekSainiEQ f9a355900a Merge remote-tracking branch 'opensource/unstable' into PRO_RELEASE_6
Former-commit-id: 60da86471f68b13e2456e113ecd4aa117d95b134
2021-11-12 21:12:53 +00:00
VivekSainiEQ 93b0482376 Resolved merge conflicts in prior commit
Former-commit-id: b88f06b16f
2021-10-21 22:35:15 +00:00
VivekSainiEQ 1d882b5ddd Merge tag '6.2.6' into Redis_626_Merge
Former-commit-id: e6d7e01be6
2021-10-21 22:33:55 +00:00
John Sully 59855278e0 Initial implementation of async commands
Former-commit-id: 0d3a4ce7e60c42b2d81962f13100bd1789bf64a8
2021-09-21 20:43:16 +00:00
MalavanEQAlpha e0af0bcb95 Merge pull request #313 from MalavanEQAlpha/fixissue295
Resolve Issue #295 by speeding up keyIsExpired and setting timeout on subkey expires.

Former-commit-id: 9e5e6c2f86
2021-08-18 17:34:18 -04:00
malavan 67ffa578d7 Merge remote-tracking branch 'opensource/unstable' into keydbpro
Former-commit-id: cf40fa5da29ba9574d20d706d0a41ca7001df95a
2021-08-11 20:41:54 +00:00
VivekSainiEQ 23c0c8171e Merge remote-tracking branch 'upstream/unstable' into redisearch_fixes
Former-commit-id: 952ae9fbe3
2021-08-11 19:28:04 +00:00
malavan 3a379f0934 use INVALID_EXPIRE instead of -1
Former-commit-id: 9e45984a97
2021-07-14 22:13:29 +00:00
John Sully 6bb2075d3c Merge branch 'unstable' into keydbpro
Former-commit-id: 81ded8a35daa5100cac7299a7d0b5f43ee7ac74f
2021-07-09 04:41:47 +00:00
John Sully da2aceabcf Merge tag '6.2.3' into unstable
Former-commit-id: 1895dbb768
2021-07-09 04:40:31 +00:00
John Sully 6b4f686f5f Merge branch 'unstable' into keydbpro
Former-commit-id: 205d8f18d2bb8df5253bab40578b006b7aa73fd5
2021-05-28 23:32:46 +00:00
VivekSainiEQ 68cdb2f1ad Initialized serverTL in more places in module.cpp
Former-commit-id: 8d81592bec
2021-05-25 16:55:47 +00:00
John Sully 5267928381 Merge tag '6.2.2' into unstable
Former-commit-id: 93ebb31b17
2021-05-21 05:54:39 +00:00
John Sully fe8efa916b Merge tag '6.2.1' into unstable
Former-commit-id: bfed57e3e0
2021-05-19 02:59:48 +00:00
VivekSainiEQ c48144356e Created and initialized seperate thread variables for modules
Former-commit-id: 3bb6b16c4a
2021-03-30 13:46:03 -04:00
VivekSainiEQ 8714a26cd9 Removed hasModuleGIL boolean and added fix from PR #292
Former-commit-id: 68d213f4c9
2021-03-19 20:10:24 +00:00
VivekSainiEQ 189967e7af added lock releasing w/ hasModuleGIL, changed module serverTL, and moved module_blocking_pipe to global scope to fix issue #276
Former-commit-id: 7d9a2ce827
2021-03-19 20:08:28 +00:00
John Sully e5343f47c2 Merge branch 'unstable' into keydbpro
Former-commit-id: e2140793f2bf565972ada799af73bf4457e2718d
2021-02-08 18:17:09 +00:00
christianEQ cca8d186aa Merge remote-tracking branch 'opensource/unstable' into keydbpro
Former-commit-id: 5bad058733de2c217340bb9ee48f02b07d754808
2021-02-03 18:10:27 +00:00
John Sully 23495c22a5 Simplify the handling of CreateTimer
Former-commit-id: 7b4e25008a
2021-02-01 15:51:22 -05:00
VivekSainiEQ 55659b1569 Modified RM_CreateTimer to prevent more that one function being posted at a time
Former-commit-id: f66e5a9c3d
2021-02-01 15:51:22 -05:00
VivekSainiEQ ad11549020 Changed RM_CreateTimer to only call aePostFunction is existing aePostFunction isn't in flight
Former-commit-id: 9954f5b4a4
2021-02-01 15:51:22 -05:00
VivekSainiEQ 73e8825266 Now post entire timer installation process as one function to make atomic with respect to global locks
Former-commit-id: 53936661c8
2021-02-01 15:51:22 -05:00
VivekSainiEQ eb83f3d47b Removed more uses of fSynchronous and the use of condition variable and mutex on the control struct.
Former-commit-id: 6ab08cc3e1
2021-02-01 15:51:22 -05:00
VivekSainiEQ 363293ef61 Added fix for scenario where module thread waiting for s_mutexModule in acquireGIL can deadlock with module thread waiting for s_mutex in releaseGIL
Former-commit-id: 3205373bb3
2021-02-01 15:51:22 -05:00
VivekSainiEQ cba4dcffdf removed synchronous calls to aePostFunction and changed scope of g_fModuleThread in order to prevent module related deadlocks, issue #214
Former-commit-id: 3b8d1f7076
2021-02-01 15:51:22 -05:00
christianEQ 358debebfa Merge tag 'tags/6.0.10' into redismerge_2021-01-20
Former-commit-id: dadce055f8
2021-01-26 21:43:09 +00:00
John Sully 706ba4e1dc Merge branch 'unstable' into keydbpro
Former-commit-id: b7a1e16c0f04e8aeb3764e3681fa14fc0f97f6a3
2020-12-10 02:37:28 +00:00
John Sully b336d269ca Remove addReply*Async methods since we already know if its async or not. This is just a source of bugs
Former-commit-id: df22cdf6e9
2020-10-24 02:18:03 +00:00
John Sully 2085e21dca Fix module test failures due to locking
Former-commit-id: 420ccdfbae
2020-10-01 00:08:54 +00:00
John Sully 4f18a247e3 Merge tag '6.0.8' into unstable
Former-commit-id: 4c7e4b91a6
2020-09-30 19:47:55 +00:00
John Sully 6a57593467 Merge branch 'PRO_RELEASE_6' into keydbpro
Former-commit-id: bffe010ea5279bee869bc61cc6d933979e10bbea
2020-07-13 03:32:14 +00:00
John Sully 1ad2d96697 Merge branch 'unstable' into keydbpro
Former-commit-id: 0dafbc254a0efd5ee302d5c58fb2ca0a85110104
2020-07-13 03:31:47 +00:00
John Sully efc8e719f9 Prevent deadlock in RM_ThreadSafeContextLock() when triggered as part of a module callback in a server thread
Former-commit-id: e01312642b
2020-07-12 18:17:53 +00:00
John Sully c83fe382ff COW pointer is not thread safe so we need a global expire lock
Former-commit-id: ac12eef75301564d920e622a08e4f586486fd395
2020-06-09 19:58:42 -04:00
John Sully 44c1e6d5e7 Merge branch 'unstable' into keydbpro
Former-commit-id: 08a36155e3db9918048e87c3d691b7317787c9ab
2020-06-01 17:41:37 -04:00
John Sully 4038288c4e Fix module multithreaded test failures
Former-commit-id: 1ef35cf466
2020-05-31 23:46:12 -04:00
John Sully cece963cf3 Merge branch 'unstable' into keydbpro
Former-commit-id: a830cf85df236885558c5571c0bf23cfb23e3655
2020-05-24 14:41:53 -04:00
John Sully 27eb239f1a Fix bad merge in CI.yml
Former-commit-id: 6311d709c3
2020-05-21 22:09:06 -04:00
John Sully c4db71f971 Merge commit '024c380b9da02bc4112822c0f5f9ac1388b4205b' into unstable
Former-commit-id: 7676f5b15f
2020-05-21 17:36:53 -04:00