periodic: ensure elections are retried after a timeout
Ignore the current node state, and execute the gray-failures routines
anyway. We need to ensure that the election timeout is reached, another
election is launched, regardless of the state of the last election (if
it has not suceedeed, of course).
Ensure the resign are sent to a single node instead of all the nodes,
since it will guarantee a non-working election as the result.
Clarified some comments here and there.
Patch: https://lists.sr.ht/~ne02ptzero/libfloat/patches/58069
Signed-off-by: Louis Solofrizzo <lsolofrizzo@scaleway.com>
Acked-by: Florian Florensa <fflorensa@scaleway.com>
Acked-by: Saalik Hatia <shatia@scaleway.com>
periodic: fix dynamodb like elections
Some bugs were raised on network partitions. In short, we're changing
the election behavior to ensure it is trigerred only when needed.
- On a partition, do not trigger an election and simply step-down as the
election does not do anything (we're partitionned) and might be
harmful when we're re-introduced to the cluster: Our term will be
bigger than the actual recovered term, thus triggering an election for
nothing.
- Do not trigger an election if an election is already on-going on
dynamodb gray failures
- Do not trigger an election if timeout has been reached on
gray-failures, simply retry
- Changed the majority check to a single node check for leader count: If
a single node reports a sucessfull leadership, do nothing.
- Wait for at least a majority of responses to ensure a majority of
nodes is without leader (and none of the other ones have one) before
triggering an election.
Patch: https://lists.sr.ht/~ne02ptzero/libfloat/patches/58069
Signed-off-by: Louis Solofrizzo <lsolofrizzo@scaleway.com>
Acked-by: Florian Florensa <fflorensa@scaleway.com>
Acked-by: Saalik Hatia <shatia@scaleway.com>
libfloat: add new callback is_node_online
If implemented, it should return wether a target node is online or not.
If not, libfloat will not send any log-data to it, but simply hertbeats
until it comes back online.
Updated the core-logic of libfloat_send_append_entries to reflect that
change, and added a small check to not send any data to a node we've
never heard from (yet).
Patch: https://lists.sr.ht/~ne02ptzero/libfloat/patches/58069
Signed-off-by: Louis Solofrizzo <lsolofrizzo@scaleway.com>
Acked-by: Florian Florensa <fflorensa@scaleway.com>
Acked-by: Saalik Hatia <shatia@scaleway.com>
libfloat: add max_logs_size configuration token
Which limit the total size of the AppendEntries packet to be sent either
on log-replay or log-sync. This token defaults on 65KB, but the final
packet might be bigger than this:
- First of all, there's headers and overhead that are not counted in the
total limit
- The limit act as a send threshold, not an exclusive limit: We can't
cut a log in half.
Updated DEBUG to reflect this new logic.
Patch: https://lists.sr.ht/~ne02ptzero/libfloat/patches/58069
Signed-off-by: Louis Solofrizzo <lsolofrizzo@scaleway.com>
Acked-by: Florian Florensa <fflorensa@scaleway.com>
Acked-by: Saalik Hatia <shatia@scaleway.com>
log: Apply deep-sleep state from heartbeats / leaders instead of computing locally
We're now applying the deep-sleep state from the AEs/Hearbeats from the
leader, instead of each node computing it locally. This way, only the
leader computes it, and gossip this state to other nodes via AEs, and
those node simply apply it. Solve some issues of desync deep-sleep
timers seen on production.
I've also reworked the deep-sleep routines a bit, in order to have a
single entry-point for it.
Patch: https://lists.sr.ht/~ne02ptzero/libfloat/patches/50314
Signed-off-by: Louis Solofrizzo <lsolofrizzo@scaleway.com>
Acked-by: Julien Egloff <jegloff@scaleway.com>
Acked-by: Florian Florensa <fflorensa@scaleway.com>
________________________________________
/ Just because they are called \
| 'forbidden' transitions does not mean |
| that they are forbidden. They are less |
| allowed than allowed transitions, if |
| you see what I mean. -- From a Part 2 |
\ Quantum Mechanics lecture. /
----------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
raft: add soft snapshot feature
This patch add soft snapshot feature: if no logs are received for the
soft_compact_time seconds, a snapshot will be made based on the
soft_compact_after_n value.
Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>
log: Accept snapshots logs if the term is higher than our snapshot term
It's stucking replication on some clusters in production
Patch: https://lists.sr.ht/~ne02ptzero/libfloat/patches/41159
Signed-off-by: Louis Solofrizzo <lsolofrizzo@scaleway.com>
Acked-by: Patrik Cyvoct <pcyvoct@scaleway.com>
_____________________________________
/ "Nature is very un-American. Nature \
| never hurries." -- William George |
\ Jordan /
-------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
compilation: Add a build.zig compilation file
Working as intended, some small fixes in the code to compile with clang
without warnings.
Patch: https://lists.sr.ht/~ne02ptzero/libfloat/patches/41085
Signed-off-by: Louis Solofrizzo <lsolofrizzo@scaleway.com>
Acked-by: Patrik Cyvoct <pcyvoct@scaleway.com>
________________________________________
/ If I had only known, I would have been \
\ a locksmith. -- Albert Einstein /
----------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
election: Fix follower condition and be less strict on term
Revert "dynamo: Do not discard leader when timeout is reached when using leader-check dynamo like elections"
Fix no_leader count for gray-failures
Add no wake-up if the leader has been recovered from a gray-failure check
Load leader before deep-sleep states in order not to force a wake-up on restarts
Patch: https://lists.sr.ht/~ne02ptzero/libfloat/patches/40645
Signed-off-by: Louis Solofrizzo <lsolofrizzo@scaleway.com>
Acked-by: Patrik Cyvoct <pcyvoct@scaleway.com>