Class KetchReplica
- java.lang.Object
-
- org.eclipse.jgit.internal.ketch.KetchReplica
-
- Direct Known Subclasses:
LocalReplica
,RemoteGitReplica
public abstract class KetchReplica extends java.lang.Object
A Ketch replica, eitherLocalReplica
orRemoteGitReplica
.Replicas can be either a stock Git replica, or a Ketch-aware replica.
A stock Git replica has no special knowledge of Ketch and simply stores objects and references. Ketch communicates with the stock Git replica using the Git push wire protocol. The
KetchLeader
commits an agreed upon state by pushing all references to the Git replica, for example"refs/heads/master"
is pushed during commit. Stock Git replicas useKetchReplica.CommitMethod.ALL_REFS
to record the final state.Ketch-aware replicas understand the
RefTree
sent during the proposal and during commit are able to update their own reference space to match the state represented by theRefTree
. Ketch-aware replicas typically use aRefTreeDatabase
andKetchReplica.CommitMethod.TXN_COMMITTED
to record the final state.KetchReplica instances are tightly coupled with a single
KetchLeader
. Some state may be accessed by the leader thread and uses the leader's ownKetchLeader.lock
to protect shared data.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
KetchReplica.CommitMethod
How this replica wants to receive Ketch commit operations.static class
KetchReplica.CommitSpeed
Delay before committing to a replica.static class
KetchReplica.Participation
Participation of a replica in establishing consensus.static class
KetchReplica.State
Current state of a replica.(package private) static class
KetchReplica.WeakRetryPush
Weakly holds a retrying replica, allowing it to garbage collect.
-
Field Summary
Fields Modifier and Type Field Description private KetchReplica.CommitMethod
commitMethod
private KetchReplica.CommitSpeed
commitSpeed
private java.lang.String
error
private long
lastRetryMillis
private KetchLeader
leader
(package private) static org.slf4j.Logger
log
private long
maxRetryMillis
private long
minRetryMillis
private KetchReplica.Participation
participation
private static byte[]
PEEL
private java.util.List<ReplicaPushRequest>
queued
private java.lang.String
replicaName
private long
retryAtMillis
private java.util.concurrent.Future<?>
retryFuture
Scheduled retry due to communication failure.private java.util.Map<java.lang.String,ReceiveCommand>
running
private java.util.Map<ObjectId,java.util.List<ReceiveCommand>>
staged
private KetchReplica.State
state
What is happening with this replica.private ObjectId
txnAccepted
Value known for"refs/txn/accepted"
.private ObjectId
txnCommitted
Value known for"refs/txn/committed"
.private java.util.Map<java.lang.String,ReceiveCommand>
waiting
-
Constructor Summary
Constructors Modifier Constructor Description protected
KetchReplica(KetchLeader leader, java.lang.String name, ReplicaConfig cfg)
Configure a replica representation.
-
Method Summary
All Methods Static Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description (package private) void
afterPush(Repository repo, ReplicaPushRequest req)
Callback fromReplicaPushRequest
upon success or failure.protected abstract void
blockingFetch(Repository repo, ReplicaFetchRequest req)
Fetch objects from the remote using the calling thread.(package private) boolean
canDelete(Ref ref)
private static ReceiveCommand
copy(ReceiveCommand c)
private boolean
defer(ReplicaPushRequest req)
private static void
delete(java.util.List<ReceiveCommand> cmds, java.util.List<ReceiveCommand> createCmds)
protected java.lang.String
describeForLog()
Get description of this replica for error/debug logging purposes.private void
doRetryPush()
private static boolean
equals(ObjectId a, LogIndex b)
KetchReplica.CommitMethod
getCommitMethod()
Get how Ketch will commit to the repository.KetchReplica.CommitSpeed
getCommitSpeed()
Get when Ketch will commit to the repository.(package private) static ObjectId
getId(Ref ref)
KetchLeader
getLeader()
Get leader instance this replica follows.java.lang.String
getName()
Get unique-ish name for debugging.KetchReplica.Participation
getParticipation()
Get how the replica participates in this Ketch system.KetchSystem
getSystem()
Get system configuration.(package private) ObjectId
getTxnAccepted()
(package private) boolean
hasAccepted(LogIndex id)
(package private) void
initialize(java.util.Map<java.lang.String,Ref> refs)
Update the leader's view of the replica after a poll.protected java.util.Collection<ReceiveCommand>
prepareCommit(Repository git, java.util.Map<java.lang.String,Ref> current, ObjectId committed)
Build a list of commands to commitKetchReplica.CommitMethod.ALL_REFS
.private void
prepareTxnCommitted(java.util.List<ReceiveCommand> cmds, ObjectId committed)
private void
pushAsync(ReplicaPushRequest req)
(package private) void
pushCommitAsync(LogIndex committed)
(package private) void
pushTxnAcceptedAsync(Round round)
Schedule a proposal round with the replica.private static ObjectId
readId(ReplicaPushRequest req, ReceiveCommand cmd)
private void
removeStaged(java.util.List<ReceiveCommand> cmds, ObjectId committed)
private void
retryLater(ReplicaPushRequest req)
private void
runNextPushRequest()
Determine the next push for this replica (if any) and start it.(package private) boolean
shouldPushUnbatchedCommit(LogIndex committed, boolean leaderIdle)
protected void
shutdown()
Called by leader to perform graceful shutdown.(package private) ReplicaSnapshot
snapshot()
protected abstract void
startPush(ReplicaPushRequest req)
Begin executing a single push.private void
updateView(ReplicaPushRequest req, ObjectId acceptId, ReceiveCommand commitCmd)
private boolean
waitingForRetry()
-
-
-
Field Detail
-
log
static final org.slf4j.Logger log
-
PEEL
private static final byte[] PEEL
-
leader
private final KetchLeader leader
-
replicaName
private final java.lang.String replicaName
-
participation
private final KetchReplica.Participation participation
-
commitMethod
private final KetchReplica.CommitMethod commitMethod
-
commitSpeed
private final KetchReplica.CommitSpeed commitSpeed
-
minRetryMillis
private final long minRetryMillis
-
maxRetryMillis
private final long maxRetryMillis
-
staged
private final java.util.Map<ObjectId,java.util.List<ReceiveCommand>> staged
-
running
private final java.util.Map<java.lang.String,ReceiveCommand> running
-
waiting
private final java.util.Map<java.lang.String,ReceiveCommand> waiting
-
queued
private final java.util.List<ReplicaPushRequest> queued
-
txnAccepted
private ObjectId txnAccepted
Value known for"refs/txn/accepted"
.Raft literature refers to this as
matchIndex
.
-
txnCommitted
private ObjectId txnCommitted
Value known for"refs/txn/committed"
.Raft literature refers to this as
commitIndex
. In traditional Raft this is a state variable inside the follower implementation, but Ketch keeps it in the leader.
-
state
private KetchReplica.State state
What is happening with this replica.
-
error
private java.lang.String error
-
retryFuture
private java.util.concurrent.Future<?> retryFuture
Scheduled retry due to communication failure.
-
lastRetryMillis
private long lastRetryMillis
-
retryAtMillis
private long retryAtMillis
-
-
Constructor Detail
-
KetchReplica
protected KetchReplica(KetchLeader leader, java.lang.String name, ReplicaConfig cfg)
Configure a replica representation.- Parameters:
leader
- instance this replica follows.name
- unique-ish name identifying this replica for debugging.cfg
- how Ketch should treat the replica.
-
-
Method Detail
-
getSystem
public KetchSystem getSystem()
Get system configuration.- Returns:
- system configuration.
-
getLeader
public KetchLeader getLeader()
Get leader instance this replica follows.- Returns:
- leader instance this replica follows.
-
getName
public java.lang.String getName()
Get unique-ish name for debugging.- Returns:
- unique-ish name for debugging.
-
describeForLog
protected java.lang.String describeForLog()
Get description of this replica for error/debug logging purposes.- Returns:
- description of this replica for error/debug logging purposes.
-
getParticipation
public KetchReplica.Participation getParticipation()
Get how the replica participates in this Ketch system.- Returns:
- how the replica participates in this Ketch system.
-
getCommitMethod
public KetchReplica.CommitMethod getCommitMethod()
Get how Ketch will commit to the repository.- Returns:
- how Ketch will commit to the repository.
-
getCommitSpeed
public KetchReplica.CommitSpeed getCommitSpeed()
Get when Ketch will commit to the repository.- Returns:
- when Ketch will commit to the repository.
-
shutdown
protected void shutdown()
Called by leader to perform graceful shutdown.Default implementation cancels any scheduled retry. Subclasses may add additional logic before or after calling
super.shutdown()
.Called with
KetchLeader.lock
held by caller.
-
snapshot
ReplicaSnapshot snapshot()
-
initialize
void initialize(java.util.Map<java.lang.String,Ref> refs)
Update the leader's view of the replica after a poll.Called with
KetchLeader.lock
held by caller.- Parameters:
refs
- map of refs from the replica.
-
getTxnAccepted
ObjectId getTxnAccepted()
-
hasAccepted
boolean hasAccepted(LogIndex id)
-
pushTxnAcceptedAsync
void pushTxnAcceptedAsync(Round round)
Schedule a proposal round with the replica.Called with
KetchLeader.lock
held by caller.- Parameters:
round
- current round being run by the leader.
-
copy
private static ReceiveCommand copy(ReceiveCommand c)
-
shouldPushUnbatchedCommit
boolean shouldPushUnbatchedCommit(LogIndex committed, boolean leaderIdle)
-
pushCommitAsync
void pushCommitAsync(LogIndex committed)
-
prepareTxnCommitted
private void prepareTxnCommitted(java.util.List<ReceiveCommand> cmds, ObjectId committed)
-
removeStaged
private void removeStaged(java.util.List<ReceiveCommand> cmds, ObjectId committed)
-
delete
private static void delete(java.util.List<ReceiveCommand> cmds, java.util.List<ReceiveCommand> createCmds)
-
runNextPushRequest
private void runNextPushRequest()
Determine the next push for this replica (if any) and start it.If the replica has successfully accepted the committed state of the leader, this method will push all references to the replica using the configured
KetchReplica.CommitMethod
.If the replica is
KetchReplica.State.LAGGING
this method will begin catch up by sending a more recentrefs/txn/accepted
.Must be invoked with
KetchLeader.lock
held by caller.
-
pushAsync
private void pushAsync(ReplicaPushRequest req)
-
defer
private boolean defer(ReplicaPushRequest req)
-
waitingForRetry
private boolean waitingForRetry()
-
retryLater
private void retryLater(ReplicaPushRequest req)
-
doRetryPush
private void doRetryPush()
-
startPush
protected abstract void startPush(ReplicaPushRequest req)
Begin executing a single push.This method must move processing onto another thread. Called with
KetchLeader.lock
held by caller.- Parameters:
req
- the request to send to the replica.
-
afterPush
void afterPush(@Nullable Repository repo, ReplicaPushRequest req)
Callback fromReplicaPushRequest
upon success or failure.Acquires the
KetchLeader.lock
and updates the leader's internal knowledge about this replica to reflect what has been learned during a push to the replica. In some cases of divergence this method may take some time to determine how the replica has diverged; to reduce contention this is evaluated before acquiring the leader lock.- Parameters:
repo
- local repository instance used by the push thread.req
- push request just attempted.
-
updateView
private void updateView(ReplicaPushRequest req, @Nullable ObjectId acceptId, ReceiveCommand commitCmd)
-
readId
@Nullable private static ObjectId readId(ReplicaPushRequest req, @Nullable ReceiveCommand cmd)
-
blockingFetch
protected abstract void blockingFetch(Repository repo, ReplicaFetchRequest req) throws java.io.IOException
Fetch objects from the remote using the calling thread.Called without
KetchLeader.lock
.- Parameters:
repo
- local repository to fetch objects into.req
- the request to fetch from a replica.- Throws:
java.io.IOException
- communication with the replica was not possible.
-
prepareCommit
protected java.util.Collection<ReceiveCommand> prepareCommit(Repository git, java.util.Map<java.lang.String,Ref> current, ObjectId committed) throws java.io.IOException
Build a list of commands to commitKetchReplica.CommitMethod.ALL_REFS
.- Parameters:
git
- local leader repository to read committed state from.current
- all known references in the replica's repository. Typically this comes from a push advertisement.committed
- state being pushed torefs/txn/committed
.- Returns:
- commands to update during commit.
- Throws:
java.io.IOException
- cannot read the committed state.
-
canDelete
boolean canDelete(Ref ref)
-
-