Delete plz topic closed

the fox must be here.

Hey,

Have you read https://arxiv.org/abs/1807.04938 ? It mostly answers your questions.

Why TCP? Tendermint is designed for Gossip Networks not point to point networks. It might make sense to have a FIBER like block relay network at scale eventually but TCP is a good fit our target scenario.

Why no view change? Cause Jae invented a novel termination mechanism that eliminates the need for the view change.

You couldn’t be bothered to read the paper I just linked to could you? It has • Formal specification • Consensus proof for Tendermint. • A complete bibliography. Of course the Gossip model vs the TCP model matters, PBFT models a network topology where all the nodes are directly connected to each other. The consensus protocol can provide the fault detection and recovery instead of doing it at the transport layer. Tendermint works in a model where there is a heterogeneous mix of validator nodes and full nodes in the network. Under normal conditions, we don’t expect to have any sort of direct network link between validators nodes. Validators nodes are also gossiping about many things other than consensus like new transactions, evidence of Byzantine Faults etc. It make sense to only have one p2p layer for all that. Tendermint also is designed to operate over the internet not enterprise networks. This means our communication protocols need to be as friendly to middleboxes as possible. While it is possible to deploy new internet scale UDP protocols like QUIC, Google has been working on this for years and run into numerous challenges. There is certainly room for…

Excerpt (1188 of 1282 characters). Read the whole post on the forum ↗

the fox must be here.

Okay I think we moved past the complaints of lack of intellectual rigor in Tendermint consensus and into a networking stack design debate and your last comment sort of moves in the direction of a concrete proposal. Here is the logic what we send a peer when a peer is behind in the same height/round. github.com tendermint/tendermint/blob/c64a3c74c870d725ba1356f75b4afadf0928c297/consensus/reactor.go#L658-L710 ``` func (conR *ConsensusReactor) gossipVotesForHeight(logger log.Logger, rs *cstypes.RoundState, prs *cstypes.PeerRoundState, ps *PeerState) bool { // If there are lastCommits to send... if prs.Step == cstypes.RoundStepNewHeight { if ps.PickSendVote(rs.LastCommit) { logger.Debug("Picked rs.LastCommit to send") return true } } // If there are POL prevotes to send... if prs.Step <= cstypes.RoundStepPropose && prs.Round != -1 && prs.Round <= rs.Round && prs.ProposalPOLRound != -1 { if polPrevotes := rs.Votes.Prevotes(prs.ProposalPOLRound); polPrevotes != nil { if ps.PickSendVote(polPrevotes) { logger.Debug("Picked rs.Prevotes(prs.ProposalPOLRound) to send", "round", prs.ProposalPOLRound) return true } }…

Excerpt (1196 of 1805 characters). Read the whole post on the forum ↗

Hi, Thanks for taking an interest in our project and taking the time to raise your concern. You’re certainly right that we could/should do a better job of documenting this important design decision. First thing I would point out is that the most successful live BFT systems in the world today use TCP connections. These are of course Bitcoin and Ethereum. Note these are full production systems that run on an internet scale decentralized p2p network and handle many concerns beyond just active consensus - there’s peer exchange, broadcasting transactions, helping old peers sync, etc. Of course these are probabilistic systems, unlike PBFT and Tendermint. But note that even BFT-Smart, the apparent leading implementation of PBFT-like protocols besides Tendermint, also uses TCP. Correct me if I’m wrong, but I don’t believe Castro’s system has ever been used for or destined for production. We have very deliberate reasons for using TCP. We want to have connection oriented protocols with our peers because we have to multiplex many concerns over a single connection (transactions, peer exchange, etc.), as Zaki already mentioned. While of course we could use UDP, that would put significant…

Excerpt (1199 of 4158 characters). Read the whole post on the forum ↗

What I demonstrated in the application layer code is the unless you rewrite the application layer so that you don’t have to send POLVotes , head of the line blocking is irrelevant to Tendermint. The primary advantage of switching to a datagram oriented protocol from a connection oriented protocol is the ability of the application layer logic to control HOL blocking. What happens in Tendermint, the application layer also blocks on sending the first phase of the two phase commit so UDP doesn’t buy us anything.

the fox must be here.

Comparing the p2p stack to Ethereum’s is perfectly fine, since it doesn’t matter at the p2p layer. The UDP / TCP discussion in a gossip network is pretty independent of the algorithm used for consensus. Our requirements are message delivery guarantees, and multiplexing. You’re right that TCP provides message ordering (and associated overhead) which I don’t think Tendermint needs after the handshake. (We definitely want message delivery guarantees, and multiplexing is a nice plus) QUIC does this with UDP + message ordering per stream for several streams getting multiplexed together. It took google engineers several years to develop this. A proposal to create a new scheme based off QUIC (but perhaps without the message ordering) once QUIC itself is sufficiently standardized / deployed would be reasonable, though it would take tons of work to build. However such a thing does not yet exist today, and is not something we need to block the launch of the cosmos ecosystem for over a year on. TCP seems to me to be a good choice for the needs of tendermint given the existing infrastructure that exists today. Especially since TCP is already implemented on every system, and months don’t…

Excerpt (1195 of 2374 characters). Read the whole post on the forum ↗

youcantbeserious123: The ONLY case that it might make sense is to use TCP to propose blocks – and use UDP to run consensus. What is this “multiplexing” thing here? UDP can handle all sorts of requests with different ports. You can handle the type of the messages at the application layer. This sounds like progress. Multiplexing is for the many protocols we run in Tendermint. It’s not just consensus. We also help old peers sync blocks, we broadcast recent transactions in the mempool, we gossip evidence of byzantine behaviour, we gossip about peers, and we do this all over the same connection with the peer. It’s quite nice having TCP guarantees in all of these protocols. As for the consensus. In fully connected point-to-point networks, we all agree, it could be better to use UDP, because you don’t really care about dropping messages - every node sends messages to every other node and the PBFT mechanism handles the faults. That’s great. But in Tendermint, we expect the network to be not fully connected and for there to be many hops between nodes, which means we have to be more intelligent about what messages we send to who and the reliability of those sends. We can’t…

Excerpt (1197 of 2551 characters). Read the whole post on the forum ↗

Speaking as a network engineer, UDP has the following advantages: • Eliminates head-of-line blocking • Can avoid retransmission of stale data when messages are dropped These are both micro-optimizations over TCP, and there is presently no evidence either of these are bottlenecks in Tendermint. UDP has the following disadvantages: • Decongestion: where TCP provides built-in decongestion, decongestion with UDP is generally solved by userspace decongestion algorithms, which are tricky to get right and take years to develop. There are off-the-shelf solutions for UDP decongestion like DCCP, however performance results with DCCP vary wildly: sometimes it can provide performance improvements over TCP or UDP w\ userspace congestion, and other times it’s slower than TCP. It really depends on the particular problem and the particular DCCP stack. • Transport encryption with UDP is significantly more difficult. Replay defense in a datagram-oriented protocols requires explicit nonces and a sliding window protocol rather than a simple, implicit, incrementing nonce. DTLS is almost universally reviled for these reasons. The arguments being made in this post are grossly…

Excerpt (1189 of 1719 characters). Read the whole post on the forum ↗

Adding another point to the discussion: filtering TCP DDoS attacks is a well-understood problem, whereas it’s very difficult for UDP. Effective DDoS mitigation is inherently stateful, and this is much harder to get right for UDP protocols since each protocol has its own handshake and session handling mechanisms. Stateless UDP protocols like DNS, NTP, SSDP and (particularly annoyingly) memcached are also the reason the DDoS issue is as bad as it is. The QUIC protocol has been carefully designed to avoid both issues. I implemented a custom UDP DDoS mitigation system, so I’m well-aware of the tradeoffs at play. This is an important consideration for Cosmos, which is very likely to experience DDoS attacks (an attacker might attempt to attack each validator whose turn it is to propose, for instance). Middleboxes and ISPs also tend to treat UDP traffic worse than TCP and rate limit it much more aggressively. This is due to the DDoS issue and the fact that UDP protocols tend to have bad congestion control and won’t react to packet loss and ECN the same way TCP does (i.e. by reducing the window). UDP protocols are harder to troubleshoot - any competent network engineer knows how…

Excerpt (1197 of 2025 characters). Read the whole post on the forum ↗

I think the above thread demonstrates that there was a great deal of due diligence and thought that went into the design. There is also some excellent insight in the discussion about the relative merits of UDP and TCP for different applications. This discussion appears over so I’m going to go ahead and lock this thread.