Introducing the testnet incentive program
Introduction Hypha Worker Co-op currently stewards the Cosmos Hub testnet program, which was established over the course of 2023 and now includes two live gaia testnets, multiple consumer chain testnets, and regular Testnet Wednesday launch/upgrade events. In the coming months, we intend to build out the testnet program’s capacity for chaos and stress testing with the introduction of Wargame Thursdays, which will put the network to the test with production grade security attacks and testing. Testing for Interchain Security Over 2023, Interchain Security has become the core product offering on the Hub, yet its economics remain unsustainable for validators who provide the infrastructure required to support consumer chains. Like all networks, the Replicated Security testnet depends on the quality of its validator set. As a testing platform for the Cosmos Hub, it especially depends on validators who bring their Hub mainnet infrastructure and skills to the testing environment for both rehearsals and stress testing. Chaos testing and wargaming, in particular, benefit from a robust and highly participatory testnet as we seek to emulate mainnet conditions and see how the Hub will…
Excerpt (1198 of 5448 characters). Read the whole post on the forum ↗
lexa:
- Validators in the middle 33% by voting power will be eligible for an additional $200 USD for a total of $300 USD per month.
- Validators in the lowest 34% by voting power will be eligible for an additional $400 USD for a total of $500 USD per month.
To clarify, this is by voting power not rank. At this point in time, there are 7 validators in the top 33%, 15 validators in the middle 33%, and 155 validators in the bottom 33%
Looking forward to the Testnet Wednesdays.
Could you check the google form? Looks like we dont have access right now.
Should be fixed. Thanks for catching that!
I’m really glad AADAO was able to fund this initiative from the Hypha team. Incentivizing validators to ‘show up’ and do crucial prep work in prevision of Mainnet is a no brainer, especially in the context of Replicated Security which is a more costly process.
lexa:155 validators in the bottom 33%
The testnet incentive program is an interesting idea. Several months ago you suggested with @ala.tusz.am the UBI for validators running all consumer chains as an incentive to run consumer chains (https://forum.cosmos.network/t/scaling-up-with-conditional-basic-income). Why not fund a similar incentive program for this validator UBI? It could be tiered also with these 33% ranges for example. What do you think about this @jtremback @Youssef?
I had always intended Conditional Basic Income as a stopgap measure while waiting for further developments in cost reductions (which are now underway, as Jehan’s team researches things like atomic ibc and partial set security; and folks discuss new economic standards for consumer chains that might be coming to the Hub).
As long as we are doing a stopgap measure, I would rather get two birds with one stone and improve our testing ability, since this will also improve whatever new features are developed for ICS.
Thanks for the thoughtful feedback. I am in agreement with @lexa
I see conditional basic income (or universal basic income) as a temporary measure while Informal works on improving the current model to reduce the cost to run Replicated Security chains.
On the testnet incentive program above, we see it as a pilot from AADAO’s perspective and a step in the right direction. This pilot could potentially be renewed (and why not improved on) for a longer period and a larger budget once we hear back feedback from Hypha and the community at large.
Again, the best way to address costs of running consumer chains is to fix the root cause (which is closely tied to the inherent design)
hey @lexa can new validators join the testnet program?
Certainly, we want as many validators as possible. Feel free to message me about it.
Sure where can i connect with you to get some more details , here or discord?
Telegram or Discord. I’m @lexamichaelides on both.
Greetings, I think that program is a great idea!
One thing that is unclear: can you clarify the requirements for this part?
Run mainnet-grade infrastructure (nodes, sentries, etc) on all chains.
My concern is that my servers at home are fully utilised and there’s no way to run the testnet and all its consumer chains at home, so I was thinking of running everything at cloud instead, but the rest should stay the same (KMS, monitoring, security patterns etc.), so I wonder if that would make me eligible.
Also, would be nice to hear what is the list of the requirements for infra to be eligible in general.
We can’t reliably detect issues that might occur on mainnet unless validators are using their same mainnet infrastructure. The thing we are trying to avoid is having everything go well on testnet and then realize that we really hadn’t duplicated mainnet at all because mainnet involves hardware setups but testnet is entirely in the cloud. This is particularly important for wargaming, where we want to attack the testnet and see exactly what we might face if the same attack is performed on mainnet.
No, running things in the cloud does not meet the ‘mainnet-grade infrastructure’ requirement if you are not doing the same on mainnet. If you are operating bare metal on mainnet, you need to do the same for testnet to be eligible.
There are no hard requirements for eligible infrastructure aside from being the same as what you use on mainnet.
There have been some concerns about associating a mainnet validator key with a testnet validator key, so I’m introducing another way to verify that the team applying for the incentive is the same team operating the mainnet validator:
On mainnet Cosmos Hub, send a trivial amount of ATOM (e.g. 1 uatom) from your validator address to the multisig address cosmos1078gmgntta9qguve7c52dhxrs45xumjvcwrutd.
The memo field of this transaction must contain the cosmosvaloper address of your testnet validator.
If you have already submitted proof via the form using the delegate tx method, there is no need to submit proof again. Either of these methods is acceptable, but we recommend this mainnet-only one as it is simpler. The Google form and top-level post have been updated to reflect this.
I appreciate everyone’s feedback and patience as we work out the bugs in this pilot program!
Today marks the end of Period 1 (Nov 22 - Dec 20) of the pilot program! Over the last month, Hypha has tracked participating validators’ eligibility and refined the criteria based on feedback. Criteria • Be an active mainnet validator. Submit proof via our Google form • Validate all available consumer chains secured by the Cosmos Hub (Neutron and Stride). • Remain unjailed on the provider chain for the entire period (tracking available here ). • Participate in every Testnet Wednesday event according to event criteria (e.g. sign within five blocks of a major upgrade, sign by a designated block after an upgrade) (tracking available here ). • Run a testnet infrastructure setup that closely mimics mainnet. Note – for Criteria 5 , the following differences were counted as not meeting the criteria: • Disclosure via the Google form of “I am running different infrastructure or opsec on testnet and mainnet” • Running mainnet nodes on dedicated machines but running all testnet nodes on the same machine (including with containers) • Running mainnet nodes on bare metal but testnet nodes on VMs These decisions are made in the name of keeping validator nodes on the…
Excerpt (1192 of 3070 characters). Read the whole post on the forum ↗
Yesterday was the end of Period 2 (Dec 21 - Jan 24) of the pilot program. This was an eventful period with two upgrades (one of which was an emergency immediately following Neutron’s smooth mainnet emergency upgrade), one chain launch, and our first-ever game day! Overall, we had 51 validators participate in period 2 (although around 4 are no longer on the testnet – maybe they’ve left without saying goodbye). Criteria | Pass | Fail | 1: In the active set | 43 | 1 | 2: Validators all consumer chains | 42 | 2 | 3: Unjailed for entire period | 26 | 18 | 4: Participated in all events | 29 | 15 | 5: Running mainnet-grade infra | 39 | 5 | Eligible for payment | 16 | 28 | $7.6k USD was disbursed this period, equating to roughly 832 ATOM. All transactions are viewable from the multisig account wallet and emails have been sent out to all participating validators with more details. What to do if you were ineligible this period? • If you are joining the program, let us know using the Google form ! • If you are already in the program. check your email to see which criteria you didn’t hit. • Make sure…
Excerpt (1197 of 2719 characters). Read the whole post on the forum ↗
This week closed out Period 3 (Jan 29 - Feb 28) of our pilot program. We only had three official Testnet Wednesdays this period (one event moved to a Thursday so it didn’t count as a TIP criteria) but fit in nine different criteria. We’re down to 48 validators participating in the program (some have confirmed that they’ve left, others have joined). The feedback from validators who have left is largely that it’s still not financially viable to participate on the testnet if they must hit every criteria in order to get the incentive. This feedback will be incorporated into future iterations of this program if approved by AADAO, but will not change the criteria for the pilot program. Criteria | Pass | Fail | 1: In the active set | 46 | 2 | 2: Validators all consumer chains | 45 | 3 | 3: Unjailed for entire period | 44 | 4 | 4: Participated in all events | 21 | 27 | 5: Running mainnet-grade infra | 44 | 4 | Eligible for payment | 20 | 28 | $9.6k USD was disbursed this period, equating to roughly 818ATOM. All transactions are viewable from the multisig account wallet and emails have been sent out to…
Excerpt (1196 of 2001 characters). Read the whole post on the forum ↗
Period 4 ended this week. We’re getting to the end of the pilot period, which also means that we’ll soon be able to review our criteria and make adjustments based on feedback from validators. Many validators have offered us input on what makes the program valuable, what makes it a pain to participate in, and what would make it more valuable for less pain (ideal). Results this period were really promising – nearly all validators are meeting all criteria except the most challenging and changeable one – hitting every Testnet Wednesday event. Criteria | Pass | Fail | | 1: In the active set | 42 | 0 | | 2: Validators all consumer chains | 42 | 0 | | 3: Unjailed for entire period | 41 | 1 | | 4: Participated in all events | 24 | 18 | | 5: Running mainnet-grade infra | 41 | 1 | | Eligible for payment | 24 | 18 | | We paid out $11.6k USD this period (1084 ATOM) and transactions are viewable from the multisig account wallet . Emails have been sent out to all participating validators with more details. Last period, I reported that we had roughly $30k USD remaining. I have one correction to…
Excerpt (1196 of 1797 characters). Read the whole post on the forum ↗
In the midst of touring ISLE , period 5 of the Testnet Incentives Program has come to an end, Criteria | Pass | Fail | | 1: In the active set | 42 | 0 | | 2: Validators all consumer chains | 42 | 0 | | 3: Unjailed for entire period | 35 | 6 | | 4: Participated in all events | 30 | 12 | | 5: Running mainnet-grade infra | 41 | 1 | | Eligible for payment | 29 | 13 | | This is our highest number of eligible validators yet – practice makes perfect and many of these validators have been with us since before TIP was even happening! We’ve paid out the entirety of our remaining 1596 ATOM from AADAO this period. Due to a low ATOM price, it unfortunately meant that validators only received about 85% of the expected amount. TIP has been renewed by AADAO with different eligibility criteria for ongoing use, so I’ll be back on the forum to report on our findings from this pilot period, our new criteria, and what the Hub’s ICS testnet will look like going forward. Right now, we are still running the Interchain Security Lightning Experiment on the testnet and it’s not too late to join and earn…
Excerpt (1193 of 1241 characters). Read the whole post on the forum ↗