[PROPOSAL #687][Passed]Replicated Security 3rd Party Audit

Interchain Security 3rd Party Audit Update 16/02 Update 15/03 - Put Proposal On-Chain and changed the post below to be more in-line with what went live on the proposal. TL;DR - Commissioning Oak Security to conduct a third-party audit of the Interchain Security (ICS) code with a similar scope as to the audit conducted by the Informal team. This proposal aims to use community pool funds to commission a third-party audit for the Interchain Security code. This audit is to be conducted by Oak Security, one of the most-reputable auditors in the space. The quote provided is $102K and has a timeline of around 2.5 - 3.5 Weeks for the audit. They will require 50% upfront to start the audit. Therefore, it is in the community’s best interest to get the on-chain proposal process going. Since this is a community pool spend proposal, we want to ensure the community that the funds will arrive at the designated recipient by creating a multi-sig. The multi-sig should comprise of: • Jehan (Informal Systems) • Zaki (Core Cosmos Contributor) • Jacob (Notional) • Kai (Neutron) Breakdown of Fees: With this proposal, We (Simply Staking) will be the main point of contact with…

Excerpt (1197 of 4771 characters). Read the whole post on the forum ↗

It is imo always good to have another set of eyes on code which is meant to be as big as a gamechanger as Replicated Security is projected to be.

I am not sure which auditor needs to do this though, I hear often mixxed signals on the quality of audits.

I agree that any ad all security audits can only be a good thing.

Strongly support this idea.

Agree on this proposal, but should not be a blocker for launch

I also strongly support this idea, and would be curious to know the perspective of the consumer chains set to launch about

Damien:

Whether ICS code should be enabled before the audit is completed or not

As for the other points you raised:

Damien:

• The form of the proposal. This includes how an auditor is to be chosen which could be done through mutual agreement in discussions, list and poll method or even through an ecosystem wide tender for work.

• The funding mechanisms for the audit. How much the auditor should be paid and how the funds are managed and distributed should be clearly defined beforehand.

• Criteria for the audit. The auditor should be given an idea of what they would need to audit. This can be done by a person/entity with the option of getting paid for it.
  Formation of an Audit Committee.

Would it be appropriate to open up a public working group to hash those points out via synchronous calls, and then proceed to publish minutes to the forum for others to see and comment on the outcomes? It seems to me that synchronous comms about these items would be more effective and efficient than trying to do this asynch.

some months ago i tried to push that a part of the game of chains incentives’ funding should be used for external security checks,

then glad you made this proposal ! strong support.

Strong supprot for using community pool funds for additional third-pary audits!

Agreed, it should not be a blocker for launch but as supplementary security.

Agree it should not block the launch. However, we may want to speed up the process!

While I agree that it would be ideal to have this done before the launch, it is probably not realistic. We need to ensure everything is done proper with all the proper checks and balances in place. Also, we need to ensure that the auditor which will be chosen is up to the job to carry out this audit.

An audit of this scale can also take some time.

So here are some thoughts: • Informal is the best or second best auditor in cosmos, and my opinion is that they’re the best • Our team has worked with their audit reports before. Basically, on cleanup of their findings. This particular audit was not even done using their TLA plus techniques, it was a fully manual audit, which I believe greatly benefited the chain that was audited by making it more secure and also by paying attention to detail like code readability. • The osmosis team would be my second choice but I strongly believe that their workload is too high. Remaining options • Entersoft - avoid at all costs • Oak - mixed feedback but generally positive • Halborn - seems to rely mainly on the codeql tooling built by crypto.org • Certik - seems to mainly focus on readability without deep focus on logic So those are all of the options that I am aware of. Personally, I recommend informal to people who ask me about audits. Notional is offering audits, but there are caveats: • We have not built out report tooling the same way that informal has, but frankly we do want for our tooling to mirror theirs. It’s really good. • We have massive experience…

Excerpt (1196 of 3649 characters). Read the whole post on the forum ↗

Generally support the idea. Two heads are better than one. There can never be enough audit.

1 idea i had for years is to create some sort of national audit system. A system where each citizen (token or staked token holder) would be able to review upgrades in the lightest way possible. Of course for a reward in governace tokens. This process would be automatic and easy. I have actaully seen an ETH project work on this. But i lost who it was,

What about Notional and Jacob? The guys are well skilled to do this… Maybe we can hire them?

Notional is not really a 3d party, right ?

Maybe a dumb question but, what about something like code4arena ?

We are definitely a third party in this case. Basically we would be auditing code from informal.

The other thing is just that there are very very few qualified auditors.

I don’t know the company that you just mentioned, maybe they’re a possibility and maybe not.

Currently, we do this on an unpaid basis. Look at the volume of our code contributions to the cosmos hub, and compare it to literally any other validator that does not have a contract to work on the hub or SDK.

yeah i’m sure notional is qualified to review cosmos-kind-code. no doubt about your contribs as well.
i may have a definition of 3d party that is not the good one.

to me 3d party is some entity that have 0 tie with environment it has to review.

Hence my proposition to pay you

I would support probably any party to fulfill such an audit. A quote from a non-cosmos-specialized entity will probably cost a significant amount but willing to get a quote from various auditors (including and excluding the list Jacob posted above) as i have connections with some of them.

Example of pricing can be found here: StarShell Security Audit Funding Request - Secret Governance - Secret Network

This is a small wallet and their codebase audit quote was 20k from Certik and 100k+ from Least authority.
I am sure ICS would go into the 150k-400k ballpark.

Hey everyone,

Thanks for all the discussions that have been going on.

Now that we have seen that there is a need and want for this audit, we would like to shift the discussion towards finding the right auditor to carry out this task.

I can see that @jacobgadikian has shared some names with us including:

• Oak

• Halborn

• Certik

• And even his own team - Notional

There are also other auditors such as:

• Least Authority

• Trail of Bits

We are aware that these audits can carry a hefty price tag, especially with the scale of the audit needed. Thanks to @Ertemann for sending over that very informative post from a different forum.

There is also the need to outline and finalise a set of criteria regarding what we want from this audit and the auditor. This should be done before approaching any auditor for quotations.

With this in mind, we would like to gauge interest in having a Twitter Space (in around a week’s time) as a place to discuss this further as @ala.tusz.am mentioned regarding a working group. This will have a focus group type setting where everyone is allowed to discuss and after I will post the minutes/important points from the talks.

I think that this thread is going to form a very valuable resource for information about audit organizations who are capable of auditing Cosmos code.

I would like to formally and bluntly request that we rule out any auditor who has not previously worked on Cosmos code because I do not believe that they would be able to make an effective audit on a reasonable timescale.

Is a contest between auditors feasible ?

I’ve seen Axelar ($50k - april2022), and Gravity ($100k - august2021) doing this that way.

I honestly don’t know if it’s efficient, but i think it’s worth exploring this path. Isn’t it ?

I don’t really think so and I think that we’re already talking about a higher ticket price

Yeah yeah, it has to be way higher for what we’re talking about here.
Was just curious about the concept.

Fair points raised here. I’m in agreement with what you’ve said and we should focus on those who have some familiarity with the Cosmos stack.

I think having a twitter space to iron out some of these details would be beneficial given the timeline we need.

Finally a potential CF allocation that makes a lot of sense.

As long as there are no conflict of interest, the auditor is chosen publicly and propositions of all potentials auditors are publicly shared can be seen by anyone.

The only issue I have is the timing with ICS release, we’d need to make sure it won’t block any process/ put any disorder in all current work done.

We are trying to put it forward in a way that won’t inhibit any progress or timelines for ICS. There already was an audit conducted on ICS by a separate team at Informal so this 3rd party audit would just be supplementary. Always better to have a second look-through and opinion on the code especially of this scale.

I think we may have missed something so far: if there is to be a 3rd party audit of ICS, it will require some degree of participation from Informal to submit the material, provide additional context to the auditor and possibly implement bug fixes.

@jtremback would your team be ready to collaborate with a 3rd party auditor if an agreement can be reached for the Cosmos community pool to sponsor the audit?

Yea, we can collaborate on the audit of course. The report from the Informal audit team should be done very soon and the 3rd party auditor can use that as a basis to determine what to look into further.

Thanks Jehan, that’s great to read! @Damien should we pick this up and move forward with the last preparations and proposal?

Yeah - preparations are underway. Expect an update very soon for the proposal

Great news! Thanks for leading this initiative.

I support this proposal because third-party auditors offer a broad perspective and knowledge of best practices for conducting audits. They maintain ongoing, up-to-date awareness of current and pending regulatory requirements that the project should consider.
But the crucial point here is that we have to be very serious about the choice of auditors and here I agree with Jacob’s thoughts.
So the variants are:
Oak
Halborn
+Notional

Long due update here. We’ve been working really hard to make sure everything is good to go from our end. In the time since we last posted and after taking in some of the discussions from those who contributed, we have opted to go for Oak Security to conduct the audits for the Interchain Security (ICS) code. We believe given their experience that they are currently the best suited. You might ask why haven’t we gone with an open-tender proposal instead of us choosing a validator? Well, the answer is simple really. That process is too time consuming and currently the pool of possible auditors in the space is rather limited so this was the best approach. Given some more time when auditors will join the space, we would have surely gone with that approach. This proposal aims to use community pool funds to commission a third-party audit for the Interchain Security code. This audit is to be conducted by Oak Security, one of the most-reputable auditors in the space. The quote provided ranges between $85K - $120K and has a timeline of around 2.5 - 3.5 Weeks for the audit. They will require 50% upfront to start the audit. Therefore, it is in the community’s best interest to get the…

Excerpt (1199 of 3085 characters). Read the whole post on the forum ↗

just to confirm - this audit will not delay proposing v9 upgrade this or next week/delay ICS launching?

Thanks for raising this point!

No this does not affect any timeline for ICS release and the v9 upgrade.

What is the expected amount of time required to earn that $18k with the 4 people listed?

I am always surprised by the amounts paid to people (and we always see the same people in the groups… which is kinda curious if you ask me). I do agree that we need to pay people properly. But now we have a proposal on chain to fund Notional for 3 years with a huge sum of money and I see Jacob in here being compensated as well. While the prop on chain secures the availability of notional for their hours spend on the Hub? So why pay twice?

@LeonoorsCryptoman Thanks for the contribution.

So the majority of the fee is going to be directed to us (Simply Staking) as we have spent many hours doing the necessary research, contacting the relevant parties for quotes and multi-sigs etc and we will also be the main point of contact for Oak during this process. We will also be in charge of releasing the report in an efficient and transparent manner.

With respect to your other point, we chose people who we trust and have a good rapport with in order to streamline the process we would need to collaborate with.

I hope this answers your concerns.

Thank you for taking the time to moving this situation further.

Can we get the official invoice as written by OAK security that shows the higher and upper bound?
Is there a reason there is a range and not a set fee?
Can we get information regarding the distribution of funds amongst the multisig members. I applaud you for being honest and taking something for yourself but am not clear in why the multisig members deserve a fee at this stage.
What is the guarantee that OAK wont just ask for the high amount because they know the community pool spend went through anyway? if this is the case the community pool wont get the return of the funds.
I assume payment is due in USDC, including a buffer will help realise that the payment can be executed.

Would be great if you could answer the above questions, with those answers provided you have our full support!

@Ertemann thanks for the contribution! Allow me to answer some of your questions and concerns: Ertemann: Can we get the official invoice as written by OAK security that shows the higher and upper bound? Is there a reason there is a range and not a set fee? I have asked for an official invoice recently, in the meantime I can supply you with what they told me in my exchanges with them: "Happy to provide a quote for an audit of ICS. As a rough first estimate, based on the current main branch of the GitHub - cosmos/interchain-security: Replicated security (aka interchain security V1) is an open sourced IBC application which allows cosmos blockchains to lease their proof-of-stake security to one another. repository and possible integrations into other repositories, we would require 2.5-3.5 weeks for this audit, with a fee between $85k and $120k." Ertemann: Can we get information regarding the distribution of funds amongst the multisig members. I applaud you for being honest and taking something for yourself but am not clear in why the multisig members deserve a fee at this stage. This is a fair question to be making. The multisig members are…

Excerpt (1199 of 2301 characters). Read the whole post on the forum ↗

@Damien any progress to report on this? Audit happening or paid for?

We’re finalising the contract with Oak. Expecting this to go up on chain next week!

awesome looking forward to it, has my vote YES

Thank you for the initiative. Support this.

Hi all,

Thanks so much for all the support so far. We’ve gone ahead and put the proposal up for voting!

Interchain Explorer by Cosmostation

Interchain block explorer and data analytics for sovereign blockchain networks.

I concur with the proposal put forward. We have already conducted an internal audit, and I believe that engaging a third-party auditor would be highly beneficial. Doing so would increase the confidence of larger teams in joining the ICS initiative.

However, I recall that Jacob from Notional had previously stated that they would provide audit services to teams within the Cosmos Hub, and that the community had funded Notional with the aim of safeguarding the Cosmos Hub. It may be worth exploring whether the proposed audit falls within their scope of work, and if not, obtaining a quotation from them.

To summarize, I remain uncertain about the selection of the proposed auditor and suggest that we consider other options, including the possibility of working with Notional, given their previous commitments and the community’s support.

Thank you for your contribution @wassie!

With respect to the choice of auditor, we spent quite a while looking at different options and speaking to different people to gather their input.

From what we saw, OAK Security were an experienced firm who have done many audits in the industry and for the magnitude and importance of this audit, we needed someone who has the necessary tooling and experience at hand to deal with this.

Notional’s proposal was a good one but even Jacob had mentioned that they never had completed an audit of this size and scope before.

I agree, we need this. But it is expensive service. Hope it is worth it!

I’m always for improving security and checking for vulnerabilities. Curious to understand why Oak was selected over other third-party auditors though. If I missed it in the forum please link me

Since the proposal is now live and with the imminent launch of Neutron’s proposal to be onboarded as a consumer chain, I just want to re-iterate the point that this proposal/audit WILL NOT have any effect on Neutron’s timeline.

Hey @Othman - Happy to answer this. I mentioned in a reply above that the reasoning behind choosing Oak was because of discussions we had with multiple parties.

The general sentiment was that in the Cosmos Ecosystem there are a limited number of experienced auditors. Usually people would recommend Informal - but since they already conducted the initial audit, we asked around to see what people would recommend after Informal. The majority said that OAK Security were an experienced and reputable firm to handle such a task having done audits for Stride, Mars, Persistence, LIDO and more. So after seeing this, we opted to engage with them.

You can see what they have audited in the past here: GitHub - oak-security/audit-reports

As can some persons see, there is a large conflict of interest of paying a validator to start an audit not committed by this validator.
This is not the role of a validator and it’s simple not ethical to ask a percent of the price of the audit.

I understand the time it takes but as I said, it’s just a conflict of interest you should not ask.

Regards, Valentin.

I have read this proposals good such a nice idea. But any appointments Project from me to approve it take Times. Because I need the real Confirmation from our community’s to started Processing all together"

Thank you for the info, Chainflow will be voting YES