Project Proposer and Milestone Reviewer Perspectives

Project Builder Perspective

As a Project Builder, using the Milestone Module presents a new challenge to learn and use, but it feels like a move in the right direction. During Fund 10, in 2023, funded projects had to use the Milestone Module AND STILL DO monthly reporting. That was the worst of all possible worlds, and I felt like I spent more time reporting than actually working on projects. Starting in January 2024, all active projects that are using the Milestone Module ONLY have to report in the Milestone module when milestones are achieved. So if we plan our Milestones well, then they align well with the pace and the nature of the work we are doing, and Milestone Achievement reports feel like a fairly natural extension of the work at hand.

The Milestone Module forces teams to get explicit and organized about their project plan; if they haven’t done that work already, it might seem like an obstacle. However, having a clear project plan is a skill that is important to learn. When mastered, it’s incredibly helpful for the project team themselves, not just for the auditors. As experienced participants in Project Catalyst, the Lido team quickly learned that it’s good to be clear about the project plan in the original proposal itself. We and other proposers are now commonly including clear milestones and deliverables in our proposals. This has the dual benefit of pleasing voters, as well as translating easily into the Milestone Module if you win funding.

It also forces us to be clear and correct in reporting on our progress along the way. Every time we complete a Milestone, we must submit evidence, and this “Proof of Achievement” must be approved by two reviewers and the Catalyst team. On several occasions, we have been challenged to provide better evidence. This explicit approval process forces us to be more accountable, and to improve how we share and explain what we are building. It is iron sharpening iron, and I’ve been proud to learn how to submit excellent “Proofs of Achievement.”

Milestone Reviewer Perspective

As a Milestone Reviewer, I was assigned to 15 projects. At the beginning of the Fund Onboarding process, it felt like a lot. I had to familiarize myself with 15 projects and evaluate all their milestones. If each project had an average of 4 milestones, that means it was about 60 milestones to evaluate. For each milestone, I had to comment on its outputs, acceptance criteria, and evidence plan. Each of these elements could be approved or rejected, and each one required my comments and justification. If they were rejected by myself or another reviewer, the milestone would be revised and resubmitted by the project team, and the work would start all over again. So I conservatively had to review and write about 200 comments and justifications, over the course of just a couple weeks as the milestones rolled in. It was a little overwhelming, and given the overall level of other commitments in my life, I thought it a bit too much to be able to do an excellent job on each one. However, I really enjoyed participating and contributing. I think a future improvement might allow Milestone Reviewers some level of control over how many projects they want to advise on. A person who already juggles a heavy workload but wants to contribute might opt for a lower number, while a person who has time to devote and wants to work for the extra income could opt for a given maximum.

With Milestones finalized, I find myself excited for the next phase of the work, which would be expected to spread out over the coming year. As each project submits Proof of Achievement for each milestone, I will be prompted to approve or reject the proof. In either case, I must provide a rationale for my decision. If I reject the proof, I should provide an explanation of what I found lacking, and how the team can meet my expectations. My expectations of what I ask of them should also be clearly aligned with the Outputs, Acceptance Criteria and Evidence that was approved for that milestone.

In practice, I have already run into one exciting auditability puzzle! For one project, part of the “Milestone 1” deliverable was to establish the project team. As it happens, the project team they reported was the very same team that was named in the original proposal, so it seemed straightforward. Since I was personally interested in the project, I decided to connect with them on twitter. The person I connected with was unequivocally the same person listed on the proposal - but they had never heard of the project, and had no connection to Cardano at all. They had no knowledge of the other supposed “team members.” It appeared to be a matter of stolen identity. Since we haven’t gotten to the bottom of the matter, I’m not naming names, or projects. However, this unsettling discovery did lead me to “reject” the milestone. In my justification, I explained how I had learned of some irregularities in the supposed project team. Passing that milestone will now require a high degree of verifiable proof that the project team exists and are who they claim to be. Depending on what is discovered, the project could face consequences, including possible termination. Regardless of how this particular scenario plays out, it has been a good experience as a Milestone Reviewer to learn how to effectively examine the evidence provided to see if it stands up to a little poking and prodding.

In conclusion

The conclusion of this story has not been written. With Fund 12 launching in the next month, the Catalyst Team has announced that more layers are coming to the accountability framework. The community, including yours truly, will be eager to jump in with opinions, complaints, feedback, advice, and ideas, in no particular order. It’s all a bit messy, but it’s exciting too. We hope you’ll be a part of it - because the future is for everyone!