Nothing came up for

Improving Integrity of CA Processes

$15,000.00 Received
$15,000.00 Requested
Ideascale logo View on ideascale
Problem:

カタリストの提案に対する(v)CAの評価では、最良の提案を確実に選択するために、可能な限り高いレベルの完全性を要求されます。

挑戦: Community Advisor Improvements
completed Awarded 15% of the fund.
コミュニティ・アドバイザー・レビュー:
4.56 (36)
Yes Votes:
₳ 97,518,810
No Votes:
₳ 2,225,358
Unique Wallets:
299

エクスペリエンス

Dr. Kenric Nelson, Photrek, is the past Circle rep. for the CA community and delivered the research on Diversifying Voting Influence.

Allison From, CryptoAlley, is the past Circle rep. for the General Voter and is leading a parallel proposal to develop CA digital identity.

ソリューション

Identify the community's best improvements on reputation and ranking and integrate these into the parallel effort on CA digital identity.

[IMPACT]

The assessments completed by Community Advisors and the quality assurance completed by veteran Community Advisors is a linchpin in the Cardano Catalyst program. Voters rely on the (v)CA assessments to make critical decisions regarding which proposals to fund. Proposers depend on the objective impartiality of the assessments to assure that a) the valuable time they have contributed into writing proposals is treated fairly and b) that the reviews provide them with meaningful paths to future improvement. As such, the integrity of the (v)CA process is critical for the long-term success of Cardano’s development efforts.

During 2021, Cardano Catalyst rapidly grew into the largest DAO in the world. During this rapid growth, intense scaling of the (v)CA capacity was a principal focus. With this rapid growth came systemic weaknesses which now need to be given high priority to resolve. These problems, as documented in the Project Catalyst Iterative DropBox v2, include individuals registering duplicate Ideascale accounts, poor quality submissions requiring rejection, automated assessments lacking in informative value, a ranking and rewards system that biases toward favorable reviews, bias in the selection and completion of proposal assessments, coordination among the (v)CAs which may be perceived as collusion, and other actual and potential threats.

The Improving Integrity of CA Processes (iiCAP) will seek to prepare for implementation critical improvements by a) partnering with CryptoAlley’s development of a CA digital identity program, b) partnering with one or two of the best initiatives documented in Iterative DropBox, and c) evaluation of alternative ranking methods to remove bias toward favorable reviews using methods such as Quadratic Voting (QV).

The iiCAP project will focus on the following KPI contributions to the CA Improvements Challenge:

  1. Build a new tools for the integration of CA reputation with digital identity. KPI: Specification of reputation method and collaboration with digital identity projects

  2. Foster a culture of continuous improvement in maintenance of CA processes. KPI: number of community meetings attended to mentor improvements

  3. Complete a comparative evaluation of quadratic voting as a method for scoring proposals. KPI: Increased variance in scores as an indicator of an increase in the information content of the rankings regarding selection of proposals

  4. Identify and support the community's best initiatives in developing a reputation system. KPI: number of community leaders supported in developing out of the box ideas

  5. Provide recommendations regarding how improved reputation and scoring can facilitate improvements in the compensation algorithm. KPI: communication with IOG and Circle via documentation and meetings attended

  6. Developing a reputation system depends on the availability of a digital identity system. Nelson and Fromm have completed the Atala PRISM program, have submitted two DID-focused proposals, and are partnered with the ProofSpace team, who have F7 funding to transfer their HyperLedger Identity capabilities to Atala PRISM.

  7. Effective selection and integration of a dropbox proposal and proposal lead to partner with will be important. Nelson and Fromm are actively engaged in the CA process and will be continuously evaluating proposed improvements.

  8. Modifications to the assessment scores and compensation need to balancea) ease of implementationb) difficulty of training andc) effectiveness in equitable scores and compensation. To facilitate the experiments with Quadratic Voting, Nelson is advising the Snapbrilla on Open Source Ecosystem proposal and is participating in the RadicalXchange community.

[FEASIBILITY]

iiCAP will be executed as a four month project, tentatively scheduled for May thru Aug 2022 spending on the actual funding start time. Throughout the four months, three objectives will be pursued in parallel:

  1. Development of a CA Reputation System integrated with a digital identity
  2. Identification of and support of the best CA community initiative as described in the DropBox
  3. Evaluation of the risk and benefits of using Quadratic Voting to rank assessments

Month 1: Launching CA Integrity

* Define the reputation model for CAs

* Integrate into the project, the leader of a promising CA initiative regarding integrity

* Specify the protocol for comparative ranking of proposals using QV.

Month 2: Integration with DIDs

* Coordination with ProofSpace regarding the DID capability

* Specification regarding reputation, integrity, and ranking with a DID capability

Month 3: Evaluation of CA Integrity initiatives

* Complete experiments prototyping the reputation, integrity, and ranking initiatives

* Determination of the cost/benefits to implementation of recommendations

Month 4: Coordination with Community

* Completion of Final Reports (written and video).

* Meeting with IOG/Circle/Community to review implementation recommendations

* Plan future steps to ensure the realization of the recommendations.

The deliverable outputs for the project:

* Recommendation for reputation system with cost/benefits for implementation.

* Support for a community leader in implementing their idea for improving integrity

* Evaluation of the feasibility to improve the ranking system.

* Coordination between stakeholders regarding path for implementation.

* Final reports that will enable community to continue development and implementation.

In addition to the principals, Kenric Nelson and Allison Fromm, the project will employ a veteran Community Advisor and a developer. The vCA will be someone who has documented an important innovation for the integrity of the CA program that requires funding to prototype. The developer will have experience in Atala PRISM and or Plutus to provide guidance on tool development needed to realize the capabilities.

The $15,000 high-level budget breakdown will be:

Photrek, $9000, Kenric Nelson and developer, responsible for project management, defining and developing the reputation enhancing methods.

CryptoAlley, $2800, Allison Fromm, responsible for integration path with digital identities

vCA and assistant, $3200, responsible for prototyping an innovation for the improving CA integrity

The funding will be allocated evenly over a four-month period:

Month 1: Launch CA Integrity, $3750

Month 2: Integration with DIDS, $3750

Month 3: Experimental Evaluation, $3750

Month 4: Coordination with Community, $3750

Kenric Nelson, Ph.D.

Scientist Engineer Educator * Executive | Photrek

Innovative leader in the research, development, and deployment of systems for complex decision-making. Proven record of creative research, team building, and customer-focused development spanning materials, electro-optic devices, signal-processing algorithms, and AI/ML decision systems.

Cardano experience includes initial investment in 2017, ITN stakepool operator, F4 Diversify Voting Influence, Catalyst Circle v2 representative for Community Advisors, F7 Community Governance Oversight, and Forecasting Cardano Native Tokens.

<https://www.linkedin.com/in/kenric-nelson-ph-d-7495b77/>

CA/vCA discussion:

https://www.youtube.com/embed/eawir33lhzg?start=&amp;t=97svCA Q&amp;A:

https://www.youtube.com/embed/tE1ZVaMBLp4?start=&amp;t=3sAllison Fromm

Managing Director at Crypto Alley

A leader with a track record of strategic planning, sales development, risk mitigation, and client relationship management in both the blockchain and corporate banking sectors. Catalyst Circle v2 representative for General ADA Holders. F7 Community Governance Oversight.

Skills include: leadership, financial analysis, problem-solving, development of strategic partnerships, compliance, and networking.

<https://www.linkedin.com/in/allisonfromm/>

[AUDITABILITY]

Our quantitative indicators will be:

  1. Number of Dropbox ideas moved from Draft - Finalized to Under Review, Approved, and/or Implemented.
  2. Number of experiments completed to document potential impacts on reputation
  3. Number of influencers engaged in discussions regarding review and approval process
  4. Number of developers collaborating with project team members to contribute to and/or evaluate reputation methods.
  5. Number of new proposals submitted to the Iterative DropBox

Successful project completion will consist of three components:

  1. Having a CA reputation method leveraging DIDs reviewed by IOG and the Circle
  2. Mentoring a veteran CA in shepherding their improvement idea towards formal review
  3. Provide experimental evidence of the cost/benefits of ranking by Quadratic Voting.

iiCAP is a new initiative that builds upon the work by Nelson to educate the Catalyst community about secure, equitable voting methods in the F4 Diversify Voting Influence project (https://drive.google.com/drive/folders/1V1dRJ0WydqFDv2Kmx_FByMgfrUNsHhhA?usp=sharing) and upon his efforts to improve the integrity of the CA process while he was the Catalyst Circle v2 representative.

The project will leverage capabilities developed in related F8 projects if funded:

PRISM DIDs for CA Onboarding: <https://cardano.ideascale.com/c/idea/399036>

PRISM DIDs for CA Process: <https://cardano.ideascale.com/c/idea/399040>

Cardano Quadratic Voting Funding: <https://cardano.ideascale.com/c/idea/396834>

SDG Rating

Goal 2. End hunger, achieve food security and improved nutrition and promote sustainable agriculture

Goal 4. Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all

Goal 5. Achieve gender equality and empower all women and girls

Goal 8. Promote sustained, inclusive and sustainable economic growth, full and productive employment and decent work for all

Goal 9. Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation

Goal 10. Reduce inequality within and among countries

Goal 11. Make cities and human settlements inclusive, safe, resilient and sustainable

Goal 12. Ensure sustainable consumption and production patterns

Goal 16. Promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable and inclusive institutions at all levels

Goal 17. Strengthen the means of implementation and revitalize the Global Partnership for Sustainable Development

コミュニティ・アドバイザー・レビュー

Addresses Challenge

4.8 / 5
12 レビュー

Does the proposal effectively addresses the challenge?

コミュニティーレビュー (12)

Commenter gravatar

The proposal offers a way to manage CA quality through reputation and digital identity technology. I find this to be a good idea. This helps CAs work more seriously and prevent fraud cases. I agree with this idea. The idea is very suitable for the requirements that Challenge offers. I hope the project will be implemented to bring fairness and transparency.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_823
Total QA Ratings
7
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

As the CA/VCA work is strongly correlated to votes results, improving the integrity of the CA process is indeed critical for the improvement of the Catalyst process. Fund 8 opens the path to the build of a CA and VCA reputation model, and a research work is required to provide fuel to this change. This proposal aims a reputation method leveraging DIDs for CAs. I really appreciate how the solution is well detailed in this proposal: build of CA reputation tools, quadratic voting evaluation, seeks for community initiative about reputation systems. I strongly agree that this proposal will improve the Community Advisor process.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_78
Total QA Ratings
8
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

This meaningful proposal seeks to identify the community's best improvements on reputation and ranking and integrate these into the parallel effort on CA digital identity. In terms of impact, some issues that should be noted are as follows: In general, this proposal has great significance and impact on the sustainable development of Catalyst - the largest and most serious DAO today; Addressing the issue of increasing the integrity of vCA through a reputation system is, as the author states, of decisive importance. The proposers defined a suitable problem within the challenge which needs to be solved, explains it in detail with rationale; they demonstrate a certain understanding of the field they are discussing. In addition, the problem raised shows the authenticity and urgency to be solved in order to achieve the goals of the challenge in particular and the entire ecosystem in general. The solution has been well-written supported by a detailed roadmap with milestones and measurable metrics for measuring the success of this proposal, as a result, a transparent positive impact for the Cardano ecosystem in the long term can be seen. This solution shows the possibility of adding value to the Cardano ecosystem and scaling to address future challenges by further directions beyond the scope of this proposal. However, in my opinion, some content related to the reputation that is expected to be built should also be clarified, including answering some questions as follows: currently, there are many views and approaches when building reputation systems, so what's the team's approach? Why does the team think that approach is more reasonable than the rest?… If the above points are clarified, I think the proposed impact will be more convincing. Based on the given information, I can rate the impact of this proposal with a score of 4.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_679
Total QA Ratings
7
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

Based on the proposal objectives, it clearly appear that there's an impact on the Catalyst and Cardano, but also to the members (CAs & VCAs( who play a role from within. Those objectives include: Development of a CA Reputation System integrated with a digital identity; Identification of and support of the best CA community initiative as described in the DropBox; and Evaluation of the risk and benefits of using Quadratic Voting to rank assessments

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_478
Total QA Ratings
9
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

One of the key elements for voting a Catalyst proposal is the assessment provided by CA. By having an integral process to assess proposals it will enforce the CA community to perform the work more efficiently and improve quality. It is really great to see this proposal that promises to deliver such a solution.

The deliveries of this proposal are within the scope and completely aligned with the challenge. According to the objectives of the proposal, the delivered solution will add extra value to the Cardano ecosystem by leveraging DID capabilities of the Atala PRISM program, maintaining and improving CA processes and increasing fairness and reputation among CA, vCA and proposers and providing better ranking methods for the CA work. The proposed solution is clearly described, KPIs well defined and the clarity of the proposal does not leave any doubts to the reader.

I strongly believe that the goals of this proposal are fully aligned with the ones in the challenge so I rate it with a score of 5.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_614
Total QA Ratings
6
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

The proposal and proposers have considered the complexity of the CA process, and they are correctly addressing some of the issues in this process. It is an exploratory proposal, but it has tangible outcomes that can be introduced to improve integrity within the system.

The proponents could have clarified a bit better the options such as the Quadratic Voting and other terms, in order to make the proposal more understandable to voters who are not aware of the issues in the assessment/voting processes.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_386
Total QA Ratings
6
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

By improving the way vCAs do their assessments and enabling quality work output, improving upon reputation and ranking of vCAs, and integrating these into the parallel effort on CA digital identity the proposal effectively addresses the challenge of improving the intergrity of CA processes.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_1656
Total QA Ratings
9
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

This is another excellent proposal that strives to incrementally make the CA / VCA processes more efficient and better aligned with the overall Cardano community values and long term goals. Specifically, the proposers intend to build a new tool to integrate a mechanism for CA reputation with digital identity.

Problem identification is spot-on, since the assessments of CAs / VCAs are important quality assurance for the ecosystem, and both proposers and voters depend on them to make sure that funding capital is allocated efficiently. As part of this overall framework, the integrity of the CA/VCA process / persons behind is key in the long run for the model to function properly.

Hence the proposal perfectly fits with the challenge and would be an extremely useful feature to have within the community.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_1454
Total QA Ratings
11
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

The "problem statement" given is a bit to vague and unspecific. The actual problem statement of this proposal is better described in the [Impact] section as "With this rapid growth [of Cardano Catalyst] came systemic weaknesses which now need to be given high priority to resolve", which is closely aligned to this challenge. The proposer correctly lists a number of different "problems" that have been highlighted by the community, but it would have been better if the proposer had been more specific on which problems he intends to tackle, as tackling all these problems is not feasible for a 4 months project. Which of the problems listed would the the creation of a CA digital identity program tackle ?

Most (if not all) the problems regarding the (v)CA process are summarized in some way in the many proposals put forward by the community via Iterative Drop box, and its great that someone will finally make use of these to improve the CA process. The evaluation of an alternative ranking method solution, is perhaps not as directly related to the problem of improving the integrity of the system, but still a useful exercise for the community.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_125
Total QA Ratings
9
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

Strongly addresses the challenge, particularly the issue of CA reputation, which is highlighted in the challenge setting. In its focus on CA integrity, it recognises that the CA process is part of Catalyst's governance, as the challenge setting describes. It also represents an instance of a team of CAs coming together to propose a solution (the proposal team's idescale profiles show that both have been CAs, and one of them has represented CAs at Catalyst Circle), which is an approach favoured in the challenge setting. It is also aligned with the challenge in that it aims to reserch the costs and benefits of quadratic voting; an evaluative approach to new models is supported by the challenge setting.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_1157
Total QA Ratings
11
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

As a nascent approach to DIDs and a reputation system for CAs, I am all in. There are really 3 aspects to this proposal, not necessarily connected to or playing off of eachother, but fitting under the larger umbrella of the initiative, which is almost like an ad-hoc CA process auditing body. Under the name iiCAP, they are ogranizing parallel threads, the first being a framework for auditing and oversight for a DID/reputation protocol also seeking funding in Fund8. This oversight includes the design phase of the reputation system, so these two projects are closely intertwined. I also assume that this oversight would capture and likely partner with or co-develop any other reputation initiatives that are sprouting in the CA space at the same time. The second component is delivery of a detailed analysis of how to incorporate quadratic voting into the proposal assessment process in order to achieve more informative final scoring, particularly in the rankings represented by the "star system". The final component appears to be mentoring and training a CA who will be leading an initiative, selected from the community, to iterate a process from start to finish to fix some integrity related issue for the CA ecosystem. Presumably this training will include being onboarded to the DID protocol, though that isn't specified here. The end result (my phrasing here) will be an "integrity improvement" protocol, which will be presented as a report alongside a description of a reputation system, rooted in DIDs for security, that is able to express the results of the discovered protocols. I have my concerns about the need for this type of governance to be a discovery and not an imposition, but I also think this will be a good example of the kind of experimentation that can lead to that discovery.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_1193
Total QA Ratings
16
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

I believe that this proposal is directly in line with the challenge setting because it addresses some key problems in the current CA/vCA system. The proposers are advocating for and planning to implement new tools, such as a CA reputation system using digital identity. A reputation system is not only critical for the challenge setting, but it is also incredibly important for the future of cardano catalyst. As a community advisor, I have seen some of the negative effects of scaling as listed by the proposers, so I do agree that this proposal will provide a huge benefit to the entire CA/vCA community as well as the overall cardano ecosystem. While an identity system is not currently fully developed and ready for deployment, the proposers plan to work with the ProofSpace team to hopefully implement their HyperLedger Identity capabilities once it is brought to Atala PRISM. Additionally, the proposers will look into quadratic voting as a potential method for scoring proposals. This idea is another way to address the challenge setting, and, at the same time, can provide an important service to the community by bringing more depth to the proposal assessment stage of catalyst. Thus, voters would have a better understanding of what the scores mean since there would be a larger variance of scores/interpretations. Overall, this proposal addresses the challenge setting in multiple ways. (score given: 5)

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_310
Total QA Ratings
7
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Feasibility

4.5 / 5
12 レビュー

Given experience and plan presented is likely that this proposal will be implemented successfully?

コミュニティーレビュー (12)

Commenter gravatar

Regarding the plan and implementation time, the plan is very specific and clear about the work items and necessary time. I appreciate the proposer's way of planning. Regarding the budget, the budget provides a detailed spreadsheet for each item and specifies the person in charge of implementation and the salary appropriate to the job content. Regarding team members, the members have appropriate experience and skills to develop the project. I have seen the information posted on your Linkedin, very clear and complete.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_823
Total QA Ratings
7
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

Both members of the proposing team have been representatives in Catalyst Circle (CA Community and Voters rep.). By having done past research about voting processes and digital identity, they provide evidence of having skills necessary to perform the project. A detailed roadmap is given for their work, which will last 4 months. I would have liked a connection between this schedule and Catalyst Funds calendar (Fund 10 starts), as it would be easier for voters. Projects risks, budget breakdown and deliverable are well described.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_78
Total QA Ratings
8
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

In terms of feasibility, there are some issues that should be noted, including: First of all, this team profile is clearly described, showing suitable skills, experiences for implementation, especially their experiences in implementing Catalyst proposals funds since Fund 4, supported by reference/proof (documents/links,…) for verifying by the community. This proposal's goal is clear, realistic, and supported by reasons, along with a specific roadmap, some logical, measurable KPIs/metrics, and a budget breakdown for reviewing the detail of the implementation process. In similar, risks are clearly defined with some mitigation, this shows that the proposer has fully looked at what could happen during the implementation of this proposal, however, it would be better if some further explanation of how performance those mitigations would be. In addition, as analyzed above [impact], the contents aimed at clarifying the point of view and approach in building a reputation system should be made more convincing because in many cases, reputation systems are built without grounds for controversy (ex: similar cases at e-commerce platforms, ….) Based on the given information, I can rate the feasible of this proposal with a score of 4

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_679
Total QA Ratings
7
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

The proposer teams is formed by people who have been involved wit the project Catalyst for a long and have played a role within the community. This allow them to have a perspective of improving some elements based on the feedback they have hearing from the community members. This becomes then a gage for success of this proposal since it's based on the community improvement wishes. By proposing to create a system to identify CAs and evaluate their work, this will have an effect to the quality of assessments produced, since those who have a low rate, would be excluded from the process.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_478
Total QA Ratings
9
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

The heads of this proposal, Dr. Kenric Nelson and Allison From are both well reputable member of the Catalyst community and have a proven contribution to the project. Their experience in the world of blockchain is also confirmed throughout LinkedIn. The intention also to hire a developer and a vCA makes the team capable and flexible to deliver the described solution.

The budget is broken into identifiable tasks and the requested amount looks reasonable based on the experience and competencies of the proposers, the time span and the work to be executed by the team. The implementation plan is described in details and it gives a good overview of the tasks to be executed including a realistic timing.

Although the resources for the execution and their roles are mentioned, it would be nice to add some more conceptual details like a representation of the reputation model, a draft architecture showing the DID integration and the interfaces to be used to communicate with the community. This way the delivered solution would be more understandable for the voter.

The associated challenges and risks of the project are identified and well described including the corresponding mitigation actions.

Based on the above, I strongly believe that the implementation will be success so I rate it with a score of 5.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_614
Total QA Ratings
6
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

The chances are good, especially when we take into account the proposal's design and proposers' experience. However, the proponents could have considered the risks of not getting to a point of acceptance in the community, but this does not compromise the proposal, as it has a viable plan.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_386
Total QA Ratings
6
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

The 4-month plan and budget allocated for completion of the three main objectives - development of a CA reputation system integrated with a digital identity, identification of and support of the best CA community initiative and evaluation of the risk and benefits of using Quadratic Voting to rank assessments makes this proposal highly likely to be implemented successfully.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_1656
Total QA Ratings
9
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

Behind this proposal is an outstanding team of people with tremendous visibility and already very active in the community through various roles. I see very limited risks in terms of team, skills and experience and on the contrary, I think the names behind the project are very close to what you would call a guarantee of success at least in terms of delivering what they promised.

The budget would be heavily skewed towards developer and project management activities (Photrek), with 20% going to CryptoAlley for integration activities and 20% towards prototyping a functional system.

The project would be executed over a 4-months period, ending in August this year, with monthly milestones. I would like to point out that the third phase definitely caught my eye with activities comprising experiments with various models and a cost / benefit analysis, very strong indicators in my opinion that we might have a quality project delivered at the end, with the best chances of coming up with a sustainable model with limited adverse impacts and unwanted effects.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_1454
Total QA Ratings
11
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

Kenric and Allison are two well know members of the community and have a clear interest and motivation to improve the (v)CA process. Both have been elected by the community for rep. roles which speaks to their dedication and respect amongst the community.

A timetable presented is clear and shows "what" will be done, but the proposal is lacking a bit on the "how" The proposer states that he will develop a reputation model for CA's, but doesn't provided any details on what this would be based on. A reputation model has already been proposed by the community, that is currently under review -https://docs.google.com/document/d/1UUHEoI1QvXtg4tlToHuts5EcptRuPUX4BwNddgN02O0/edit, so how does your model differ and/or would complement this ? The "Month 2: Integration with DIDs" section is also not very clear. It's not clear at what stage of development the mentioned DID system is at and/or if this proposal is dependent on the "PRISM DIDs for Catalyst CA Process" proposal that is also seeking funding in this round (and what happens if this is not funded). A bit more details on these two items would strengthen this proposal

Regarding the budget, it's clearly detailed. The only issue you might encounter is that $3200 might not be enough to "convince" a vCA to join this project for 4 months, specially if this is being split between 2 people (vCA + assistant).

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_125
Total QA Ratings
9
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

The proposal is broadly feasible but I have some questions around risk. The plus side - the proposal is well planned. The deliverables are clear; each element of the proposed solution identifies a clear KPI; and although the exact dates are tentative, the overall structure is clear and sensible. The budget is similarly clear, and is broken down sufficiently (although a little more detail in some areas would not be a bad thing, e.g. the division of the payment to Photrek between project maangement and development). The risks that are identified are well mitigated. The explanation of the background to the proposal, as well as the background and skills of the proposal team, makes it clear that they have the in-depth understanding of the ecosystem that would be needed to realise this proposal; and particularly, the level of connectedness to the CA community (for example, the incorporation of ideas from the Iterative Dropbox) is positive in terms of community engagement. However, given that the CA community is very engaged and informed, I feel there is a risk involved in only consulting with them so late in the project. If they identify issues with the proposed solution, it will then be too late to resolve these, which could ultimately lead to project failure. Also, since reputation and integrity are emotive issues on which people have strong views, I feel that the proposal should outline more clearly some elements of what it will entail - for example, the "leader of a promising CA initiative regarding integrity" is not identified, and the proposal doesn't elucidate what this initiative is and what it entails. This means not only that it is hard to assess the proposal's feasibility fully, but also there is a risk of the project being derailed midway if people (either the CA community or the voters) discover that it is doing something unexpected which they don't like or agree with. (Also, as an aside, it's confusing why the "leader of a promising CA initiative", who is clearly part of the proposal and part of Catalyst, is not a member of the proposal team.) Also, I feel that the project tries to pack in too many very significant but essentially unrelated developments. Integrating CA reputation and DID is a huge and important change; and although this proposal is only a first step, it is signifiant. Quadratic voting is similarly a really important idea. In terms of winning community support, engagement and understanding of both ideas, I think it could be a mistake to cobble them together as this project does. It creates a lack of clarity around the overall feasibility of the project - what effect will it have on one element if the other element does not go to plan? Overall, I agree the project is feasible, but with these caveats.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_1157
Total QA Ratings
11
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

I think this team in conjunction with its did:prism partner Proofspace is ideally suited for this research. They have excellent communication and connections with the community, they have already delivered research on some of the ideas in this proposal, and they are and have been engaged in the work and processes of CAs so they understand all the various subtle dimensions that come into play. Their timeline is realistic and I like that they have broken the budget up both by roles serviced and by funding tranche. The way they have described their roadmap makes clear, month by month, how the project will iterate, when it will onboard its CA/trainee, when research moves from one topic to the next. The DID discussion is light, as this is more about capturing the process as it happens, and the actual process is described more in the team's other proposal. The main risks to this proposal revolve around its integration into other ideas currently underway in this ecosystem, and in another nod to the qualifications of the team, they are partnered or advising across most of those integrations.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_1193
Total QA Ratings
16
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

Given the information provided in the proposal, I believe that it will be successfully implemented. The proposers have the necessary experience in the specific fields that will be helpful in making this proposal happen. For example, Kenric Nelson has been an innovative leader in the research and development of complex decision-making systems, and Allison Fromm has experience with strategic planning. Since this proposal directly involves complex decision-making systems, I think that Nelson and Fromm will be great leaders for this project. Both also have experience from the perspective of CAs and past catalyst circle representatives. With regards to the timeline, the proposers have supplied a very thorough roadmap that is broken down by months (with a total project time of 4 months). It is very easy to see what tasks will be completed in which month. For example, the new CA reputation system will be defined in the first month, coordination with ProofSpace on the DID capability will happen in the second month, and so on. There are multiple tasks that will be completed each month, which is pretty ambitious, but I think that the proposers can achieve all of their tasks if they do not hit any unforeseen roadblocks along the way. With regards to the budget, the proposers have listed out their requested funds in a broken-down manner, which makes it very easy for me to see what the funds will be put towards. Although I do not have experience with this kind of development, given the scope of the project and the length of the project, I think that this budget is sufficient for the proposal. (score given: 5)

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_310
Total QA Ratings
7
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Auditability

4.4 / 5
12 レビュー

Does the proposal provides sufficient information to assess and audit progress and completion?

コミュニティーレビュー (12)

Commenter gravatar

The proposer provides quantitative indicators to measure the progress and development of the project. However, I would appreciate it more if you could give an expected KPI for each indicator. And I think you should add a quantitative indicator that shows the results that contribute to the improvement of CA quality. For example, as a result of applying digital identity and reputation assessment methods, how many % will the number of quality CAs increase compared to the current one? I very much look forward to the efficiency improvement this approach brings to the CA system.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_823
Total QA Ratings
7
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

The problem is clearly stated, as well as are the solutions (see impact rationale). This proposal is also related to 3 other projects, 5 Kpis are proposed, which are measurable and relevant enough to enable a good monitoring of the development of the project.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_78
Total QA Ratings
8
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

Regarding this proposal's auditability, there are some issues that should be noted, including: First of all, as can be seen, this team profile is clearly described, with reference/proof for confirmation. Second, the roadmap is clearly described. For measuring the success of this proposal, the proposer(s) choose some measurable KPI/metrics, including a number of Dropbox ideas moved from Draft - Finalized to Under Review, Approved, and/or Implemented; a number of experiments completed to document potential impacts on reputation;… however, in my opinion, these indicators should be clarified in terms of definitions as well as measurement methods. This proposal provided items structure of the requested budget, however, there is no explanation of calculation base [number of employees, number of working hours, labor price,…]. At the same time, as analyzed, a number of contents related to views and approaches in building component systems, including the reputation system, have not been clearly expressed. Based on the provided information, I am neutral with this proposal's auditability

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_679
Total QA Ratings
7
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

The proposal provides a consistent roadmap with milestones and within a certain time frame. It suggests clear quantifiable indicators in order to audit the progress and measure the success of the proposed solution. The components that will mark the project completion and its successful implementation are sufficient and well described.

For the above reasons, I strongly believe that the progress of a potential implementation is auditable and measurable. Therefore I rate it with a score of 5.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_614
Total QA Ratings
6
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

The progress of this project implementation would be followed through online patform where documents will beb store, ex. Dropbox. The roadmap is realistic and the deliverables quantified, But from previous experience, we have seen a lot of suggestions that haven't been taken into action with the Catalyst community. Wouldn't this be the same again? It would be a risk.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_478
Total QA Ratings
9
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

Yes, there's enough information to audit, to verify deliverables and to recognize the impact of this proposal within the CA process. Proposers have listed clearly the deliverables, and each activity for each month of the project. The budget is also detailed and reasonable. Therefore, a score of 5 to this item is fair.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_386
Total QA Ratings
6
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

The proposal names quite a few quantitative indicators to be used as KPIs to measure the project's success (number of ideas, experiments performed, devs collaborating etc.). In order to turn the project’s auditability into an excellent one (from a good one), I take the liberty of suggesting perhaps a few step that might be taken into consideration

  • as early as possible during the development / experimentation to try to onboard any external competence with experience in game theory, since such a high-impact project might have unwanted consequences in the long run if the right incentives are not correctly implemented for CAs / VCAs. Or in other words, finding a way of measuring not only the success, but also the non-appearance of certain adverse effects after the implementation.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_1454
Total QA Ratings
11
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

Auditability of the project progress is possible because the timelines for the various stages of work are clearly laid out. Quantitative indicators that lead to the achievement of the three main objectives outlined make the project progress trackable.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_1656
Total QA Ratings
9
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

The deliverables are well detailed in the [Feasibility] section and provide a clear expectation on the outputs of this proposal, that the community can follow up on.

The "Quantitative indicators" presented mostly make sense although the "Number of influencers engaged in discussions regarding review and approval process" will be difficult to measure as the community is already engaged in discussions of this nature. It's also not clear to me how this proposal will affect "Number of new proposals submitted to the Iterative DropBox". The community is constantly adding to the Iterative Dropbox, and the objective of this proposal is to move existing proposal from draft status (which is correctly identified as one of the measures).

The Success criteria described are just a repetition of the plan described in the feasibility section i.e. i) creating a reputation model, ii) integrating a vCA into the team and iii) output of the evaluation of the QV method. Success criteria should seek to quantify how the proposal positively impacted the integrity of the CA process e.g. Reduction in the number of Filtered Out proposals over the next funds following implementation of reputation model

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_125
Total QA Ratings
9
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

They have laid out eminently achievable milestones here, with the main blockers being slow uptake on the DIDs. Even with that possibility, it shouldn't hamper too much the design of the reputation system. There is some disconnect between the relevant KPIs here and the deliverables…almost all measurables really fall under the adoption of iiCAP protocols, which will be under constant evolution through the scope of this project. The deliverables, on the other hand, are a reputation sytem for consideration, a trained CA under an iiCAP protocol, and an analysis of quadratic voting for proposal comparative ranking (the star system overhauled). Rather than KPIs, the teams funding tranches will probably be released upon having some kind of percentage complete metric on these deliverables, to IOG and the community, perhaps as "draft, in-progress, finalization" (an interesting reflection of existing protocols in the iterative dropbox they reference). However, I have no doubt this team can report something to keep this project funded and on course, and completion, by delivery of the aforementioned items, is an easy to gauge metric of success.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_1193
Total QA Ratings
16
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

Auditability is good, with transparent and measurable quantitative indicators - although for a proposal on a topic that elicits such strong opinions, and affects CAs so significantly, I would like to have seen some attempt to audit qualitatively how CAs feel. (Not necessarily about the proposed solution itself, which is in a sense incidental to auditing a proposal to develop it - but about the process, and how engaged, involved and heard they have felt, how the process has gone.) The same is true of qualitative audit of the engagement process with IOG, another key stakeholder. Also, the audit metrics identified don't seem to reference how we will audit the integration of DIDs. I suppose we will just say "well, it's been done, DIDs are part of the proposed solution" or "it's not been done, and they're not" but it would be interesting to be able to have and record more audit information - how easy was it, how scalable is it, what pitfalls were there and how were they overcome, etc. Overall the proposal is sufficiently auditable, but I think there is room for improvement.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_1157
Total QA Ratings
11
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Commenter gravatar

Given the information provided, I think that this proposal will be auditable. The proposers listed three main components that would make this proposal successful if completed. First, a successfully created reputation system that uses DIDs reviewed by IOQ and the catalyst circle. Secondly, mentoring a vCA towards making their improvement idea ready for formal review. Thirdly, providing a report on the pros and cons of quadratic voting. All three of these main goals can be tracked by the cardano community to see if the proposal is successful. In addition, the proposers also listed out some good numerical KPIs regarding the number of dropbox ideas moved from draft to later stages of completion, number of reputation experiments completed, and the number of individuals collaborating with the proposers on developing the reputation system, just to name a few. One question I do have for the proposers is what format or platform will be used to share this auditing information with the cardano community? Will the proposers take time during a town hall to update the community? Will they use a gitbook page? As it is, I personally wouldn’t have time to go through each page that the proposers publish and check every little detail and cross reference it to see if it matches with their goals, so providing a summary of the completed items and KPIs at the end of the project would be very helpful. If the proposers could answer this question, it would strengthen the auditability of the proposal and make it more clear for me and voters to know how people can follow this proposal’s success. (score given: 4)

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_310
Total QA Ratings
7
QA Rating Outcome
人間性の確認

コメントを書く

Replying to

Monthly Reports

Thank you to the Catalyst Community for supporting Photrek's initiative to improve the CA processes. We look forward to working with the community.

Disbursed to Date
$15,000
Status
Still in progress
Completion Target
9/30/2022

The IICAP project launch has been successful. Paul Erikson and Michael Cenkner of Modig Development have joined the team to develop a process for demonstrating core competency in the PA/vPA processes. Snapbrillia will provide their quadratic voting tools to facilitate feedback from the PA/vPA community regarding each proposed competency. Snapbrillia's bounty system will be used to reward community members who participate in the review.

Disbursed to Date
$15,000
Status
Still in progress
Completion Target
9/30/2022
Attachment(s)

Photrek originally contracted with Modig Development for the development of the Study Buddy curriculum. Michael Cenkner, who is leading the effort, has decided to complete the work as an individual rather than through Modig. As such, the contract with Modig was ended and a new agreement was completed with Cenkner.

Disbursed to Date
$15,000
Status
Still in progress
Completion Target
2. In the next 3 months
Attachment(s)

Our work on the Study Buddy curriculum is documented at the website https://sites.google.com/view/study-buddy-catalyst-pa-projec/background?authuser=0

Disbursed to Date
$15,000
Status
Launched
Completion Target
2. In the next 3 months
EP1: 'd' Parameter
0:00
/
~1:00
1x