This proposal addresses the CA improvement mechanism , reporting clearly defined problems that match the challenge . The main part of this project is building and attracting people with expertise to contribute to the development of Cardano . this will increase the positive impact for the Cardano ecosystem and possibly scale the development to address future challenges. In order to complete the above idea, the proponent has provided solutions that have defined reward and punishment mechanisms, but I need a more detailed explanation of this problem, more specific examples are suggested. . The proposed solution I fully believe will solve the problem of the challenge, describe the solution in great detail as for financial purposes, some CAs use bots to evaluate projects. , leading to shoddy reviews. a proposal always goes hand in hand with success and risk, the proposed team has come up with some risk identifications such as capital sources, inflation, recession, let me ask if the technical risk factor is What will happen to the proposal and how to fix it. I also highly appreciate the proposal that also outlines risk reduction options. good luck project

The team has very well-known in the CA community with past reviews vCA and CA experience. The solution is proposing the reward & penalized mechanism for CA review which is really understanding the game theory and the motivations of the CAs to go for quantity instead of quality and helpful reviews for the community. The solution is the database API pools to track and then have the incentivized and penalized scheme. Thus, the team should be considered with dRep current approach to have incorporated the more experienced vCA or DRep at the combined solutions to improve the quality of the CA communities. One positive thing is that the team well-thought about the risk of funding and at the end of the date is the CA rewards/penalty is such at the later stage after getting funded. The risk of war and economic recession is a bit overthinking given in the realm of the Cardano blockchain.

The CA improvement challenge is about improving the impact the Catalyst ecosystem relies. The bigger diversity and opinion of different background the more wholesome the community becomes. This challenge aims to regulate and limit the interaction of a wide backgound of CA's to a small micro bubble around their expertise. The proposed challenge would infact redcue the positive impact of the CA in catalyst and render the exercise limited to a certain experts reviewers and the typical newcommer would find difficulty joining in, thus rendering the CA a very limited role within the community.

This proposal will attract people with expertise and desire to contribute to the development of Cardano by building reward and penalty mechanisms for CA. CAs that do well will be rewarded and vice versa. Direct targeting of financial benefits under this proposal will push CAs to further improve themselves to meet the requirements from there will reduce fraud and poor-quality reviews. I totally agree this proposer will effectively solve the challenge.

The main point of this proposal is to create a reward/punishment system for CA to attract people with skill and a willingness to contribute to Cardano's progress. As seen, the statement of the problem is clear and accurate for the given challenge. Further to that, the solution is well-defined, with convincing short-and long-term potential benefits for the Cardano ecosystem. However, there are some aspects that, in my opinion, this team should reconsider. Firstly, I agree that the CA's professional/expertise in relevance to the respective proposal's area of ​​expertise, which he/she assess, is important; it affects the reliability of their assessments, as a whole. However, the classification of the area of ​​expertise is various between different communities/countries, or in other words, it will not be easy to find our community's consensus on this issue by any simple classification. For example, let's say, if your mechanism is applied (in some way), if a CA is penalized/rewarded (in some way) due to being determined to have inappropriate expertise in line with the professional field of the proposal that he or she has evaluated, controversy may arise. In my opinion, you can refer to / look up the principles/classification system of professional professions issued by international organizations as a reference basis. With that, perhaps, the impact of this solution will be enhanced and more obvious. In addition, in terms of risk management, "Risk identification" is in conflict with "Risk mitigation", specifically: (i) at risk identification, "Funding may arrive later than expected, affecting project progress" is determined to be 1/3 risks, however, at "Risk mitigation", the proposer denies the existence of the above risk by saying: "Founder team directly develop the project, so the work progress is less dependent on the project funding progress"; (ii) in risk identification, "Inflation and economic recession may affect project team such as epidemics, wars" is also identified as 1/3 risks, however, once again, at "Risk mitigation", proposer also denies the existence of this risk by saying that: "Founder team is directly involved in the development of the mechanism, so the funding progress, inflation, and war has little effect on the project progress". The proposer should reconsider the reasonableness of these two contents because if something you are sure will not happen, it will not be considered as a risk anymore. As a result, I am neutral with this proposal's impact

Problem statement Through 7 rounds of Catalyst funding, the role of CA is still controversial such as signs of using bots, poor reviews. Solution develop a reward/penalize mechanism for CA to attract people with expertise and desire to contribute to the development of Cardano. This is something that is commented a lot in fund 6 and fund7, each phase will have changes and progress, Catalyst also needs to attract people to participate after the change to improve quality for management. decentralized value. I agree with this suggestion and it fits the challenge. The proponent gives the implementation steps (The expertise of the CA versus the main expertise of the project. Signs of using bot for evaluation. And The reward/penalty mechanism that is proportional to the quality (contribution to the project owner, to the voter) of the review.) in fund 6 and fund 7 also detected bots and removed many reviews, technologies. Using AI or rules to detect non-conformities is simple, the proponent should consult the community and list the rules. The professional assessment should follow the current mechanism to determine the evaluations as excellent/good from which to evaluate the expertise of the CA instead of evaluating on the declaration. Solution of reward and punishment mechanism, to do this, the proponent should build a ranking system of CAs from which there will be corresponding policies. The proponent also offers some risks and solutions, with the 3rd party working solution verifying the proposals for the CA bonus/penalty mechanism that needs to be adjusted, the proponent needs to clearly know who the 3rd party is. or what role do they play?

The team proposes to improve the CA process from the current mechanism where a CA review is always rewarded, either good or bad. The proposers would like to give a reward for a good review and a penalty for a poor one. They seek to create a mechanism that minimizes poor reviews, the use of bots or skimming reviews for quantity. At the same time, the reward for good reviews will be increased to an attractive amount to encourage competent and dedicated CAs to participate in proposals' review. The discussion the that been going on about arbitration mechanism or some kind of transparency about why a review has been flagged would be good for the CA to interact with before they begin the work and understand their terms of reference. This could also be incorporated so the CAs know form the beginning what is expected. I think this may be a good idea to consider and directly addresses the challenge that seeks to make improvements to the CA process

The proposers aim to develop a sort of Community Advisors rewarding and penalising system, based on 3 criteria that they consider important, while also building a database of all CAs and as such decide on the remuneration the CAs should receive from IOG. The criteria they base their ratings for the system are:

1. The CAs expertise vs the proposal’s domain
2. Bot usage
3. Quality of the review - with the added option of penalising the CA for a bad review However, there are several questions arising from this proposal:
4. Why limit proposers based on their expertise? What constitutes an expertise - a university diploma? Online courses? Where is the line drawn?
5. What does penalise mean in this context, the term is not described in the proposal clearly and having a negative effect on a CA is very important factor to have clearly explained, before it can be voted upon.
6. How will the database be built? Will the proposers gain access to Ideascale, or to IOGs internal documents? Shouldn’t CAs be asked if they agree to it, before such a drastic violation of their (possible) identity? From my CA experience, I do not see a penalising system having a positive effect on the CA Catalyst community and the impact it might have on it could end up being negative, as such I find it does not fit the challenge brief of improving the CAs performance.

Different from the current way of evaluating CAs, this proposal makes the assessment of CAs more complicated, with a stricter reward and punishment mechanism. Overall, this improves the quality of the CA, but it can also reduce the enthusiasm of many people wanting to become a CA. Certainly, at the current stage of development of the Cardano ecosystem, many people are needed to participate in the assessment, including those who are enthusiastic but do not have much experience and expertise with each challenge. The process of joining as a CA will help them grow in knowledge and experience. So why dampen the enthusiasm of the community? Ideally, this proposal should focus on the solution of detecting and excluding CAs using BOT. I rated: 3/5.

The proposal aims to develop a more symmetrical reward system for Community advisors through the introduction of a reward / penalization mechanism. Scary and interesting at the same time!

While half of my brain fantasizes right now about an ideal world in which the interests of the CAs would be perfectly aligned with the interests of the broad community, the other half is frightened thinking about all kinds of unwanted / unwarranted effects that such a tool might have if not very, very carefully designed from the beginning. I shall not go into more details, but just pointing out that a proper discussion about aligning incentives can be very very complex to have and perhaps the community needs to have more processing time before jumping into this boat.

Overall I remain neutral towards the proposal, while pointing out that it is a high risk / high reward type of project.

The problem statement of this proposal is not very clear, it is obvious that poor reviews are a problem and the use of bots for assessments is not allowed, but how that connects to the CA role being controversial I could not understand. However further down in the text we find the real problem statement divided in three parts, but with the same underlining if I may summarize it: the poor quality of assessments is incentivized by the lack of penalties. The solution for the poor reviews and the use of bots problem by developing a “reward/penalize mechanism” is spot on the challenge objectives and aligns perfectly with “Develop reputation system(s) for (v)CAs” and “Develop and assess new remuneration mechanisms for (v)CAs” both of which are potential development directions as stated in the challenge brief. The reward/penalties mechanisms for CAs will be emanated by CA audit tools and plugins but parallel to that the proposer will create a database for CAs and vCAs to be used in the future with the tools for bigDATA simulation algorithms. This again means that there will be new tools to be used by the CAs and also new tools to be used ON the CAs. The latter is not very consistent with the challenge setting I believe. The impact of the implementation of such proposal will be significant for the Catalyst ecosystem and Project Catalyst performance. Therefore considering all parts of the proposal overall I agree that this proposal effectively addresses the challenge.

The idea of the proposal is to build a reward/penalty mechanism for CA - Community Advisor. Starting from the fact that more and more CAs are now participating in the evaluation of the Catalyst , the number of CAs who actually have the qualifications is very small. Or qualified but to be able to get as many reviews as possible while ignoring the deep knowledge for financial gain. Therefore, it is very difficult for the proposer to receive positive contributions from the CA to improve and improve the quality of the proposal to increase the success rate of applying for capital Therefore, the impact of this proposal on the challenge posed is completely clear, partially solving the shortcomings of the current CA mechanism.

This project to create a dual expertise-checking mechanism and a new model for financial rewards and penalties for (v)CAs would definitely fall under two of the potential directions laid out in the challenge: "Develop reputation system(s) for (v)CAs" and "Develop and assess new remuneration mechanisms for (v)CAs". Fixing the bot problem would definitely improve vCA work too.

That being said there are some questions about how this model fits with the current system. It raises a mix of philosophical questions–such as how much a lack of expertise should prevent an assessor from offering their perspective–with some uncertainty about functionality. Some clarifications on how these new mechanisms would find harmony with the current vCA model for checking the quality of assessments would be much appreciated in the future, whether this proposal is funded or not.

There are a number of significant concerns with the ability of this proposal to effectively address the points brought up by the Challenge Team for this challenge setting. The first is the use of a software plugin to parse the alignment of a CAs expertise with the expertise required of the challenge the CA is assessing. Outside of CAs explicitly stating their qualifications in every review they do (which is currently neither required nor forbidden) the ability to parse the context of a CAs assessment to specify their expertise requires not only for the CA to give insight that would be related to their expertise, but for the reader of the assesment (voter or porposer) to also have sufficient expertise to recognize that context for what it is. The idea that a CA might be punished because their expertise is not recognized, or rewarded because their lack of expertise is unnoticed introduces a major moral hazard. Applying a software solution only makes it worse by putting mistakes in both areas on auto-pilot. This absolutely is not something that software can help with, nor even be said to be of significant importance for the quality of an assessment. The second concern is the design of a plugin that can parse assessments for bot activity by "filter out reviews that show signs of using bots based on the repetition rate of reviews for different proposals". The proposer does not seem to be aware that such a tool already exists and will be officially running on assessments in Fund8, called the "Similarity Analysis" which filters out assessments based on self-similarity and similarity to others using a %similar parameter that can be dialed in to spot bots and generalized, non-informative assessments. It does this with data from all the funds. All in all, there does not seem to be any value added with the deliverables being suggested by this proposal.

The proposed team has made a measurement of the progress and development of the project after approval, very transparent and detailed progress, the indicators have been determined and checked, the test rate is always absolute. such as building a mechanism to pay rewards and penalties for CA is 100%. construct an assessment that the correlation between CA's expertise and the expertise of the appraisal proposal is 100%, the timelines are highly appreciated, so associated with the proposal is the success of the described project. The results are clear and involve challenges such as increasing the number of excellent reviews, reducing unqualified reviews. Because I haven't seen the proposed group talk about testing yet, and this is also a new project, I realize this is a potential project that I hope the evaluation community considers and puts it into practice soon. thank you

The deliverable of evaluation criteria and penalization is not clearly indicated. In addition the decision tree of handling the penalty and rewards system seems to be centralized around the same team members. I would recommend the team to review this proposal and establish a legal and DAO system that would eliminate the centralization. Finally in terms of audit, there is mention of third party review without the identification of that gourp. Kindly provide additional data on how the progress would be shared and the deciding factors would be evaluated.

This is a part of the proposal which is really tough to be monitored. The team should think about the combination with the Fund 9 piloting instead of waiting for the next 6 months to be fully implemented. The step of “” selecting a competent third party that can verified the proposed mechanism“ is a bit generic and thus the criteria to select this party. Thus the success look likes are very vague and mostly depends on this steps.

proposer to give a detailed implementation planned in 6 months, divided into 4 phases. The objectives to be achieved in each stage are clearly defined and measurable. I recommend adding specific metrics to measure such as how many times QA is achieved,… so that the assessment can be representative of the dynamics.

Full project development metrics are also provided however most metrics measure 100% performance, which would be difficult to verify. It is possible to apply the data already in the previous funds to give the results and from there give the community a reasonable assessment. The success of the project is increasing the number of excellent reviews, I agree with this opinion. Here is a new proposal

Basically, this proposal has provided a lot of useful information. First of all, this team identity is well-defined, with supporting documents (documents/links, etc.) for confirmation. The roadmap is fully described for understanding, and metrics are mentioned and measurable. The total budget includes item hierarchy and a specific explanation of the calculation base. In short, the information provided is very complete. However, if this team reconsiders the issues I mentioned above (in "impact" and "feasibility"), there would be more information and issues that they would have to clarify in order to achieve their ultimate goal - create a convincing and fair reward/punishment mechanism to apply to CAs. As a result, based on the information provided, I can give this proposal's auditability.a score of 3

The technical aspect of the proposal is what is important to this proposal which aims at building an APIs pool; Architecting and building the original database. Realization of statistics on the web. CA criteria functions; a virtual reward model in the system similar to Catalyst's current reward mechanisms; essential statistics and indicators for Idea scale. This sort of criteria building scenario for the CA to be evaluated may work in improving the quality of assessments that we get. It is yet to be experimented and tested but I think it deserves a chance as we do need to improve the CA process

For the auditability section, at the progress development question, the team has copy/pasted the same answer as in the feasibility section, for the detailed timeline. They haven’t, however, added any measurement info for those milestones, that would have helped the community in self-assessing the advancement of the project. The success description is somewhat vague, some clearly defined milestones would have been very helpful in assessing the positive outcome of the project.

Technical solutions are specific. Budgets are clear and easily verifiable. The staff has a high reputation in the Cardano community in Vietnam. The quality of the product (accuracy of the rewards/penalties) will be recognized or denied by the CAs themselves on forums or by vCAs. I rated: 4/5.

There are a significant number of metrics proposed to measure the project’s success. A few of them are spot on in being able to capture the added value of the project, for instance: (1) the number of excellent / good / filtered out reviews or (2) the % of ADA participating in voting. Others however are much less straightforward in this sense, like the number of mechanisms for assessing proposals, which I do not view as a valid or useful metric. To offer a few alternative paths forward, i can think of the following to measure the project’s success:

• if incentives are properly aligned, we should observe a decrease in the economic price paid for a good / excellent review. Perhaps a A/B type of testing where the new mechanism would function for only one challenge during a funding round, and at the end compare and contrast reward density / proposal or average reward / proposer?
• develop an economic model to arrive at the point where the marginal elasticity of the supply of reviews (how inclined are new reviewers to do an additional review considering 1 extra USD for example in average pay / review).

The proposal uses the action plan (copy/paste) as a backbone to create auditable milestones. However there is nothing to show how and where the auditor could see the progress of this proposal. How will the progress be presented to the auditors/community stays uncertain. The definition of success for this project is not measurable and comparable to a benchmark. Only general areas are stated and there is no distinction between natural growth vs growth due to the project being implemented. In summary I disagree that this proposal provides me with sufficient information to assess the progress in attaining its stated goals.

I agree that the level of project success will be a reduction in the number of poor reviews, maybe not completely but also a reduction of 30 40%. I mean that number would be more specific. All team members can check by the link included in the proposal. The community can easily contact to see how the progress is.

However, I have a suggestion that it is recommended to create a Github page to make it easier for the community to track progress.

The milestones laid out in the proposer's auditability section is almost a direct copy and paste of the roadmap laid out in the proposer's feasibility section. And the metrics for success seem to assume that once their work developing their new model is complete, it will be implemented into the system. I'll reiterate that I think it would benefit this team to slow down a touch and focus. There's too much here all at once with no guarantee these models would even be adopted after they're developed and tested.

The team has been funded in the past and would no doubt be able to deliver a feasible project, but unfortunately that is not this project. As engaged and productive members of the community they should be encouraged to take this proposal back to the drawing board and reconsider how their software development proficiencies could better serve to improve the methods and protocols of the CA process, and Catalyst at large. In particular, the ability to create modular plugins could perhaps be excellent ways to deliver tooling to CAs who are trying to improve the quality of their assessments, maybe by giving popups with advice on the different fields that CAs assess, or running real time similarity analysis to let an assessor know if they are relying to heavily on common phrases and perhaps need to consider including more unique or insightful information. The proposal team is encouraged to keep looking at the problems faced in the process of developing quality CAs, and keep bringing their unique insights to solutions and possibilites to contribute to that growth.