CA Improvement Mechanism

Through 7 rounds of Catalyst funding, the role of CA is still controversial such as signs of using bots, poor reviews.
Équipe
Expérience
Founder team has varying expertise, namely: Business owner; Digital marketer; Full-stack developer; Project manager; Community manager, Data analysis; Dev team maker and Software product manager. Each with more than 10 years of experience.
Solution
Develop a reward/penalize mechanism for CA to attract people with expertise and desire to contribute to the development of Cardano.
Open English text in another tab. Open
Error! Try saving againAvis des conseillers communautaires
Addresses Challenge
Does the proposal effectively addresses the challenge?
Revues de la Communauté (14)
This proposal addresses the CA improvement mechanism , reporting clearly defined problems that match the challenge . The main part of this project is building and attracting people with expertise to contribute to the development of Cardano . this will increase the positive impact for the Cardano ecosystem and possibly scale the development to address future challenges. In order to complete the above idea, the proponent has provided solutions that have defined reward and punishment mechanisms, but I need a more detailed explanation of this problem, more specific examples are suggested. . The proposed solution I fully believe will solve the problem of the challenge, describe the solution in great detail as for financial purposes, some CAs use bots to evaluate projects. , leading to shoddy reviews. a proposal always goes hand in hand with success and risk, the proposed team has come up with some risk identifications such as capital sources, inflation, recession, let me ask if the technical risk factor is What will happen to the proposal and how to fix it. I also highly appreciate the proposal that also outlines risk reduction options. good luck project
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_958
- Total QA Ratings
- 7
- QA Rating Outcome
The team has very well-known in the CA community with past reviews vCA and CA experience. The solution is proposing the reward & penalized mechanism for CA review which is really understanding the game theory and the motivations of the CAs to go for quantity instead of quality and helpful reviews for the community. The solution is the database API pools to track and then have the incentivized and penalized scheme. Thus, the team should be considered with dRep current approach to have incorporated the more experienced vCA or DRep at the combined solutions to improve the quality of the CA communities. One positive thing is that the team well-thought about the risk of funding and at the end of the date is the CA rewards/penalty is such at the later stage after getting funded. The risk of war and economic recession is a bit overthinking given in the realm of the Cardano blockchain.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_782
- Total QA Ratings
- 5
- QA Rating Outcome
The CA improvement challenge is about improving the impact the Catalyst ecosystem relies. The bigger diversity and opinion of different background the more wholesome the community becomes. This challenge aims to regulate and limit the interaction of a wide backgound of CA's to a small micro bubble around their expertise. The proposed challenge would infact redcue the positive impact of the CA in catalyst and render the exercise limited to a certain experts reviewers and the typical newcommer would find difficulty joining in, thus rendering the CA a very limited role within the community.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_785
- Total QA Ratings
- 7
- QA Rating Outcome
This proposal will attract people with expertise and desire to contribute to the development of Cardano by building reward and penalty mechanisms for CA. CAs that do well will be rewarded and vice versa. Direct targeting of financial benefits under this proposal will push CAs to further improve themselves to meet the requirements from there will reduce fraud and poor-quality reviews. I totally agree this proposer will effectively solve the challenge.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_706
- Total QA Ratings
- 5
- QA Rating Outcome
The main point of this proposal is to create a reward/punishment system for CA to attract people with skill and a willingness to contribute to Cardano's progress. As seen, the statement of the problem is clear and accurate for the given challenge. Further to that, the solution is well-defined, with convincing short-and long-term potential benefits for the Cardano ecosystem. However, there are some aspects that, in my opinion, this team should reconsider. Firstly, I agree that the CA's professional/expertise in relevance to the respective proposal's area of expertise, which he/she assess, is important; it affects the reliability of their assessments, as a whole. However, the classification of the area of expertise is various between different communities/countries, or in other words, it will not be easy to find our community's consensus on this issue by any simple classification. For example, let's say, if your mechanism is applied (in some way), if a CA is penalized/rewarded (in some way) due to being determined to have inappropriate expertise in line with the professional field of the proposal that he or she has evaluated, controversy may arise. In my opinion, you can refer to / look up the principles/classification system of professional professions issued by international organizations as a reference basis. With that, perhaps, the impact of this solution will be enhanced and more obvious. In addition, in terms of risk management, "Risk identification" is in conflict with "Risk mitigation", specifically: (i) at risk identification, "Funding may arrive later than expected, affecting project progress" is determined to be 1/3 risks, however, at "Risk mitigation", the proposer denies the existence of the above risk by saying: "Founder team directly develop the project, so the work progress is less dependent on the project funding progress"; (ii) in risk identification, "Inflation and economic recession may affect project team such as epidemics, wars" is also identified as 1/3 risks, however, once again, at "Risk mitigation", proposer also denies the existence of this risk by saying that: "Founder team is directly involved in the development of the mechanism, so the funding progress, inflation, and war has little effect on the project progress". The proposer should reconsider the reasonableness of these two contents because if something you are sure will not happen, it will not be considered as a risk anymore. As a result, I am neutral with this proposal's impact
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_679
- Total QA Ratings
- 8
- QA Rating Outcome
Problem statement Through 7 rounds of Catalyst funding, the role of CA is still controversial such as signs of using bots, poor reviews. Solution develop a reward/penalize mechanism for CA to attract people with expertise and desire to contribute to the development of Cardano. This is something that is commented a lot in fund 6 and fund7, each phase will have changes and progress, Catalyst also needs to attract people to participate after the change to improve quality for management. decentralized value. I agree with this suggestion and it fits the challenge. The proponent gives the implementation steps (The expertise of the CA versus the main expertise of the project. Signs of using bot for evaluation. And The reward/penalty mechanism that is proportional to the quality (contribution to the project owner, to the voter) of the review.) in fund 6 and fund 7 also detected bots and removed many reviews, technologies. Using AI or rules to detect non-conformities is simple, the proponent should consult the community and list the rules. The professional assessment should follow the current mechanism to determine the evaluations as excellent/good from which to evaluate the expertise of the CA instead of evaluating on the declaration. Solution of reward and punishment mechanism, to do this, the proponent should build a ranking system of CAs from which there will be corresponding policies. The proponent also offers some risks and solutions, with the 3rd party working solution verifying the proposals for the CA bonus/penalty mechanism that needs to be adjusted, the proponent needs to clearly know who the 3rd party is. or what role do they play?
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_697
- Total QA Ratings
- 5
- QA Rating Outcome
The team proposes to improve the CA process from the current mechanism where a CA review is always rewarded, either good or bad. The proposers would like to give a reward for a good review and a penalty for a poor one. They seek to create a mechanism that minimizes poor reviews, the use of bots or skimming reviews for quantity. At the same time, the reward for good reviews will be increased to an attractive amount to encourage competent and dedicated CAs to participate in proposals' review. The discussion the that been going on about arbitration mechanism or some kind of transparency about why a review has been flagged would be good for the CA to interact with before they begin the work and understand their terms of reference. This could also be incorporated so the CAs know form the beginning what is expected. I think this may be a good idea to consider and directly addresses the challenge that seeks to make improvements to the CA process
The proposers aim to develop a sort of Community Advisors rewarding and penalising system, based on 3 criteria that they consider important, while also building a database of all CAs and as such decide on the remuneration the CAs should receive from IOG. The criteria they base their ratings for the system are:
- The CAs expertise vs the proposal’s domain
- Bot usage
- Quality of the review - with the added option of penalising the CA for a bad review However, there are several questions arising from this proposal:
- Why limit proposers based on their expertise? What constitutes an expertise - a university diploma? Online courses? Where is the line drawn?
- What does penalise mean in this context, the term is not described in the proposal clearly and having a negative effect on a CA is very important factor to have clearly explained, before it can be voted upon.
- How will the database be built? Will the proposers gain access to Ideascale, or to IOGs internal documents? Shouldn’t CAs be asked if they agree to it, before such a drastic violation of their (possible) identity? From my CA experience, I do not see a penalising system having a positive effect on the CA Catalyst community and the impact it might have on it could end up being negative, as such I find it does not fit the challenge brief of improving the CAs performance.
Different from the current way of evaluating CAs, this proposal makes the assessment of CAs more complicated, with a stricter reward and punishment mechanism. Overall, this improves the quality of the CA, but it can also reduce the enthusiasm of many people wanting to become a CA. Certainly, at the current stage of development of the Cardano ecosystem, many people are needed to participate in the assessment, including those who are enthusiastic but do not have much experience and expertise with each challenge. The process of joining as a CA will help them grow in knowledge and experience. So why dampen the enthusiasm of the community? Ideally, this proposal should focus on the solution of detecting and excluding CAs using BOT. I rated: 3/5.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_1913
- Total QA Ratings
- 4
- QA Rating Outcome
The proposal aims to develop a more symmetrical reward system for Community advisors through the introduction of a reward / penalization mechanism. Scary and interesting at the same time!
While half of my brain fantasizes right now about an ideal world in which the interests of the CAs would be perfectly aligned with the interests of the broad community, the other half is frightened thinking about all kinds of unwanted / unwarranted effects that such a tool might have if not very, very carefully designed from the beginning. I shall not go into more details, but just pointing out that a proper discussion about aligning incentives can be very very complex to have and perhaps the community needs to have more processing time before jumping into this boat.
Overall I remain neutral towards the proposal, while pointing out that it is a high risk / high reward type of project.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_1454
- Total QA Ratings
- 8
- QA Rating Outcome
The problem statement of this proposal is not very clear, it is obvious that poor reviews are a problem and the use of bots for assessments is not allowed, but how that connects to the CA role being controversial I could not understand. However further down in the text we find the real problem statement divided in three parts, but with the same underlining if I may summarize it: the poor quality of assessments is incentivized by the lack of penalties. The solution for the poor reviews and the use of bots problem by developing a “reward/penalize mechanism” is spot on the challenge objectives and aligns perfectly with “Develop reputation system(s) for (v)CAs” and “Develop and assess new remuneration mechanisms for (v)CAs” both of which are potential development directions as stated in the challenge brief. The reward/penalties mechanisms for CAs will be emanated by CA audit tools and plugins but parallel to that the proposer will create a database for CAs and vCAs to be used in the future with the tools for bigDATA simulation algorithms. This again means that there will be new tools to be used by the CAs and also new tools to be used ON the CAs. The latter is not very consistent with the challenge setting I believe. The impact of the implementation of such proposal will be significant for the Catalyst ecosystem and Project Catalyst performance. Therefore considering all parts of the proposal overall I agree that this proposal effectively addresses the challenge.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_1417
- Total QA Ratings
- 9
- QA Rating Outcome
The idea of the proposal is to build a reward/penalty mechanism for CA - Community Advisor. Starting from the fact that more and more CAs are now participating in the evaluation of the Catalyst , the number of CAs who actually have the qualifications is very small. Or qualified but to be able to get as many reviews as possible while ignoring the deep knowledge for financial gain. Therefore, it is very difficult for the proposer to receive positive contributions from the CA to improve and improve the quality of the proposal to increase the success rate of applying for capital Therefore, the impact of this proposal on the challenge posed is completely clear, partially solving the shortcomings of the current CA mechanism.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_1394
- Total QA Ratings
- 9
- QA Rating Outcome
This project to create a dual expertise-checking mechanism and a new model for financial rewards and penalties for (v)CAs would definitely fall under two of the potential directions laid out in the challenge: "Develop reputation system(s) for (v)CAs" and "Develop and assess new remuneration mechanisms for (v)CAs". Fixing the bot problem would definitely improve vCA work too.
That being said there are some questions about how this model fits with the current system. It raises a mix of philosophical questions–such as how much a lack of expertise should prevent an assessor from offering their perspective–with some uncertainty about functionality. Some clarifications on how these new mechanisms would find harmony with the current vCA model for checking the quality of assessments would be much appreciated in the future, whether this proposal is funded or not.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_1275
- Total QA Ratings
- 11
- QA Rating Outcome
There are a number of significant concerns with the ability of this proposal to effectively address the points brought up by the Challenge Team for this challenge setting. The first is the use of a software plugin to parse the alignment of a CAs expertise with the expertise required of the challenge the CA is assessing. Outside of CAs explicitly stating their qualifications in every review they do (which is currently neither required nor forbidden) the ability to parse the context of a CAs assessment to specify their expertise requires not only for the CA to give insight that would be related to their expertise, but for the reader of the assesment (voter or porposer) to also have sufficient expertise to recognize that context for what it is. The idea that a CA might be punished because their expertise is not recognized, or rewarded because their lack of expertise is unnoticed introduces a major moral hazard. Applying a software solution only makes it worse by putting mistakes in both areas on auto-pilot. This absolutely is not something that software can help with, nor even be said to be of significant importance for the quality of an assessment. The second concern is the design of a plugin that can parse assessments for bot activity by "filter out reviews that show signs of using bots based on the repetition rate of reviews for different proposals". The proposer does not seem to be aware that such a tool already exists and will be officially running on assessments in Fund8, called the "Similarity Analysis" which filters out assessments based on self-similarity and similarity to others using a %similar parameter that can be dialed in to spot bots and generalized, non-informative assessments. It does this with data from all the funds. All in all, there does not seem to be any value added with the deliverables being suggested by this proposal.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_1193
- Total QA Ratings
- 13
- QA Rating Outcome
Feasibility
Given experience and plan presented is likely that this proposal will be implemented successfully?
Revues de la Communauté (14)
The proposal has identified a timeline to complete the work. The timelines arranged for completion are quite consistent with the proposal where the proposed work has broken down the cost of each month of work such as construction. theorize the correlation between CA expertise and review recommendation expertise, set up and program a tool to filter reviews with signs of bot usage, provide recommendation mechanism to selected parties To rate I highly recommend the idea presented on the route of putting . The budget of the proposal with the summary table has been broken down into small sections for easy identification, but the contingency fee should be stated for what purpose the fee is used and the cost for which part is reasonable. one thing i wonder is with the required capital is 29,600 usd, i add up each amount only 28,600 usd i need an explanation about 1,000 usd which is different from the detailed one, the writer of the proposal needs to consider this issue more carefully. About the team of the proposal specified each member and assigned tasks to each person with authenticated identity, the links to each person's personal page are easy to verify. this is a good idea, the proposed topic needs to promote the good aspects and fix the small errors, thank you very much
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_958
- Total QA Ratings
- 7
- QA Rating Outcome
The teamembers are well versed in various fields and show interest in providing a succesful proposal. In addition the timeline at various stages is well categorized. However the different deliverables require internal data from catalyst and will be manipulated to expose the anonymity of the CA memebers as well as their personal information. I highly doubt the information would be atteindable. I would recommend the team to include a legal member to review the legal implication this has since it requires to segregate certain rewards from CA memebers. In addition the regulation and decision of evaluation would be removed from the vCA and handled by a few key members.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_785
- Total QA Ratings
- 7
- QA Rating Outcome
The budget request is 30% of the total challenge setting. The team requested an idea however the detailed solution of the software and the resource for it is very vague at the moment with the contingency fee of 3000$ is almost 3% of the total fund. The resource for the human resource are not yet transparent and In the next 3 months after approved: the team just developed the mechanism and the application filter (which should be planned from the beginning of the idea proposal (especially the mechanism) This is a long term and over the next 6 months until the Fund 10 which should be considered to be considered by the dRep solutions.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_782
- Total QA Ratings
- 5
- QA Rating Outcome
The proposal team consists of 4 members who cite their experience to be able to successfully implement the proposal using links that the community can check. I rate them accordingly. They come up with a plan and how they implement it with a 6-month implementation roadmap divided into 4 phases. Possible risks, as well as solutions for those risks, have also been calculated and prevented by the proposer. The requested budget of 29600 is broken down into 7 categories which are quite specific and clear and I consider it reasonable. In summary, I agree this proposal will be successfully implemented. However, I suggest that they explain more specifically the cost for items 5 and 7, what they plan to do, how to do it, for how long, this will help the reader better in assessing the cost feasibility of the project.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_706
- Total QA Ratings
- 5
- QA Rating Outcome
The implementation plan roadmap is given by the proponent within 6 months, the contents for implementation each month are appropriate. The theoretical programming part to create a software to evaluate the professional correlation between the CA and the proposal, this also requires careful research or the input information about the CA needs to be collected more fully. The proposed expenses is also clearly stated in 7 including contingencies, calculation methods as well as unit prices provided in full. In addition, the proposal also provides full information about the team from each member's experience / obligations / social networking sites. I appreciate this
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_697
- Total QA Ratings
- 5
- QA Rating Outcome
As one can see, this team identity is well-defined, illustrating quite appropriate experience and expertise for successful execution, along with giving references/proof (documents/links, etc.); but it would be better if a member with expertise in related to policy research/ manager may increase the feasibility of this proposal. I think the proposer should consider adding some similar demonstration, pilot, or prototype done by this team/team's member(s). Following the opinion I presented above, the feasibility of this proposal is influenced by two main factors: (i) input information related to the assessor's expertise: due to some assessors may have many different professions, how to ensure that the information they declare is reliable ?; (ii) how to determine which field of expertise a proposal is in? because there are many proposals at the same time related to many different fields of expertise. As a result, I am neutral with this proposal's possible
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_679
- Total QA Ratings
- 8
- QA Rating Outcome
The team displays a wide range of expertise, from Business owner to Digital marketer; Full-stack developer; Project manager; Community manager, Data analysis; Dev team maker and Software product manager. Each with more than 10 years of experience. I feel that this is sufficient to bring the proposal to a conclusion and mitigates the risk of falling off by having enough people on the team. The plan is for there to be an Increment in the number of new mechanisms for assessing and scoring proposals; an Increment in the number of excellent reviews; a decrease in the number of unqualified / poor reviews; an increase in voter participation; an increase in the ratio of the number of voting wallets to the total number of wallets. This is measurable and will definitely be an improvement to the process as a whole
The detailed plan and roadmap of the proposal are comprehensively described, for the first 6 months after the potential funding. At this point in the proposal, I do not see the need for any additional information. The budget is meticulously drafted, having all the needed costs and numbers worked out. The proposer have written very short descriptions of themselves, however they have not posted any professional social profiles that would back up their expertise - they have only posted links to their telegram profiles, which is not suitable for a professional environment, and as such have not been able to provide confidence in their abilities to implement the proposal.
The plan is very detailed and clear. However, how to ensure the correct correlation between the CA's expertise and the proposal/challenge's expertise? This challenge has not been raised and convincingly presented in this proposal. I rated: 4/5
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_1913
- Total QA Ratings
- 4
- QA Rating Outcome
Judging the team’s skills and experience, I can see a lot of technical competence gathered there and I have no reason to believe that technically the execution has a high chance of being delivered as promised. My main point of concern is whether the team has gathered enough support from the community regarding these proposed changes, without risking a painful backlash in the form of CAs drop-outs and low future enrollment numbers due to decreased expected rewards.
Judging the feasibility of this project is also no easy feat: it would require extensive and deep changes to the already existing models. I wonder however if such a system in which rewards and penalizations cannot be envisaged under the current model with much less disruption to existing systems in place. I offer two possible paths forward:
- turn the existing model of Filtered / good / Excellent review from a (0 / x / 3x) reward system to a (- y / x / 3x) system in which filtered rewards are punished immediately, with perhaps 5% of the coding requirements needed
- also, or maybe in addition, such a system can be designed separately and maybe run as a simulation after next funding rounds using real data from CA reviews, and run a comparative analysis of how rewards are distributed according to each model, and judge which one might be beneficial for the community in the long term.
Overall again I remain neutral towards the proposal, while pointing out that it is a high risk / high reward type of project.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_1454
- Total QA Ratings
- 8
- QA Rating Outcome
The proposers know what tools and resources they will need to implement the proposal. The proposers identify risks and provide a mitigation plan for the set risks. What worries me here is that the proposers does not find the risk of IOG not allowing punishment/reward mechanism for the Project Catalyst for one reason or another, the first that comes to mind is that Cardano is build on mathematical models and the implementation of such mechanism could disagree with the mathematical reasoning for not having one up to this moment. Another risk that I see is that the community could consider the review of the assessments of CAs from third party (as briefly mentioned in the proposal) as unacceptable concentration of power in organization that is external for the community. In both these cases the whole project will lose the reason to exist but they were not mentioned in the foreseen risks. The proposal has a plan and timeline for development and implementation. The detail is lacking in many places, I suppose due to the theoretical work that needs to be done prior to actual software development. There is a question that arises in me, how the proposers set timeline when there are plenty of scientific research that is the basis of the work. For example,“ Developing a theory on the new mechanism for rewarding and penalizing CA for their review.“ Nobody knows how much time this mathematical research could take, nobody knows if it is even possible, but the proposers gave it 3 months. Also, I have an issue with the wording “a theory” everyone could make “a theory”, that theory in my opinion should have some parameters by which to assess its quality. This is not addressed in the proposal and this prevents the voter from taking informed decision. The budget is sufficiently detailed and it turns out the theory for the reward and penalty will take 100hours. What I like about the budget is that 10% of the funding is saved for contingency. This shows preparedness for unexpected events. The team is well presented and clear roles are assigned to the team members. There are links to each team member linkedin account that are functional. From the team presentation it is not obvious who is going to create the mathematical backbone for the project main goal, the development of reward & penalize mechanism. There is no Ph.D. in mathematics among the team members. The lack of clarity on who is going to pull the weight of the theoretical part of the project, the missed obvious risks and the lack of detail in the actionable plan, lead me to the conclusion that I disagree that it is highly likely that this proposal will be implemented successfully.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_1417
- Total QA Ratings
- 9
- QA Rating Outcome
I've read through every detail in the proposal, but there are a few points I'm still wondering about:
-
On what criteria is the proposal based to measure the expertise of the evaluator relative to the main expertise of the proposal. Because gathering data from a CA review source to measure expertise I consider it difficult. The level cannot be measured by just a few paragraphs. It also comes from how we perceive what the person presents, this perception is human emotion but the machine does not. Even people in real life are difficult to judge someone's professional qualifications or not just through a few encounters. It is proposed to provide a solution that will continuously collect data on CA's project evaluation for many years to come to add to the bigDATA repository for analysis and evaluation. However, I think that this is not very feasible because in general, the data is only encapsulated in this Catalyst framework. It is necessary to collect information from more data sources of that CA outside of life.
-
The time spent in the Design of the Plugin Software to consider the correlation between the CA's expertise and the proposal's expertise is 50 hours and the route to building the theory of correlation is within the first 1 month If the average working day is 8 hours, it only takes 1 week to design. I think that's impossible. Because for the reasons that I mentioned in paragraph 1, it requires a whole process, it takes a lot of time to come up with a "general feeling" about the professional qualifications of the CA. A job cannot just code within 1 week. It would therefore make sense if the entire initial period of the project were focused on collecting data and possibly a year later it would be possible to write down the "correlation" between the CA expertise and the proposal expertise.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_1394
- Total QA Ratings
- 9
- QA Rating Outcome
The proposers are asking for almost 30% of the total fund for this challenge, which means the standards for it need to be high.
What I see here is a team with decent technical abilities and a devotion to the Cardano ecosystem. What could enhance their pitch is, when they cite their credentials, show us their accomplishments directly. For example, if you developed an app, link us to the app, or at least a demonstration of the app's basics.
A significant portion of this proposal is the development of a new incentive model. Can you link us to previous work done in this regard? If there are none, please openly say this and link your current expertise to a project of this kind. It is common for experiments to touch on new ground! But for a sum this large, you need to go the extra mile to justify the project, especially since it also contains potentially controversial elements such as credentialized gatekeeping and financial penalties.
Though honestly, this team's proposal is written with enthusiasm that is contagious and I can't help but admire it. If I may make a humble suggestion to the team: Slow down a little and focus. Choose one of their three stated goals and propose that you will zero in on a solution to that problem. I think it might be best to center your attention on preventing fraudulent CAs from leveraging bots to exploit the system. This was a serious problem last fund that devoured an enormous amount of vCA work! Fix that and you'll be heroes.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_1275
- Total QA Ratings
- 11
- QA Rating Outcome
The suggestion of a plugin to parse subtle context in written (which means a writer AND a reader) communication is simply untenable. Establishing expertise, while the anonymity factor is being preserved for CAs, is limited to readers picking up contextual clues left by writers, including club-you-over-the-head context like short qualification statements repeated ad nauseum. The team assembled to execute this task certainly seems up to the task of a legitimate plug-in design and delivery, but whatever could be built would not be able to get results that help. And because there is no need for the second tool either (a similarity script), this is also a non-issue for the timeline, budget and team abilities. Finally, the funds in the budget dedicated to ideating and designing a reward/punishment protocol under the principles laid out in the proposal do not really fit either: for one, there is already a rewards system in place that rewards both excellent quality and CAs who make sure that high dollar budgets get plenty of attention and analysis. Furthermore, the idea of punishing a CA, above and beyond the loss of their time and energy when their assessments are graded as unuseable and filtered out with no compensation, goes against principles of Catalyst. Catalyst is not an experiment that should result in punishments being meted out, and one of its guiding principles is a "safe place to fail". CAs who fail to create quality assessments require education and guidance, not punishment. Filtering mechanisms are as far as the experiment should go at this point. Punishment has no place, in my opinion.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_1193
- Total QA Ratings
- 13
- QA Rating Outcome
Auditability
Does the proposal provides sufficient information to assess and audit progress and completion?
Revues de la Communauté (14)
The proposed team has made a measurement of the progress and development of the project after approval, very transparent and detailed progress, the indicators have been determined and checked, the test rate is always absolute. such as building a mechanism to pay rewards and penalties for CA is 100%. construct an assessment that the correlation between CA's expertise and the expertise of the appraisal proposal is 100%, the timelines are highly appreciated, so associated with the proposal is the success of the described project. The results are clear and involve challenges such as increasing the number of excellent reviews, reducing unqualified reviews. Because I haven't seen the proposed group talk about testing yet, and this is also a new project, I realize this is a potential project that I hope the evaluation community considers and puts it into practice soon. thank you
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_958
- Total QA Ratings
- 7
- QA Rating Outcome
The deliverable of evaluation criteria and penalization is not clearly indicated. In addition the decision tree of handling the penalty and rewards system seems to be centralized around the same team members. I would recommend the team to review this proposal and establish a legal and DAO system that would eliminate the centralization. Finally in terms of audit, there is mention of third party review without the identification of that gourp. Kindly provide additional data on how the progress would be shared and the deciding factors would be evaluated.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_785
- Total QA Ratings
- 7
- QA Rating Outcome
This is a part of the proposal which is really tough to be monitored. The team should think about the combination with the Fund 9 piloting instead of waiting for the next 6 months to be fully implemented. The step of “” selecting a competent third party that can verified the proposed mechanism“ is a bit generic and thus the criteria to select this party. Thus the success look likes are very vague and mostly depends on this steps.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_782
- Total QA Ratings
- 5
- QA Rating Outcome
proposer to give a detailed implementation planned in 6 months, divided into 4 phases. The objectives to be achieved in each stage are clearly defined and measurable. I recommend adding specific metrics to measure such as how many times QA is achieved,… so that the assessment can be representative of the dynamics.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_706
- Total QA Ratings
- 5
- QA Rating Outcome
Full project development metrics are also provided however most metrics measure 100% performance, which would be difficult to verify. It is possible to apply the data already in the previous funds to give the results and from there give the community a reasonable assessment. The success of the project is increasing the number of excellent reviews, I agree with this opinion. Here is a new proposal
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_697
- Total QA Ratings
- 5
- QA Rating Outcome
Basically, this proposal has provided a lot of useful information. First of all, this team identity is well-defined, with supporting documents (documents/links, etc.) for confirmation. The roadmap is fully described for understanding, and metrics are mentioned and measurable. The total budget includes item hierarchy and a specific explanation of the calculation base. In short, the information provided is very complete. However, if this team reconsiders the issues I mentioned above (in "impact" and "feasibility"), there would be more information and issues that they would have to clarify in order to achieve their ultimate goal - create a convincing and fair reward/punishment mechanism to apply to CAs. As a result, based on the information provided, I can give this proposal's auditability.a score of 3
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_679
- Total QA Ratings
- 8
- QA Rating Outcome
The technical aspect of the proposal is what is important to this proposal which aims at building an APIs pool; Architecting and building the original database. Realization of statistics on the web. CA criteria functions; a virtual reward model in the system similar to Catalyst's current reward mechanisms; essential statistics and indicators for Idea scale. This sort of criteria building scenario for the CA to be evaluated may work in improving the quality of assessments that we get. It is yet to be experimented and tested but I think it deserves a chance as we do need to improve the CA process
For the auditability section, at the progress development question, the team has copy/pasted the same answer as in the feasibility section, for the detailed timeline. They haven’t, however, added any measurement info for those milestones, that would have helped the community in self-assessing the advancement of the project. The success description is somewhat vague, some clearly defined milestones would have been very helpful in assessing the positive outcome of the project.
Technical solutions are specific. Budgets are clear and easily verifiable. The staff has a high reputation in the Cardano community in Vietnam. The quality of the product (accuracy of the rewards/penalties) will be recognized or denied by the CAs themselves on forums or by vCAs. I rated: 4/5.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_1913
- Total QA Ratings
- 4
- QA Rating Outcome
There are a significant number of metrics proposed to measure the project’s success. A few of them are spot on in being able to capture the added value of the project, for instance: (1) the number of excellent / good / filtered out reviews or (2) the % of ADA participating in voting. Others however are much less straightforward in this sense, like the number of mechanisms for assessing proposals, which I do not view as a valid or useful metric. To offer a few alternative paths forward, i can think of the following to measure the project’s success:
- if incentives are properly aligned, we should observe a decrease in the economic price paid for a good / excellent review. Perhaps a A/B type of testing where the new mechanism would function for only one challenge during a funding round, and at the end compare and contrast reward density / proposal or average reward / proposer?
- develop an economic model to arrive at the point where the marginal elasticity of the supply of reviews (how inclined are new reviewers to do an additional review considering 1 extra USD for example in average pay / review).
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_1454
- Total QA Ratings
- 8
- QA Rating Outcome
The proposal uses the action plan (copy/paste) as a backbone to create auditable milestones. However there is nothing to show how and where the auditor could see the progress of this proposal. How will the progress be presented to the auditors/community stays uncertain. The definition of success for this project is not measurable and comparable to a benchmark. Only general areas are stated and there is no distinction between natural growth vs growth due to the project being implemented. In summary I disagree that this proposal provides me with sufficient information to assess the progress in attaining its stated goals.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_1417
- Total QA Ratings
- 9
- QA Rating Outcome
I agree that the level of project success will be a reduction in the number of poor reviews, maybe not completely but also a reduction of 30 40%. I mean that number would be more specific. All team members can check by the link included in the proposal. The community can easily contact to see how the progress is.
However, I have a suggestion that it is recommended to create a Github page to make it easier for the community to track progress.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_1394
- Total QA Ratings
- 9
- QA Rating Outcome
The milestones laid out in the proposer's auditability section is almost a direct copy and paste of the roadmap laid out in the proposer's feasibility section. And the metrics for success seem to assume that once their work developing their new model is complete, it will be implemented into the system. I'll reiterate that I think it would benefit this team to slow down a touch and focus. There's too much here all at once with no guarantee these models would even be adopted after they're developed and tested.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_1275
- Total QA Ratings
- 11
- QA Rating Outcome
The team has been funded in the past and would no doubt be able to deliver a feasible project, but unfortunately that is not this project. As engaged and productive members of the community they should be encouraged to take this proposal back to the drawing board and reconsider how their software development proficiencies could better serve to improve the methods and protocols of the CA process, and Catalyst at large. In particular, the ability to create modular plugins could perhaps be excellent ways to deliver tooling to CAs who are trying to improve the quality of their assessments, maybe by giving popups with advice on the different fields that CAs assess, or running real time similarity analysis to let an assessor know if they are relying to heavily on common phrases and perhaps need to consider including more unique or insightful information. The proposal team is encouraged to keep looking at the problems faced in the process of developing quality CAs, and keep bringing their unique insights to solutions and possibilites to contribute to that growth.
Assessment Quality Assurance
Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.
- Assessor ID
- z_assessor_1193
- Total QA Ratings
- 13
- QA Rating Outcome
Aucun commentaire pour le moment...