Nothing came up for
over budget

P.I.E for Community Dev. Sector

$35,636.00 Requested
Ideascale logo View on ideascale
Community Review Results (4 reviewers)
Addresses Challenge
Feasibility
Auditability
Solution

Propose and apply P.I.E framework to assess the result of F7's + F8’s funded proposals (sector of “Community Development”)

Problem:

To grow auditability, we need to evaluate the implementation result of Catalyst’s funded proposals with systematic framework

Yes Votes:
₳ 29,345,383
No Votes:
₳ 13,996,045
Votes Cast:
105

  • download
  • download
  • download
  • download
  • download
  • download
  • download
  • download
Help Translate! login

Open English text in another tab. Open

Error! Try saving again
Translation saved
Expand

Community Reviews (4)

Addresses Challenge

4.3 / 5
4 Reviews

Does the proposal effectively addresses the challenge?

Commenter gravatar

According to the proposers, in order to grow auditability, there is need to evaluate the implementation result of Catalyst’s funded proposals with systematic framework. this definitely addresses the issue of auditability an da clear framework that is workable to the proposers will be of great help to improve auditability

Confirming your humanity

Write comment

Replying to

Commenter gravatar

This proposal has ambitious plans for auditing (ballpark: 200) a larger number of proposals but limits this to Fund-7 and Fund-8 F7 and F8 top 30% of funding in a selection that is called "Community Development" including 15 challenge settings for those 2 funds. Instead of using the existing communities growing around Town Hall, Swarm, Circle, popular Telegram channels for Proposers, CAs etc., it includes a lot of "branding" for the auditors, namely a DAO in 2 years, and several styles like P.I.E. under the umbrella of an existing multi-chain crypto website. This then requires a lot of resources to be expended on liaison with other efforts across the two communities. While on the surface this has the potential to achieve several of the goals set in the challenge, it seems to be a poor use of funding and should have been proposed with a more selective scope to produce a strong proof-of-concept. Consuming about a sixth of the total challenge budget and assessing over 200 proposals over a period spanning much longer than those proposals will be in their implementation phase will result in relatively less "bang for our buck (or Ada)". Therefore, although relevant to the challenge setting, it may not have a good return in terms of impact on the investment.

Confirming your humanity

Write comment

Replying to

Commenter gravatar

One of the most important areas of passing self-governance from IOG to Project Catalyst, mostly because it is such a pressing issue, is in the auditing of funded proposals. This Proposal correctly identifies the issue: newly funded proposals are increasing rapidly, while previously funded proposals are slow to reach completion and be removed from the auditing workflow. The proposer is submitting here a prototype that can operate alongside the existing auditing protocol, and if it proves itself, would be able to take a portion of the auditing load onto itself in any given fund, if not all of the auditing. This is a highly relevant area of focus for this challenge, and prototypes like this are a necessary first step before such a critical step of the funding mechanism can be handed over. If successful, the project is scalable with the addition of auditors, and the only step left would be the integration of the audit results with the Treasury, in order to continue to release funds until projects reach completion. The technique being used here (labeled as PIE) uses well known conventions around the ideas of Conformance evaluation and Performance evaluation that delivers a project grade in several areas that is compared to a reference index that the proposal team builds out in relation to the challenge setting metrics and goals. Exploration and execution of these reference indexes is as important a part of the project as any, since they determine the quality and success of the auditing process.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_1193
Total QA Ratings
14
QA Rating Outcome
Confirming your humanity

Write comment

Replying to

Commenter gravatar

This proposal can be summarized as “proposing the PIE (Proposal Implementation Assessment) framework with criteria and indicators to assess the compliance and performance of proposals financed by Fund 8”, where in this structure “they are used to assess the degree of compliance with the goals, commitments, KPIs/metrics established in the proposal”. Theoretically, this proposal fits perfectly with the specific objectives of the challenge, from the moment it proposes to create this new method of auditing proposals, based on the aforementioned indicators. Proponents argue that as funds go by, the number of proposals being funded grows too high and it is difficult for IOG to fully audit them. I agree with this preposition, because that is precisely why this challenge was launched, and I think that coming up with this P.I.E structure is a good way to solve the problem. However, I think the bidders could have done better to explain how the audit would happen in practice, how this structure would be used to evaluate the bids. In my view, this could become clearer if they, for example, took some proposal and applied the audit through this structure. Thus, we would see how this would happen in everyday life, and we would better understand the strategies used. Therefore, my conclusion is: this proposal addresses very well what we are looking for in the challenge, as it suggests a more standardized and probably more practical way of auditing the growing number of proposals financed by each fund, but from the explanation provided in the detail of the proposal, I cannot solidly visualize how this audit would be done, even though they brought up many issues that would be evaluated in the “specifically, the implementation of this proposal will contribute to the success of this Challenge in the following KPIs” section. With that, the impact is a little affected, I still believe it's a good idea and that it will work, but because I don't understand exactly how, I don't feel comfortable giving full marks to this topic of analysis. I would also like to know how the auditing of the proposals that they encompass as “community development” would be different from the auditability they would do, for example, in the game challenge. What are the specifics here?

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_57
Total QA Ratings
10
QA Rating Outcome
Confirming your humanity

Write comment

Replying to

Feasibility

4 / 5
4 Reviews

Given experience and plan presented is likely that this proposal will be implemented successfully?

Commenter gravatar

The CryptoViews team consists of funded proposers, CA, vCA, researchers, planners who have experience in plan implementation evaluation with quite a number of funded proposals and looks capable of performing the task well. Their plan is well outlined to Percentage of funded proposals evaluated by using P.I.E framework in the defined sector of "Community Development": 30%. 8 P.I.E reports. 02 video uploaded for summarizing F7 and F8 Final P.I.E Reports. The timelines spread over a period of 15 months are realistic

Confirming your humanity

Write comment

Replying to

Commenter gravatar

This proposal gives a lot of detail for both the roadmap and budget breakdown, both split into comparable timelines. While this is excellent practice, it does not convince me that the business plan itself adds up. Almost half of the budget is reserved for poorly understood risks like Ada slippage (which can go both ways) and contingencies, and the proposal does not seem to be equipped with the resource or enforceability to achieve compliance of proposers or be able to follow through on the roadmap presented. Time needed for each proposal is assumed to be identical and some leeway is given to the proposers to avoid "difficult" or unresponsive proposals, defeating the purpose of a thorough audit of a relatively narrow area of Catalyst (community development). As the proposers have big ambitions for their own branding and community visibility, this might be taken into a direction of "shaming" competitors of the proposers own projects in community building. I therefore doubt that this proposal is likely to be implemented with adequate success.

Confirming your humanity

Write comment

Replying to

Commenter gravatar

The teams backing this have been funded multiple times by Catalyst for similar development projects, and all are heavy to the development skills side. To the extent that databases, public sheets, plugins and applications are delivered the team is highly qualified for those tasks. But the challenge for this proposal is going to be developing protocols, onboarding reluctant proposers who will be very busy with existing auditing requirements, and compiling quarterly reports that impart the needed information clearly and succintly. Here the team has much less visible experience. The team leads do claim 8 years of combined work with this kind of data and analysis, and PIE is a typical protocol for their background fields (planning and urban development). The team is wisely limiting the scope of this prototype, focusing on a group of self-similar Community Development challenge settings which likely won't have highly technical audit requirements, and only attempting to onboard 30% of the proposals in that setting (prob 60-70 proposals in my estimation). One of the biggest challenges here is getting cooperation from busy projects…the ability of the team to conduct collaborative outreach will be critical. If the team is able to engage and lead these audits, they could potentially provide the results to the projects for them to submit in the required monthly progress reports of IOG, which could actually improve the reporting rate within this cohort (and would a side effect be that these projects would stay on track better?) The budget is large but the project spans 12 months on those funds, so the pay rates for the work is actually very competitive. However it is worth noting that the low price of ADA at the time of this proposal has made the team include a "price slippage" line item, a full 1/4 the value of the whole proposal, and that line item makes the total budget much less competitive in this challenge.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_1193
Total QA Ratings
14
QA Rating Outcome
Confirming your humanity

Write comment

Replying to

Commenter gravatar

The feasibility of this proposal is well above average, I think it deserves top marks. Starting with the fact that we have a very strong script: first, a period of time is established, for example, from month 1 to 3, in which various activities would take place. These activities are described below. I think the way it is presented to us, the script conveys a lot of confidence in the process, as apparently the team has already planned and organized what would be done very precisely. What could still improve, in my opinion, is if this time division were monthly in all cases, since the first three months are still considered together, so that a very detailed linearity was explained to us, which would increase even more. more the level of understanding in relation to the construction of the process. Regarding the budget, I think they ask for a fair value that is compatible with the complexity of the proposal, and unlike most proposals, the proposers do not reserve an exclusive session to talk about the budget. They do this already tied to the script, which is very good, because we know that certain activities would cost an amount X, and that already allows us to have a very good understanding of how the requested funding would be used. Even in terms of working hours, we know how much each small part of the whole would cost, how much time to perform each task would cost, that is, I think this part of the budget is very well explained as well. Finally, the team presents its members in a very complete and detailed way. First they say their names, main functions, attached a series of profiles of social networks, including linkedin, that is, we have a way to prove what is being said, and finally a small (but satisfactory) explanation of what they have been doing is attached. So far, what are your previous experiences and so on. Apparently it is a very well qualified team that is fully capable of carrying out this project successfully. So I think the only thing I suggest to make the feasibility even better is that the script be organized on an even smaller timescale, increasing the level of specification. Other than that, everything is in order, apart from the fact that the team has strong partnerships with other teams already funded (I know the team's proposals 2 minutes review and it really is a very interesting work).

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_57
Total QA Ratings
10
QA Rating Outcome
Confirming your humanity

Write comment

Replying to

Auditability

3.8 / 5
4 Reviews

Does the proposal provides sufficient information to assess and audit progress and completion?

Commenter gravatar

This proposal would be implemented over more than a year and has a lot of ambitious goals how to create a framework and even ultimately evolve into a DAO, but does not present good auditability metrics which is unfortunate in this category. After over a year after funding, only 2 videos will be created and 8 reports issues. There is no breakdown of the milestones, targets or metrics prior to the end of the assessment phase, and the proposal itself will be very difficult to audit over check-ins and its own audit for the long-term disbursal of funding. Social media views of the final report are optimistic and should not be the main objective of an audit. No discussion of possible qualitative goals except a survey are provided, and it is limited to Fund-8 proposers only and excludes other stakeholders.

Confirming your humanity

Write comment

Replying to

Commenter gravatar

The project is focused on growing auditability by evaluating the implementation result of Catalyst’s funded proposals with systematic framework. The work is to be done on a clear a clearly outlined framework

Confirming your humanity

Write comment

Replying to

Commenter gravatar

How does one audit the auditer? For starters, the deliverables are 4 quarterly reports, starting with a first report that writes up all the indexes and protocols used, as well as the makeup of the first cohort of funded proposals. So every 3rd progress report should have a deliverable associated with it. Presumably this same method and protocol could be used to issue the monthly KPI reports as well, in the form of % completes towards the quarterly reports and number of funded proposals onboarded and tracked. The final deliverable for the project, a video report summarizing the results of the prototype and lessons learned, including the disposition at that time of the audited proposals that collaborated (which may have interesting results compared to their peers), also will able to be used as the final completion requirement of IOGs current auditing protocol, so that is one less thing the team will have to work on to reach completion status. The backing teams for the proposal are well known in the community, and their support will no doubt enable the leads to effectively stay in touch and stay on track. In addition, due to the nature of needing to network with proposers, they will probably be required to maintain a public communications channel which could be another means of maintaining transparency through this process.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_1193
Total QA Ratings
14
QA Rating Outcome
Confirming your humanity

Write comment

Replying to

Commenter gravatar

The auditability of the proposal is good, but I don't think it's ideal. The team is concerned with providing various KPIs and statistical data that can serve as material to analyze for measuring progress. However, how will this data reach the community? It is not feasible that such things are defined and that they do not have a direct path to reach the members of the Cardano community. Therefore, I think it would be necessary to think of ways to make such information always available and updated, in addition to being visible to us. The team can create some online tool, such as Miro, Notion, Google Docs, etc., that serve to host such metrics and make them reachable for members of the Cardano community. Updates could be made monthly or weekly, and these directly posted there and forwarded to the community through Discord, Telegram or other communication channels we already have. Regarding success and final results, I believe that more or less the same thing happens: we know the ideal scenario to be reached, but we don't know how we could visualize it. In this case, as the information is more extensive and requires a further level of detail, I think it would be necessary for the team itself to contact the community in order to summarize everything that has been achieved. This can be done through synchronous online meetings, participation in Town Hall, for example, or even videos recorded and forwarded in the community media that I have already mentioned. In this way, I believe that auditability would be better explored and ensured, and that we would have more control over project monitoring.

Assessment Quality Assurance

Assessment Quality Assurance is an offered role to veteran in the Cardano Project Catalyst Community. The purpose is to review PA assessments of proposals, providing a second layer of Quality Assurance.

Assessor ID
z_assessor_57
Total QA Ratings
10
QA Rating Outcome
Confirming your humanity

Write comment

Replying to

Comments

No comments yet…

avatar
You can use Markdown
EP1: 'd' Parameter
0:00
/
~1:00
1x