Nothing came up for

Oracle Performance Metrics

$3,000.00 Received
$3,000.00 Requested
Problem:

Decentralised applications and the oracles on which they depend have a need to demonstrate their reliability and utility over time.

Challenge: F5: Metadata challenge
completed Awarded 6% of the fund.
Community Advisor Reviews:
4.7 (14)
Yes Votes:
₳ 250,852,558
No Votes:
₳ 13,791,667
Unique Wallets:
1212

Experience

Data Science professionals who have been responsible for building/deploying/monitoring analytics products in highly regulated environments.

Solution

Periodically store performance metadata that has been appropriately signed for authenticity and provide a tool for aggregating this data.

The Plan Use the Cardano blockchain to periodically store performance meta data for deployed services such as oracles, and machine learning algorithms. Create a tool capable of querying this meta data and aggregating metrics to produce a performance summary of automated services. Publicly accessible via the blockchain and appropriately signed for authenticity, this meta data can be aggregated and summarised over a time series to assess reliability and utility of deployed services, providing added transparency, monitoring and validation throughout the system. This meta data and metrics could be used by organisations who maintain oracles as well as by communities who maintain decentralised oracles. Furthermore, the communication of performance metrics amongst machine-learning algorithms enables reinforcement learning use cases and distributed AI a la SingularityNET. Some background and rationale for the proposal can be found here: https://flyncd.github.io/ The Team Each team member brings over a decade of experience as Data Science professionals responsible for building, assessing, deploying and monitoring credit risk, fraud, sentiment analysis, graph analytics and propensity models. This is the team’s first Catalyst proposal. Team members have sufficient experience to deliver all five proposal deliverables within a single calendar month without sourcing additional developer cost. Team Members Michael - https://www.linkedin.com/in/michael-hodder-417a4ba7/ Thushare - https://www.linkedin.com/in/thushare/ Steve - https://www.linkedin.com/in/steve-pirois-8b373593/ The Deliverables Define generic assessment criteria for automated services Performance reporting should be as flexible as possible to support different use cases (data feeds, price oracles, Machine Learning bots, risk models) while providing enough meaningful information that can fit within the 16k meta data limit.

Deliverable #1 - Extend the schema.org SolveMathAction and MathSolver schemas to create a standards-based structure for or registration and assessment of automated services. Choose a specific application to create a proof of concept reporting tool There is a great collection of dApps currently being built on the Cardano ecosystem. The qCreditScore proposed by Liqwid Finance, is a good example of an automated service where users would desire periodic assessments of the score’s predictive-power, accuracy, stability and calculation. Adding this layer of transparency and validation to the DeFi system would serve to promote take up and usage. Deliverable #2 - Create a simulated demo of a “Credit Risk” model which is capable of assessing the likelihood that an entity (such as an Atala Prism Decentralised Identity) will default in the next 3 months. Deliverable #3 - Produce a script to periodically collate and publish metrics. The summary of which is published as a blockchain transaction. Demonstrate reporting of metadata Reporting on stored metadata, particularly using metrics that can be assessed over long time periods is where the long-term utility of a proposal such as this will help the Cardano community. Deliverable #4 - Provide a tool which can query and aggregate metadata stored on the Cardano blockchain that targets a specific metric (eg Gini or KS for the qCreditScore model). The tool should be portable enough so that any user capable of querying the underlying Cardano blockchain metadata can produce reports. Report on findings Once the project is finalised, write up the results and publish to the project website. Deliverable #5 - Add a supplementary section to project website detailing lessons learnt, unsolved questions and future work. FAQ Why is the blockchain an appropriate store of this information? A decentralised, transparent reporting of performance metrics for oracles on a blockchain ensures authenticity of the performance data. Transparent reporting and verification of these metrics over time can produce an authentic permanent record which can be assessed using time series analysis. What data will be used to build the credit score simulation? Peer-to-peer lending data. For example, the Udacity dataset for Prosper. What will you use the requested funds for? Funds to be used as remuneration for one month of development effort, standing up development infrastructure for testnet and a mainnet demonstration application. What is the desired outcome of this project? The goal is to see the framework and tool being used amongst the Cardano community, ideally making future projects possible and successful. In the medium to long term, we hope to help the community to develop future projects and/or improve tools by releasing details about the successes, pinch points and other lessons learnt during the project development.

Definition of Success

Received emails from [email protected], How my proposal impacts the challenge metrics, Broken down my budget requirements, Defined expected public launch date., How I address the challenge question, Submitted this proposal to only one challenge, Definition of success after 3, 6 and 12 months, Included identifying information about all proposers

Community Advisor Reviews

Addresses Challenge

5 / 5
5 Reviews

Does the proposal effectively addresses the challenge?

Community Reviews (5)

Commenter gravatar
October 23, 2021

This proposal identifies a trust problem in oracle data and proposes a multifaceted solution using the cardano metadata.

Confirming your humanity

Write comment

Replying to

Commenter gravatar
October 23, 2021

This proposal address the clear (and soon to be pressing) need for assuring the reliability and performance of the oracles upon which smart contracts and other services depend. It falls squarely within the metadata challenge because it analyzes on-chain oracle metadata and also records its results as transaction metadata. (In fact, the tool could be used to measure its own reliability.) The emphasis on the transparency and the availability of the framework is righly important for building confidence it in and in the oracles that it assesses. The proposal did not justify the necessity/relevance of the credit-risk demo for its goal of providing oracle reliability metrics.

Confirming your humanity

Write comment

Replying to

Commenter gravatar
October 23, 2021

The authors propose to monitor Dapp and oracle performance, publishing results to the blockchain. The data can later be aggregated and reported. This is a great use of metadata and will strengthen the entire DeFi/Dapp ecosystem of Cardano.

Confirming your humanity

Write comment

Replying to

Commenter gravatar
October 23, 2021

This is a good use case for the immutability of the blockchain. I would add that just have the ability to store predictions immutably when they are made to later go back access their accuracy would be very useful

Confirming your humanity

Write comment

Replying to

Commenter gravatar
October 23, 2021

Oracles will be provinding decition making inputs to smart contracts. There is a need to independently evaluate these inputs by more than one agency to arrive at a robust quantification. The metric system will enable comparison between oracles. Therefore the proposal will fill a void and provide reliability to the entire system when implemented.

Confirming your humanity

Write comment

Replying to

Feasibility

4.6 / 5
5 Reviews

Given experience and plan presented is likely that this proposal will be implemented successfully

Community Reviews (5)

Commenter gravatar
October 23, 2021

The Authors are experienced in deploying solutions in highly regulated environments. Their linkedin profiles are available. They expect to produce the deliverables in 1 month time. My only question is their familiarity/technical understanding of the blockchain aspects of the proposal. Nevertheless this looks like a team that can achieve the goals of the project.

Confirming your humanity

Write comment

Replying to

Commenter gravatar
October 23, 2021

The proposers have background in the area and should have the skills to deliver. They are also asking for a very small sum.

Confirming your humanity

Write comment

Replying to

Commenter gravatar
October 23, 2021

Verifiable credentials have been provided by the proposer and they have the required qualification and experience to complete the task.

Confirming your humanity

Write comment

Replying to

Commenter gravatar
October 23, 2021

Based on the experience of the team, insight presented in the proposal, and clear scope definition there is a great chance that this will be implemented successfully.

Confirming your humanity

Write comment

Replying to

Commenter gravatar
October 23, 2021

The proposal team is well qualified, and the description of the plan and deliverables describe a viable and achievable approach. Accomplishing the goals (five deliverables) in a single calendar month seems overly ambitious, but none of the goals are particularly risky. The proposal is also not specific about the language/platform on which the tool/framework will be implemented–that makes it more difficult to assess the software-development risks.

Confirming your humanity

Write comment

Replying to

Auditability

4.5 / 5
4 Reviews

Does the proposal provides sufficient information to assess and audit progress and completion?

Community Reviews (4)

Commenter gravatar
October 23, 2021

The proposed system has been covered in detail and an adequate amount of documentation has been included to sustain the claims. The road map and the phases are achievable and pragmatic.

Confirming your humanity

Write comment

Replying to

Commenter gravatar
October 23, 2021

this proposal’s success would be easily tracked by whether or not the metadata schema is added to the blocks, and whether the data is explorable.

Confirming your humanity

Write comment

Replying to

Commenter gravatar
October 23, 2021

All deliverables are specified in enough detail to be definitively assessed for progress. However, it would have been helpful to quantify terms like “periodically”, to detail what is meant by “portable”, and to discuss data volumes or other numerical metrics.

Confirming your humanity

Write comment

Replying to

Commenter gravatar
October 23, 2021

Not a lot of detail is provided as to how the funds will be allocated, but they are asking for only 3k and expect to produce a product in a short time. It will be easy to judge the success of the proposal. I imagine they will come back for more funding in future rounds to extend the project. I think this is a great model for building trust on the Catalyst platform.

Confirming your humanity

Write comment

Replying to

Impact

0 / 5
0 Reviews

The proposal is able to scale to address future challenges.