funded
Db-Sync Enterprise: high-availability setup of Cardano’s Db-Sync to run queries and access on-chain data as a service
Current Project Status
in_progress
Amount
Received
₳25,200
Amount
Requested
₳60,000
Percentage
Received
42.00%
Solution

We are developing and maintaining a high-availability infrastructure running Db-Sync to minimise downtime. Cardano projects can be hooked up to our Db-Sync Enterprise solution in short time.

Problem

Db-sync is the main system for accessing Cardano on-chain data. Unfortunately the cost of running it is high, setting it up is complex and it is not very stable: having to fully re-sync takes days.

Impact Alignment
Feasibility
Value for money
  • Project Information
  • Community Reviews
  • Team Information
Db-Sync Enterprise: high-availability setup of Cardano’s Db-Sync to run queries and access on-chain data as a service
Start date
-
Milestones
-
Next Milestone
-

Please describe your proposed solution.

We will offer on demand all the on chain data in a relational database that can be used by any client. In this way the pain of querying on-chain data, combining realtime and historical data will be mitigated.

Please define the positive impact your project will have on the wider Cardano community.

We will offer to Cardano’s ecosystem a low barrier entry for querying realtime and historical on-chain data. It will be “Db-Sync as a Service”, pay as you go, and scalable to the user’s project needs.

What is your capability to deliver your project with high levels of trust and accountability? How do you intend to validate if your approach is feasible?

We have more than two years experience with our own developed Db-Sync Enterprise protocol which minimises downtime of a Db-Sync pipeline.

The project team covers all aspects required to achieve the stated goals.

What are the key milestones you need to achieve in order to complete your project successfully?

M1: setup redundant and high-availability infrastructure for the project (2 months)

This process entails setting up the servers to run the Cardano node, db-sync, and a Postgres database. The setup includes configuring two fully functional db-sync pipelines, including the mentioned components. The criteria for acceptance will focus on ensuring these pipelines run smoothly, testing failover capabilities.

>M2: create monitoring and dashboard view (2 months)

This milestone's outputs will be the monitoring of the whole system. It will involve monitoring and alerting at different levels: underlying servers, running systems, database usage.

Server monitoring:

We will monitor CPU, I/O, disk and memory for every server, setting up relevant alerts for unexpected usage levels,

Running systems:

We will monitor process health and uptime

Database usage:

We will monitor typical database metrics (replica lag, queries) as well as db-sync related metrics (eg. block height lag).

The acceptance criteria will be having 3 dashboards where all the server/process instances will be monitored and test the successful delivery of alerts for all the relevant metrics

>M3: automatise user signup (2 months)

This milestone involves setting up a web App, where each user will be able to signup and select what type/how many database instances would like to use. After the successful signup, the relevant resources will be created automatically and the connection details will be shared with the user.

The user will have the option to tear down the spawned instances and pay only for the time they were used.

Acceptance criteria: Perform a successful user signup and use the connection details to connect to the read replica and perform a query successfully. Perform a successful tear down. Validate pay-as-you-go charging.

Who is in the project team and what are their roles?

Alexander Diemand (cardanobigquery - <https://www.linkedin.com/in/axeld/>): architecture & design, project management, communication, documentation

Thomas Kaliakos (thomaska - <https://www.linkedin.com/in/tkaliakos/>): data engineering, data quality responsibility, documentation

Bitseat Tadesse (bitseatt - <https://www.linkedin.com/in/bitseat/>): data science, social networks, documentation

Please provide a cost breakdown of the proposed work and resources.

B1: 20 PD architecture & design, project management, communication

B2: 20 PD system engineering (Devops) for pipeline setup

B3: 20 PD software engineering

B4: 20 PD data science: dashboard development

B5: hardware costs: est. $1000 per month (x6 months)

(PD = person day; 8 hrs/day; 1 hr = $90)

Total Budget: $63600

How does the cost of the project represent value for money for the Cardano ecosystem?

By sharing our redundant high-availability Db-Sync pipeline we can provide users with a sophisticated and reliable Db-Sync connection for a fair price.

We believe this offering enables a number of Cardano projects as it lowers entry barriers.

Community Reviews (1)

Comments

Monthly Reports

close

Playlist

  • EP2: epoch_length

    Authored by: Darlington Kofa

    3m 24s
    Darlington Kofa
  • EP1: 'd' parameter

    Authored by: Darlington Kofa

    4m 3s
    Darlington Kofa
  • EP3: key_deposit

    Authored by: Darlington Kofa

    3m 48s
    Darlington Kofa
  • EP4: epoch_no

    Authored by: Darlington Kofa

    2m 16s
    Darlington Kofa
  • EP5: max_block_size

    Authored by: Darlington Kofa

    3m 14s
    Darlington Kofa
  • EP6: pool_deposit

    Authored by: Darlington Kofa

    3m 19s
    Darlington Kofa
  • EP7: max_tx_size

    Authored by: Darlington Kofa

    4m 59s
    Darlington Kofa
0:00
/
~0:00