Risk Assessment Framework for Evaluating Lending Protocols: A Weighted Scoring Approach

Introduction

We have seen multiple innovative strategies in new lending protocols striving to provide equitable access to DeFi for every user. However, with these innovations comes a unique set of risks that requires careful assessment and thorough evaluation to ensure the security and stability of these protocols. As we continue building in this space, it is very important for DeFi protocol developers, builders and even users to understand these risks and how to mitigate them.

In this piece, we will dive into a comprehensive framework that is designed to evaluate the safety and health of lending protocols. We will also examine key parameters such as protocol architecture, codebase integrity, governance model, liquidity depth, rate predictability, liquidation mechanisms, bad debt history, external risk management, oracle security, asset depeg risk and capital efficiency. This framework works to provide a structured approach to risk evaluation for any DeFi app or protocol.

Each parameter is assigned a rating and weighted based on its importance to the overall security of the lending protocol. The weighted scores are then aggregated into an Ultimate Safety Score, which is a quantitative measure of the protocol's risk profile. This framework not only helps in identifying potential vulnerabilities but also enables users to make informed decisions when choosing a lending protocol.

Risk Assessment Framework

This is a structured approach we will use to evaluate the security, stability, and overall health of any lending & borrowing protocol. It encompasses a range of critical parameters, each contributing to the overall risk profile of a lending protocol.

These parameters are outlined in no particular order:

1. Protocol Architecture: The architecture of a lending protocol significantly influences its risk exposure. This parameter considers whether the protocol operates in an isolated, aggregated, or monolithic manner:

   Isolated: Risks are confined to specific markets, reducing systemic risk.

   Aggregated: Risks are distributed across interconnected markets, posing moderate systemic risk.

   Monolithic: A single structure handles all operations, increasing systemic risk.

2. Codebase Integrity: Codebase integrity is crucial for ensuring the security of the protocol. This parameter evaluates whether the protocol’s codebase is a direct fork with no changes, and whether it has been audited for vulnerabilities. It also assesses how the “known/discovered vulnerabilities” are managed and patched.

3. Governance Model: Effective governance is essential for the decentralized management of a protocol. The framework examines whether the protocol uses a multi-signature (multi-sig) setup or a Decentralized Autonomous Organization (DAO) for governance. The assessment includes the number and reputation of signers in a multi-sig model, the participation rate, and the transparency of the DAO.

4. Liquidity Depth (Collateral): This parameter evaluates the availability of liquidity for collateral assets in decentralized exchanges (DEXes). Adequate liquidity is crucial for efficient liquidation processes and minimizing bad debt during market downturns.

5. Rate Predictability: The predictability and stability of interest rates directly impact the risk for both borrowers and lenders. This parameter assesses the mechanisms used to set and adjust rates, as well as the historical data on rate fluctuations.

6. Liquidation Mechanism: The efficiency of the liquidation process is vital for maintaining the health of the lending protocol. This includes assessing the transparency and speed of the liquidation process.

7. Bad Debt History: Historical data on bad debts helps in understanding the protocol’s risk management capabilities. This parameter evaluates the incidence of bad debts and the effectiveness of measures taken to mitigate them.

8. External Risk Management: This parameter looks at the involvement of third-party entities in managing risks. It includes assessing the credibility and performance of these external parties.

9. Oracle Security: Oracles are crucial for providing accurate data feeds to the protocol. This parameter examines the reliability, accuracy, and decentralization of the oracles used, as well as any potential single points of failure.

10. Asset Depeg Risk: The stability of the assets used as collateral or lent out is critical. This parameter evaluates the risk of these assets losing their peg (de-pegging), and the protocol’s response to such events.

11. Capital Efficiency Score: This metric measures how effectively the protocol uses collateral to generate returns. A higher capital efficiency score indicates better utilization of assets and potentially higher returns for users.

Ultimate Safety Score: To provide a comprehensive assessment, each parameter is rated on a scale of 1% to 20% and weighted based on its importance. The weighted scores are aggregated to calculate an Ultimate Safety Score, which offers a quantitative measure of the protocol’s overall risk profile.

Parameter Weights (Σ100%)

• Protocol Architecture: 9%

• Codebase Integrity: 18%

• Governance Model: 8%

• Liquidity Depth (Collateral): 11%

• Rate Predictability: 9%

• Liquidation Mechanism: 13%

• Bad Debt History: 8%

• External Risk Management: 6%

• Oracle Security: 8%

• Asset Depeg Risk: 5%

• Capital Efficiency Score: 5%

Here is the logic behind the rating ⬇️

When we evaluate these parameters, the importance of each one depends on the specific goals and risk tolerance associated with it.

A typical example would be this;

User A can argue that Codebase Integrity(16%) > Capital Efficiency (5%), in other words... It is more important that your assets are in a more secure protocol than making more yield.

Now in order to calculate the final result, which is the Ultimate Safety Score we simply multiply each rating by its corresponding weight and sum the results.

Protocol Evaluation: Using Compound as a Case Study

1. Protocol Architecture: Compound operates with a monolithic architecture. All functionalities—such as lending, borrowing, interest rate calculations, and liquidation processes—are managed within a single, cohesive protocol structure.

2. Codebase Integrity: Compound’s codebase has been audited and critical vulnerabilities fixed. You can check out OpenZeppelin‘s audit here.

3. Governance Model: Compound uses a Decentralized Autonomous Organization (DAO) governance model. Holders of it’s native token (COMP) participates in decision making process.

4. Liquidity Depth: The liquidity depth for Compound's main collateral assets as at time of this writing is as follows:

• Total 24-hour trading volume: $250,000,000

• Average price impact: 0.00925 (or 0.925%)

  This indicates good liquidity and trading volume. It also suggests that assets can be liquidated efficiently with a minimal market disruption.

5. Rate Predictability: Compound employs a dynamic rate model where interest rates are adjusted based on the supply and demand for each asset. As the utilization rate (the ratio of borrowed assets to supplied assets) increases, the interest rate for that asset also increases to incentivize more supply and less borrowing, and vice versa. Although the rates are algorithmically determined and are generally predictable, they can still experience significant changes due to market dynamics.

6. Liquidation Mechanism: The liquidation process in Compound is very efficient. See more info on the determination of liquidation factors and functions responsible for calling the account liquidation here.

7. Bad Debt History: There have been some instances of bad debt in the history of Compound.

For example, the DAI Liquidation Event in Nov 2020 and Oracle Issues and Liquidations in March 2020. But as of now, there has been minimal issues of bad debt and the protocol‘s developers and community are continuously working to improving the protocol's resilience to market volatility and technical issues.

8. External Risk Management: There are a lot of external risks that are managed by the protocol which includes oracle usage for price feeds, third party insurance protocols, protocol reserve, audits and security assessments, risk assessment of integrated protocols, community and governance.

9. Oracle Security: Compound uses Chainlink Price Feeds and API3 as an oracle solution

10. Asset Depeg Risk: In the past, there has been events where an some assets on the Compound Protocol lost it’s peg, especially during high market volatility and liquidity imbalances. As of now, Compound takes a comprehensive approach to managing asset depeg risk, employing a combination of conservative collateralization ratios, diverse collateral options, efficient oracle systems, and community governance. While the risk of de-pegging cannot be entirely eliminated, these measures help minimize its impact on the protocol's stability and protect users.

11. Capital Efficiency Score: The capital efficiency score for Compound is high due to several factors which includes; Interest Rate Model (IRM), collateral utilization, asset liquidity, automated market operations, ecosystem integration, liquidity mining and incentives.

Assessment Table For Compound With Ratings

Step-by-Step Calculation:

1. Protocol Architecture: 6 × 0.09 = 0.54

2. Codebase Integrity: 16 × 0.18 = 2.88

3. Governance Model: 5 × 0.08=0.4

4. Liquidity Depth (Collateral): 11 × 0.11=1.21

5. Rate Predictability: 7 × 0.09=0.63

6. Liquidation Mechanism: 13 × 0.13=1.69

7. Bad Debt History: 5.5 × 0.08=0.44

8. External Risk Management: 5.5 × 0.06=0.33

9. Oracle Security: 7 × 0.08=0.56

10. Asset Depeg Risk: 3.5 × 0.05=0.175

11. Capital Efficiency Score: 5 × 0.05=0.25

Summation of weighted scores gives:

$$$
0.54+2.88+0.4+1.21+0.63+1.69+0.44+0.33+0.56+0.175+0.25 = 8.165
$$$

∴ The Ultimate Score for the Compound Protocol using this framework = 8.165

Conclusion

The comprehensive risk assessment framework outlined in this series provides a robust methodology for evaluating the safety and health of lending protocols in the DeFi space. By examining critical parameters such as protocol architecture, codebase integrity, governance models, and liquidity depth, this weighted scoring approach offers a structured and quantitative measure of a protocol's risk profile.

Through a detailed case study of Compound, we demonstrated how each parameter contributes to the Ultimate Safety Score, highlighting the strengths and potential vulnerabilities of the protocol. This framework not only helps in identifying and mitigating risks but also empowers developers, builders, and users to make informed decisions when selecting lending protocols. The ratings given were mostly logical and derived using available resources and data from the web. Any entity or individual can give a different rating to this, but would still be within the same range of safety scores.

We invite anyone to add their comments and suggestions about the framework to help it mature. DeFi protocols, apps and projects are also welcome to use this methodology and approach in scoring their platforms. As an experimental evaluation tool, we are designing this framework to evolve based on your feedback, ensuring it remains relevant and effective.

💜