Functional Verification Challenges and Trends in Multi-Million Gate Designs

Summary:  

Focusing on functional verification in this article, we shall demonstrate some of the functional verification challenges and trends from the block to System level in today's System On Chips (SOCs).

Over the last decades, design sizes have increased to million fold since Moore’s law was simply put in 1965. Named after Gordon E. Moore co-founder of Intel Corporation, Moore’s law is used in the semi-conductor industry to guide the long term plans and set targets for research and development. The increasing size and complexities of today’s designs introduce many challenges to the design and verification goals, tasks, processes, and risks. All previously mentioned challenges, and more, result in a noticeable increase in the effort spent on verification, and have forced the industry to adopt various new functional verification techniques.

Focusing on functional verification in this article, we shall demonstrate some of the functional verification challenges and trends from the block to System level in today’s System On Chips (SOCs).

functional-verification

Introduction to Functional Verification

 

Functional Verification is the task of verifying that design functionality conforms to specification. A recent wide study by Wilson Research group[4], and commissioned by Mentor Graphics, has shown that the mean verification effort in nowadays’ SOCs is even greater than the effort to design and implement them in the first place. The study shows that the reason for the significant magnitude of verification efforts can be understood when we take a look at today’s SOC complexities:

  • Median number of gates in a SOC is ~12.5 Million.
  • Median number of cores in a SOC is ~2.25.
  • Median number of clocking domains in a SOC is ~9.36.
  • Median verification resources dedicated for power management represent 20-29% from overall verification resources.
  • The adoption of formal technology had increase by ~68% in the last few years.
  • At least ~70% of industry use code coverage, functional coverage, and assertion based verification for verifying their SOCs.
  • The adoption of constrained random verification has increased by ~39% in the last few years.
  • HW acceleration and emulation adoption has boomed w.r.t. FPGA prototyping for scalability, visibility, and solutions around emulation reasons.
  • More than half of the survey participants are behind their current projects schedule.
  • Almost half of the survey participants require a second re-spin.
  • In order to reduce the number of new code anomalies introduced, there are global trends to reuse previously implemented logic, or purchase third part IPs rather than implementing new IPs from scratch.

 

Top Most Challenges Industry Wants Solved

 

Typical nowadays functional verification environments encompass automation, coverage-driven constrained random verification, assertion-based verification, transaction level modeling, low power, and clock domain crossing verification. While sophisticated verification environments may include formal verification, analog-digital mixed signal verification, and software-hardware co-simulation, HW emulation and acceleration. Indeed, many verification technologies and methodologies adopted by the industry have succeeded in improving the verification process in general, yet many verification challenges still exist in managing the verification process, writing/maintaining tests, decreasing time to debug, reaching coverage closure, and tracking projects momentum efficiently.

Key Aspects to Overcome Verification Challenges

 

To overcome functional verification challenges, one has to keep the following in mind:

  • Adopt a process before tools
    • Adopt a process that suits your needs rather than focusing on implementing and/or adopting tools.
    • A study by Cisco Chips Momentum Research Group shows that by adopting a process at first followed by adopting the right tools and solutions for it often results in 20-30% cost savings.

  • Improve your productivity

A key aspect for delivering and meeting targets is productivity. Productivity can be measured by four different aspects:

  • Abstraction:
    • Abstraction is the process of filtering out irrelevant details from the basic pattern or structure.
    • Abstraction is a key aspect in verification productivity as it helps in building the verification environment in a much more efficient way that is not prune to many of the RTL design problems.
    • Mechanics of abstraction in functional verification include, but not limited to:
      • High-level modeling testbench components and blocks (e.g. IPs reference models, functional coverage models, scoreboards, random stimulus).
      • Defining/building transactions.
      • Routing transaction between different testbench blocks.
      • Defining and injecting stimulus.
      • Separation of test scenarios from testbench.
  • Reuse:
    • There are two types of reuse:
      • Horizontal reuse: is the ability to make use of previously created components and/or subsystems in current environments.
      • Vertical reuse: is the ability to make use of the same environment across multiple stages of the verification cycle (e.g. block to system level verification, simulation to emulation migration, etc.).
    • As can be predicted, reuse is very important in minimizing implementation turnaround times.
  • Performance:
    • When it comes to functional verification, performance is a key aspect that is often locked inside people minds to simulation runtime performance. While simulation runtime performance is important, in reality performance is much more than that.
    • Runtime Performance
      • Is the factor that measures how long your simulation takes. This more or less depends on the complexities of your environment, source code, efficiency of third party blocks, and how fast is your simulator.
      • Environment
        • It is always important to revise your current environment to observe its optimality. Make sure that you are doing all what you are supposed to do in an efficient way, i.e. nothing in what you do could be done easily in a much more efficient way.
      • Coding
        • When it comes to coding, often designers and verification engineers focus on functionality but not on efficiency and performance. This can result in surprises at runtime that simulation is taking too long. In most cases, writing code keeping performance and efficiency in mind could result in significant boosts to performance. The question that is often raised is: why the tool/simulator does not do these optimizations implicitly/automatically when executing my code? The answer is, it is not always easy and straightforward as it seems; while some of the coding styles are already (or can be) implicitly optimized by simulators, other styles could not due to various complexities. So it is important to profile your code at runtime to spot areas of unexpected overhead.
      • Tooling:
        • Make sure that you are configuring simulators (and other third party/in-house tools) in an optimal manner. In my experience I have seen the lack of tool knowledge could result in disastrous scenarios when it comes to performance. Always check the simulation configuration; vendor’s support resources can help in this front.
    • Time to Debug:
      • It is the time taken to track down the root cause of a failure.
      • Debug time is one of the most (if not the most) verification tasks consuming time, resources, and overhead.
      • Key solutions to boost time to debug:
        • Very well written code with clear comments.
        • Efficient, scalable, and wealthy debug tooling/solutions.
        • Solutions to group and sort common failures across regression run.
    • Debug turnaround time:
      • Development turnaround time is the time taken to re-run after a design change during normal development or debugging, i.e. summation of build, compilation and elaboration times.
      • Key solutions to boost time debug turnaround time:
        • Efficient packaging of tests in the verification environment to avoid multiple compilations, i.e. compile snapshot once, run multiple different tests many times using same snapshot.
        • Efficient process that incrementally re-builds only what has changed.
    • Regression run performance:
      • Regression performance it the time taken to run a full set of regressions.
      • Regression performance more or less depends on:
        • Number of tests in a regression run.
        • Simulation runtime per test.
        • Clusters compute resources performance.
        • Efficiency of the verification run management process.
      • Key solutions to boost regression run performance:
        • Analyze tests effectiveness w.r.t. verification plan goals and objectives.
        • Run only effective tests that add value.
          • Redundant tests could either be improved or taken out from regression.
        • Capture the most effective tests, often called acceptance tests, in a mini-regression. For a designer or a verification engineer to qualify his changes quickly, with an acceptable degree of risk, he can use the acceptance regression instead of waiting for a complete regression run.
    • Metrics Closure:
      • Metrics closure is how fast and accurate one reaches his verification goals and objectives.
      • Typically people define the different metrics that they want to collect, measure, analyze and cover throughout the verification cycle.
      • The most popular metric is coverage; functional and code coverage.
      • Talking about coverage, the key aspects towards closing coverage:
        • Effective and non-redundant tests (as much as possible).
        • Efficient stimulus generation.
        • Different and scalable scenarios covering corner cases.
  • Automation:
    • Automation is the tool needed to make all of the previous talks easier and applicable.
    • Typically you need automation in all design and verification phases:
      • Verification plans construction, execution, and tracking throughout the entire project cycles.
      • Metrics closure and project momentum tracking.
      • Generation of testbench skeleton and structure.
      • Self-checking tests.
      • Generation of testbench stimuli (e.g. graph based stimuli).
      • Design and testbench linting.
      • Managing Regression runs: Execution, monitoring, dumping.
      • Etc.

Ensuring scalable, efficient, and suitable process together with all productivity aspects from block to system level verification is a key to success.

In the next sections we will discuss the challenges in each of the verification stages from block to system level, while shedding some light on most popular techniques currently being used to overcome them.

Block Level Verification

 

Design blocks or IPs are the ingredients of subsystems or SOCs. Each block undergoes testing tasks specific to its verification goals and objectives.

When it comes to block level verification, the primary verification goals are:

  • Verify that each block does everything that it is specified to do.
  • Verify that each block does not do anything that it is not supposed to do, i.e. negative testing.

While primary verification tasks are:

  • Develop verification plans.
  • Develop testbench.
  • Develop measurements metrics and coverage models.
  • Develop tests scenarios and stimulus.

On the other hand, challenges to overcome in block level verification are:

  • How can I create my testbench faster?
  • How do I create more tests?
  • How do I know I have thought of the corner cases?
  • How do I get to coverage closure faster?
  • How can I verify initial states, reset, and X’s source/effects?

In order to smoothen verification challenges at the block level you need to build an environment that can encapsulate:

  • Reuse
    • Build testbench with reusable components.
    • Separate tests from testbench, and testbench from Design Under Test (DUT).
  • Abstraction
    • Write tests in sequences and scenarios form.
    • Drive transactions to DUT, not pin wiggles.
    • Debug transactions not just signals.
  • Automation
    • Self-checking tests
    • consistent place for coverage
    • Enable tool automation with structure

Directed vs. Constrained Random Verification

 

In the old days, a straightforward Directed Verification (DV) was the main method used to verify the Device Under Verification (DUV). Simple as it is, writing a test for every single DUV feature, it is not a scalable nor an efficient method to verify modern designs This is because their size and complexity would result in an exponential increase in the number of directed tests, as well the time taken to create and maintain them.

Over the years, Constrained Random Verification (CRV) became the market focus. CRV in its most ideal form was seen as a very effective way in improving verification quality; it is easier to build a single Constrained Random (CR) test that is equivalent to many directed tests (despite the fact that building a CRV environment would be more complex than its DV counterpart would be). However, CRV cannot be used in a standalone manner; it needs to go hand-in-hand with a measurement strategy to assess the DUV verification progress. Here, the term Coverage Driven Verification (CDV) arose, in which a coverage model that represents different features of the DUV to be verified is built, and coverage is collected during CR tests run. Although CR stimuli provide a tremendous value towards faster functional coverage closure over directed tests, specific corner cases can be missed. This could be related to:

  • Complex coverage models built reflecting the DUV complexity.
  • Complex and/or inefficient (e.g. contradicting, in some cases) system constraints are coded that could overburden (or disable) a random constraint solver from generating interesting scenarios.
  • Often, longer tests’ simulation time is required, which affects resources and project timeline.
  • Missing the opportunity to cover interesting scenarios when random stimuli generators generate redundant stimuli.

The figure below shows the typical coverage closure of CRV against DV over project timeline‎[1]‎[2].

Indeed, CRV super exceeds DV in achieving faster coverage ramp up. However, as seen from the graph, coverage momentum decreases overtime. Since random distribution depends on how efficient the system random constraints are, as well as how efficient a constraints solver is in generating random stimuli, coverage seizes to progress at certain point at which engineers need to try different strategies to close coverage like:

  • Write more directed-like CR tests to cover specific corner case scenarios.
  • Run testcases multiple times with different random seeds, and hope more interesting scenarios are covered.
  • Alternatively, try out other methodologies (e.g. Formal Verification) when applicable.

Typically, verification engineers adopt a hybrid methodology encapsulating direct, constrained random, formal, and other methodologies to smoothen their verification process.

The Universal Verification Methodology

 

The Universal Verification Methodology (UVM) is a standardized methodology for verifying integrated circuit designs. Being released in 2010, UVM is based on the SystemVerilog language. The UVM open source library brings much automation to the SystemVerilog language such as sequences and data automation features (packing, copy, compare, etc.). Unlike the previous methodologies developed independently by the simulator vendors, UVM is an Accellera standard (and about to become an IEEE standard) with support from big EDA vendors Mentor, Cadence, and Synopsys. The UVM objectives and foundations make it a key to productivity at block level verification.

UVM foundations

 

  • Boosts productivity
    • Raise the abstraction level of stimulus and checking
  • Boosts turnaround time
    • Avoid expensive recompilation; compile once simulation multiple times use model.
  • Separates stimulus generation from delivery and test from testbench
    • Several people can develop stimulus, and testbench environment
  • Ensures Interoperability
    • Standard open source class library & API based on standard language (SystmeVerilog), and supported by all EDA vendors.
  • Facilitates Reuse
    • Horizontal and vertical reuse.

UVM Mechanics

 

A typical UVM testbench environment is demonstrated in the below figure. A UVM testbench is compromised from components. Components are modeled using SystemVerilog classes, and hence subject to Object Oriented Programing (OOP) concepts. Components communicate with each other using Transaction Level Modeling (TLM), where high level transactions are exchanged between the different components via ports and exports implementations. This offers high degrees of reuse, generalization and smoothness in building the environment.

Generally a UVM testbench consists of the following components:

  • Stimuli generator
    • Generates constrained random or directed stimuli (transaction level abstraction).
  • Controller
    • Controls test/testbench execution flow and semantics.
  • Agent
    • Converts high level transactions to pin level activity and vice versa.
  • Scoreboard
    • Checks and verifies DUV functionality against reference models.
  • Functional coverage
    • Scores achieved coverage against verification goals and objectives.
  • Environment
    • Components encapsulating all testbench components.
  • Config
    • Adapts testbench environment to various RTL configurations.
  • Test
    • Captures testing scenarios together with goals and objectives inspired from the Verification plan.

To be continued…

 

References

 

  1. Bruce Wile, John C. Cross and Wolfgang Roesner, “Comprehensive Function Verification”, Elsevier 2005.
  2. Sasan Iman, “Step-by-Step Functional Verification with SystemVerilog and OVM”, Hansen Brown Publishing Company 2008.
  3. IEEE Standard for SystemVerilog, Unified Hardware Design, Specification, and Verification Language, IEEE Std 1800-2009, 2009.
  4. Wilson Research Group, 2012 Functional Verification Study. http://www.mentor.com/products/fv/multimedia/the-2012-wilson-research-group-functional-verification-studyview
  5. UVM User Manual, uvmworld.org.
  6. https://verificationacademy.com

Related posts:

 

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

Facebook Comments