This paper presents a rigorous methodology for benchmarking the capabilities of an FPGA family. The goal of benchmarking is to compare the results for one FPGA family versus another over a variety of metrics. Since the FPGA industry does not conform to a standard benchmarking methodology, this white paper describes in detail the methodology employed by Altera and shows how a poor methodology can skew results and lead to false conclusions.
The complexity of today’s designs together with the wealth of FPGA and computer aided design, or CAD, tool features available make benchmarking a difficult and expensive task. To obtain meaningful benchmarking results, a detailed understanding of the designs used in comparisons as well as an intimate knowledge of FPGA device features and CAD tools is required. A poor benchmarking process can easily result in inconclusive and, even worse, incorrect results. Altera invests significant resources to ensure the accuracy of its benchmarking results.