! "# $ % &#! ' ( ) + & - $* * ) 2 * & , $ . # ' & $ % & *& & !" #$ ) $) # )/ 0 1 0 2 % % & &' ' # $ 34 3 " *& ' 5 6"$ 7 " 8 .# # 3 "$& *& 39 $ 7 3 9 $ 7 # 3 ) : < ) ; & = & 4; > $ #' ? 3; A8 . & $ ' 3 $ ' ? @ # "; 4 # #( ( ) ) ' '* * + +, , # #- Verification Effectiveness on Switch Chips Problems Found By Method 250 240 230 220 210 200 190 180 Cumulative Issues 170 Chip Sim RuleBase Walk Thru Chip Sim II Gate Sim System Sim Lab Timing Designer Sim Element Sim 160 150 140 130 120 110 100 90 80 70 60 50 40 30 20 10 0 11/06/98 03/26/99 08/13/99 12/31/99 05/19/00 10/06/00 Week Ending 5 # #( ( ) ) ' ' * * + +, , # #- RuleBase Bug Discoveries PortMgt 1 CentrlMgt 8 Output Port 19 PortMgt Output Port Input Port CQ CentrlMgt CQMgt XBar LPE "The bugs I have found thus far [in the Switch Chip] using Formal Verification have been mainly control logic bugs in the Output Port and the Central Management Logic. I believe some of these would have been very difficult to hit in our Random Sim environment." R.G. Verification Lead Chip Sim Bug Discoveries Issue Discovery Area 5 10.6% 50 18 38.3% 12 25.5% 2 4.3% 2 4.3% 6 12.8% PortMgt Output Port Input Port CQ CentrlMgt CQMgt XBar LPE 40 Rulebase 30 Chip Sim Elem Sim 20 Gate Sim 10 0 04/09 04/16 04/23 04/30 05/07 05/14 05/21 05/28 06/04 06/11 06/18 06/25 07/02 07/09 07/16 07/23 07/30 08/06 08/13 08/20 1 2.1% Issues 1 2.1% Week 6 Weeks 20 /0 06 /0 22 /0 08 /0 25 /0 11 /0 7 /2 0 03 7 /2 0 03 6 /2 0 03 6 /2 0 03 5 /2 0 03 5 /2 0 03 4 /2 0 03 4 /2 0 03 3 /2 0 03 3 /2 0 03 3 /2 0 03 2 /2 0 03 2 /2 0 03 1 /2 0 03 1 /2 0 03 2 /2 0 02 2 /2 0 02 1 /2 0 02 1 /2 0 02 0 /2 0 02 0 /2 0 02 9 /2 0 02 9 /2 0 02 9 /2 0 02 8 /2 0 02 8 /2 0 02 integration 27 /0 13 /0 30 /0 16 /0 02 /0 16 /0 02 /0 19 /0 05 /0 22 /1 08 /1 24 /1 10 /1 27 /1 13 /1 29 /0 15 /0 01 /0 18 /0 04 /0 #Bugs + + ) ) 14 Data collected from IBM Ethernet Core Project 12 10 8 FV nonFV 6 discovered by FV 4 2 0 7 B # . & # . & B & C #' . & .# D & 8 # # . . E / / 0 0 F 5$ $ BG & . 9 ) ) ) ) A * " %$ A& H I H H A AJ I I AI A ; 6 J A ; %$ A A "F 6 1I H JIH J J I H J J 1 I 11 1 H 4 HI H J H 10 + + # # 1 1 2 2' '3 34 4 5 5 ( ( 6 627 27 2 '' 6 $ 1 - 97 $ ' G '' G$ & ' ( 1 # ' 8 -/ &) # $ . # '' ;$ 97 & & # . !9 0 ( ' 8( 1 0 1 : ; $ & . #) . ) 97/ 2 97/ / 8 # Ludden et al., IBM Journal of R&D 46(1), 2002 11 + + 8 ' 8 ' # # 1 1 ' ' " 9 $ 7 3 : 3 ;. . 3 $ . . M . & ' $ . '$ K D L D=" $8 & $ && & & L $ 6J '' & ) * – & 3 ;. . – – " # '' I6 $ & 3 $ ) =E ') # =E ; . &' ? $ K & $ $ #. ) A 12 & ; ; ) ) * * 0 0 < < & K & ) 9 $ A7' $ 9 $ & 9 $ #I A7 IIK N & 9 $ A7 & 9 $ .# $ A7 A7 & $ ) & & & & IBM Microelectronics, Haifa Design Center http://www/pslsugar.org/papers/ABV-in-IBM-Haifa.pdf ) 13 ) ) A& . $ & . & % = $ ; $' & & * 14 : : 9 $ 7 ) 4; : & = > < ) AE% % $ ( –% –" 6 ' &9 0 G ' 15 =, =, % > > 6 ' G ' " 3= 3( ' ) . ! B # &0 16 , , # # 5 # #' ' 8 ' 8 ' .' ' $ &# ' 3 . : 3 9 #. 7 # 8 . 6 & & $ & 3 3" A ' * M &# : ' . < $ > & # '' # <$ $ > 17 , , ' ' ( 0 9 $ 7 ( # 0 ). /E – ' : && & / ) # $' # L D=" 'M D D )))/ ' $8 & / &D '' D '$ OB# ! / ' ' .# I/97 & . . 97' ? & / ' .# #; A8 18 19 . . =? =? ) ) 1 1> > $ $ 97 Design understanding 25.6 Learning PSL 30.4 Using non-determinism 90.4 Modifying design to cope w/size 100 Writing constraints to cope w/size 65.6 Maintenance of constraints 68.8 Conceiving properties 61.6 Maintenance of properties 0 10 20 30 40 50 60 70 80 90 100 20 , , == # ' ' ; ; 6= ' ) $ $' & ' > > $* / B& # ' # * . # $ : $ () &0 ) ' 6 "$ )) # * &) ) : $ ' . $ : $ . &# % & ) 1J A * ") =F B $ " 8$ # 5 1JH 21 @ @' ' ! < & ' &' &> 22 # # - 9 $ 7 < ) : & = > ) 4; AE% % 3 3 3 5 "$ $ 8 &$ &# &# $$ $ 23 Complex chips, challenging verification problems? Welcome to the club of Formal Verification ? AB 24