Index: papers/FDL2012/exp_results.tex
===================================================================
--- papers/FDL2012/exp_results.tex	(revision 75)
+++ papers/FDL2012/exp_results.tex	(revision 80)
@@ -1,30 +1,32 @@
-We have conducted preliminary experiments to tests and compare the performance of our strategy with existing abstraction-refinement technique available in VIS. As our test platforms requires fairness constraints, we have chosen the \emph{incremental\_ctl\_verification} abstraction refinement technique as it supports CTL formulas and fairness constraints \cite{PardoHachtel98incremCTLMC} \cite{PardoHachtel97autoAbsMC}. It is an automatic abstraction refinement algorithm which generates an initial conservative abstraction principally by reducing the size of the latches by a constant factor. If the initial abstraction is not conclusive, a \emph{goal set} will then be computed in order to guide the refinement process.
+We have conducted preliminary experiments to tests and compare the performance of our strategy with existing abstraction-refinement technique available in VIS. There several abstraction-refinement techniques implemented in VIS accessible via \emph{approximate\_model\_check}, \emph{iterative\_model\_check}, \emph{check\_invariant} and \emph{incremental\_ctl\_model\_check} commands. However, among the techniques available, \emph{incremental\_ctl\_model\_check} is the only abstraction-refinement technique that supports CTL formulas and fairness constraints which are necessary in our test platforms. It is an automatic abstraction refinement algorithm which generates an initial conservative abstraction principally by reducing the size of the latches by a constant factor. If the initial abstraction is not conclusive, a \emph{goal set} will then be computed in order to guide the refinement process \cite{PardoHachtel98incremCTLMC} \cite{PardoHachtel97autoAbsMC}.
+
 
 We have executed and compared the execution time and the number of refinement iterations for two examples: VCI-PI platform consisting of Virtual Component Interface (VCI), a PI-Bus and VCI-PI protocol converter and a simplified version of a CAN bus platform consisting of 3 nodes on a CAN bus. Table \ref{StatsVCI_PI} and Table \ref{StatsCAN_Bus} give the size and the statistics concerning the VCI-PI platform and CAN bus platform respectively. All the values are obtained using the \emph{compute\_reach} command with option \emph{-v 1} in VIS except the number of BDD variables, computed using the \emph{print\_bdd\_stats} command. The experiments have been executed on a PC with an AMD Athlon dual-core processor 4450e and 1.8GB of RAM memory. 
 
+
 \begin{table*} [ht]
-\hspace*{10mm}
-\begin{tabular}{clccccc}
+\hspace*{17mm}
+\begin{tabular}{clcccc}
 
 \toprule 
-\multicolumn{2}{c}{\textbf{VCI-PI}}   & \emph{Number of}      & \emph{FSM}    & \emph{BDD}  & \emph{Number of }       & \emph{Analysis}\\ 
-\multicolumn{2}{c}{\textbf{Platform}} &\emph{BDD Variables}  & \emph{Depth}  & \emph{Size} & \emph{Reachable States} & \emph{Time (s)} \\
+\multicolumn{2}{c}{\textbf{VCI-PI}}   & \emph{Number of}      & \emph{BDD}  & \emph{Number of }       & \emph{Analysis}\\ 
+\multicolumn{2}{c}{\textbf{Platform}} &\emph{BDD Variables}   & \emph{Size} & \emph{Reachable States} & \emph{Time (s)} \\
 \midrule
 \midrule
-               & 1 Master-1 Slave   & 308 & 599 & 33 442   & 3.64116e+06 & 41.49 \\
- Concrete      & 2 Masters-1 Slave  & 439 & 186 & 75 210   & 7.3282e+04  & 14.36 \\
-  Model        & 4 Masters-1 Slave  & 709 & 408 & 156 657  & 2.15076e+07 & 414.38 \\
-               & 4 Masters-2 Slaves & 883 & 597 & 217 797  & 3.22215e+10 & 4064.09 \\
+               & 1 Master-1 Slave   & 308  & 33 442   & 3.64116e+06 & 41.49 \\
+ Concrete      & 2 Masters-1 Slave  & 453  & 140297   & 2.42518e+11 & 1922.75 \\
+  Model        & 4 Masters-1 Slave  & 737  & N/A      & N/A         & >3days \\
+               & 4 Masters-2 Slaves & 911  & N/A      & N/A         & >3days \\
  \midrule 
  \midrule
- Final         & 1 Master-1 Slave   & 197 & 2   &  76   &  5.03316e+07  & 0.01 \\
- Abstract      & 2 Masters-1 Slave  & 301 & 2   &  99   &  4.12317e+11  & 0.02 \\
- Model         & 4 Masters-1 Slave  & 501 & 2   & 147   &  3.45876e+18  & 0.03\\
- for $\phi_1$  & 4 Masters-2 Slaves & 589 & 2   & 167   &  7.08355e+21  & 0.04 \\
+ Final         & 1 Master-1 Slave   & 197  &  76   &  5.03316e+07  & 0.01 \\
+ Abstract      & 2 Masters-1 Slave  & 301  &  99   &  4.12317e+11  & 0.02 \\
+ Model         & 4 Masters-1 Slave  & 501  & 147   &  3.45876e+18  & 0.03\\
+ for $\phi_1$  & 4 Masters-2 Slaves & 589  & 167   &  7.08355e+21  & 0.04 \\
 \midrule 
- Final         & 1 Master-1 Slave   & 194 & 1   & 50    &  2.62144e+07  & 0 \\
- Abstract      & 2 Masters-1 Slave  & 298 & 1   & 73    &  2.14748e+11  & 0.01 \\
- Model         & 4 Masters-1 Slave  & 498 & 1   & 121   &  1.80144e+18  & 0.02 \\
- for $\phi_2$  & 4 Masters-2 Slaves & 586 & 1   & 141   &  3.68935e+21  & 0.02 \\ 
+ Final         & 1 Master-1 Slave   & 194  & 50    &  2.62144e+07  & 0 \\
+ Abstract      & 2 Masters-1 Slave  & 298  & 73    &  2.14748e+11  & 0.01 \\
+ Model         & 4 Masters-1 Slave  & 498  & 121   &  1.80144e+18  & 0.02 \\
+ for $\phi_2$  & 4 Masters-2 Slaves & 586  & 141   &  3.68935e+21  & 0.02 \\ 
 \bottomrule  
 \bottomrule
@@ -36,19 +38,20 @@
 
 
+
 \begin{table*} [ht]
-\hspace*{12mm}
-\begin{tabular}{cccccc}
+\hspace*{15mm}
+\begin{tabular}{ccccc}
 
 \toprule 
-\textbf{CAN Bus}      & \emph{Number of}      & \emph{FSM}    & \emph{BDD}  & \emph{Number of }       & \emph{Analysis}\\ 
-\textbf{Platform}     &\emph{BDD Variables}  & \emph{Depth}  & \emph{Size} & \emph{Reachable States} & \emph{Time (s)} \\
+\textbf{CAN Bus}      & \emph{Number of}       & \emph{BDD}  & \emph{Number of }       & \emph{Analysis}\\ 
+\textbf{Platform}     &\emph{BDD Variables}    & \emph{Size} & \emph{Reachable States} & \emph{Time (s)} \\
 \midrule
 \midrule
- Concrete Model   & 838     & 182     & 212550    & 2.87296e+08      & 167.9 \\      
+ Concrete Model                     & 822     & 140586      & 3.7354e+07     & 294.6 \\      
  \midrule 
  \midrule
- Final Abstract Model for $\phi_1$  & xx  & 2   & xxx   &  xxx       & xxxx \\
+ Final Abstract Model for $\phi_1$  & 425   & 187   &  1.66005e+12       & 0.03 \\
 \midrule 
- Final Abstract Model for $\phi_2$  & xx  & 2   & xxx   &  xxx       & xxxx \\
+ Final Abstract Model for $\phi_2$  & 425   & 187   &  1.66005e+12       & 0.04 \\
 \bottomrule  
 \bottomrule
@@ -57,4 +60,6 @@
 \caption{\label{StatsCAN_Bus} Statistics on the CAN Bus platform}
 \end{table*}
+
+
 
 
@@ -80,30 +85,30 @@
 \multicolumn{4}{l}{\textbf{\underline{2 Masters - 1 Slave :}}}   \\   
  	         & Prop. Selection   & 1    &    2.0     \\
- $\phi_1$   & Incremental       & 0    &    1.3        \\
-            & Standard MC       & -    &    1.5     \\
+ $\phi_1$   & Incremental       & 0    &  107.7        \\
+            & Standard MC       & -    & 1181.8     \\
 \midrule
  	         & Prop. Selection   & 0    &    1.0    \\
- $\phi_2$   & Incremental       & 0    &   15.3       \\
- 	         & Standard MC       & -    &   14.6    \\
+ $\phi_2$   & Incremental       & 0    &  108.5       \\
+ 	         & Standard MC       & -    & 1103.3   \\
 \midrule
 \midrule  
 \multicolumn{4}{l}{\textbf{\underline{4 Masters - 1 Slave :}}}   \\ 
  	         & Prop. Selection   & 1    &    2.1     \\
- $\phi_1$   & Incremental       & 0    &    6.8        \\
-            & Standard MC       & -    &  175.5     \\
+ $\phi_1$   & Incremental       & N/A  & >3 days        \\
+            & Standard MC       & -    & >3 days     \\
 \midrule
  	         & Prop. Selection   &  0    &   1.0 \\
- $\phi_2$   & Incremental       &  0    &   7.5    \\
-	         & Standard MC       &  -    & 195.1 \\
+ $\phi_2$   & Incremental       &  N/A  &  >3 days   \\
+	         & Standard MC       &  -    & >3 days \\
 \midrule 
 \midrule
 \multicolumn{4}{l}{\textbf{\underline{4 Masters - 2 Slaves :}}}  \\ 
 				& Prop. Selection   & 1   &  2.2     \\
- $\phi_1$   & Incremental       & N/A &  >1 day    \\
-            & Standard MC       & -   &  2231.3    \\
+ $\phi_1$   & Incremental       & N/A &  >3 days    \\
+            & Standard MC       & -   &  >3 days   \\
 \midrule
             & Prop. Selection   & 0   &  1.1\\
- $\phi_2$   & Incremental       & N/A &  >1 day    \\
-            & Standard MC       & -   &  12814.1 \\
+ $\phi_2$   & Incremental       & N/A &  >3 days    \\
+            & Standard MC       & -   &  >3 days\\
 \bottomrule       
 \bottomrule
@@ -126,11 +131,11 @@
 \midrule
 \midrule
- 	         & Prop. Selection  &  xxx        &  xxx     \\
- $\phi_3$   & Incremental      & N/A   		&  >1 day \\
- 	         & Standard MC      & -   			&  51.9    \\
+ 	         & Prop. Selection  &  0          &  1.02     \\
+ $\phi_3$   & Incremental      & N/A   		&  N/A \\
+ 	         & Standard MC      & -   			&  N/A    \\
 \midrule
- 	         & Prop. Selection   &  xxx       &  xxx \\
- $\phi_4$   & Incremental       & 0          &   57.3     \\
- 	         & Standard MC       & -          &    2.2    \\
+ 	         & Prop. Selection   &  0         &   1.01\\
+ $\phi_4$   & Incremental       & N/A        &   N/A     \\
+ 	         & Standard MC       & -          &   N/A    \\
 \bottomrule
 
@@ -142,5 +147,10 @@
 
 
-In the following tables: Table \ref{TabVCI_PI} and Table \ref{TabCANBus}, we compare the execution time between our technique (Prop. Selection), incremental\_ctl\_verification (Incremental) and the standard model checking (Standard MC) computed using the \emph{model\_check} command in VIS (Note: Dynamic variable ordering has been enabled with sift method). For the VCI-PI platform, the global property $\phi_1$ is the type $AF((p=1)*AF(q=1))$ and $\phi_2$ is actually a stronger version of the same formula with $AG(AF((p=1)*AF(q=1)))$ whereas for the CAN bus platform, the global property $\phi_3$ is the type $AG(((p'=1)*(q'=1)) -> AF((s = 1)*(r = 1))$ and $\phi_4 = AG(((p'=1)*(q'=1)) -> AG(r' = 0))$. We can see that our technique systematically computes faster than the other two methods and interestingly in the case where the size of the platform increases by adding the more connected components, in contrary to the other two methods, our computation time remains stable.
+In the following tables: Table \ref{TabVCI_PI} and Table \ref{TabCANBus}, we compare the execution time between our technique (Prop. Selection), incremental\_ctl\_verification (Incremental) and the standard model checking (Standard MC) computed using the \emph{model\_check} command in VIS (Note: Dynamic variable ordering has been enabled with sift method). For the VCI-PI platform, the global property $\phi_1$ is the type $AF((p=1)*AF(q=1))$ and $\phi_2$ is actually a stronger version of the same formula with $AG(AF((p=1)*AF(q=1)))$. We have a total of 42 verifed components properties to be selected in VCI-PI plateform and for the verification of $\phi_1$ we have restrained the selectable properties only to those without AG prefix. In comparison to $\phi_2$, we can see that, a better set of properties available will result in a better abstraction and less refinement iterations.  
+
+
+In the case of the CAN bus platform, the global property $\phi_3$ is the type $AG(((p'=1)*(q'=1)*AF(r_1=1)) -> AF((s_1=1)*AF(t_1=1)))$ and $\phi_4 = AG(((p'=1)*(q'=1)*AG(r_2=0)) -> AG((s_2=0)*(t_2=0)))$. We have at our disposal 103 verified component properties and after the selection process, 3 selected component properties were sufficient to verify both global properties.
+
+Globally, we can see that our technique systematically computes faster than the other two methods and interestingly in the case where the size of the platform increases by adding the more connected components, in contrary to the other two methods, our computation time remains stable.
 
 
