Difference between revisions of "D23 ProB"

From Event-B
Jump to navigationJump to search
imported>Leuschel
imported>Pascal
 
(29 intermediate revisions by 6 users not shown)
Line 1: Line 1:
 
= Overview =
 
= Overview =
This part of the deliverable describes improvement of the ProB animation and model checking plugin.
+
This part of the deliverable describes improvements of the ProB animation and model checking plug-in.
  
 
The improvements and development of ProB were mainly carried out by University of Düsseldorf, with some support by the University of Southampton.
 
The improvements and development of ProB were mainly carried out by University of Düsseldorf, with some support by the University of Southampton.
Line 6: Line 6:
  
 
New features:
 
New features:
* Multi-level animation and validation
+
* Multi-level animation and validation.
* B-Motion Studio
+
* B-Motion Studio.
* first steps towards test-case generation
+
* Disprover Support.
 +
* First steps towards test-case generation.
  
 
Improvements:
 
Improvements:
* Scalability improvements driven by Siemens and SAP applications
+
* Scalability improvements driven by Siemens and SAP applications.
* Using proof information to improve model checking
+
* Using proof information to improve model checking.
  
 
Other works:
 
Other works:
* First steps towards validation of ProB for usage by Siemens in SIL-4 chain
+
* First steps towards validation of ProB for usage by Siemens in SIL-4 chain.
* Evaluation against SAT/SMT/BDD-based approaches
+
* Evaluation against SAT/SMT/BDD-based approaches.
  
 
= Motivations =
 
= Motivations =
This paragraph shall express the motivation for each tool extension and improvement. More precisely, it shall first indicate the state before the work, the encountered difficulties, and shall highlight the requirements (eg. those of industrial partners). Then, it shall summarize how these requirements are addressed and what are the main benefits.
 
  
 +
== Multi-level Animation and Validation ==
  
=== Multi-level Animation and Validation ===
+
Thus far ProB only allowed single-level animation, i.e. the animator would animate a single refinement level in isolation.
 
 
Thus far ProB only alllowed single-level animation, i.e., the animator would animate a single refinement level in isolation.
 
 
This meant that ProB was not able to detect a large class of potential errors in the model:
 
This meant that ProB was not able to detect a large class of potential errors in the model:
* a broken gluing invariant
+
* A broken gluing invariant.
* an invalid witness
+
* An invalid witness.
* violation of guard strengthening
+
* Violation of guard strengthening.
* violation of variant decrease (resp. decrease or stability) for convergent (resp. anticipated) events
+
* Violation of variant decrease (resp. decrease or stability) for convergent (resp. anticipated) events.
  
 
The new validation algorithm now can animate a range of refinements together. The user can decide which levels are to be animated together.
 
The new validation algorithm now can animate a range of refinements together. The user can decide which levels are to be animated together.
 
As such, all of the above errors can now be detected by ProB.
 
As such, all of the above errors can now be detected by ProB.
 
User experience is also improved, as he or she can inspect also the abstract variables.
 
User experience is also improved, as he or she can inspect also the abstract variables.
The new algorithm has been sucessfully applied to various case studies, and thus far up to 14 levels have been animated concurrently without problem.
+
The new algorithm has been successfully applied to various case studies, and thus far up to 14 levels have been animated concurrently without problem.
  
 
+
== B-Motion Studio ==
=== B-Motion Studio ===
 
  
 
It is often very important to be able to show a formal model to a domain expert or manager, not versed in formal methods.
 
It is often very important to be able to show a formal model to a domain expert or manager, not versed in formal methods.
Line 46: Line 44:
 
To run a graphical visualisation, the ProB animator is used.
 
To run a graphical visualisation, the ProB animator is used.
  
=== Test-Case Generation ===
+
== Test-Case Generation ==
  
During deployment in the SAP workpackage it became clear that test-case generation from the Event-B models
+
During deployment in the SAP workpackage it became clear that test-case generation from the Event-B models is required for success.
is required for success.
 
 
In this task, we have developed a first algorithm for test-case generation, which ensures complete transition coverage of a high-level model, and translates the test-cases into traces of a refined model, so that the tests can be run on the "real" system.
 
In this task, we have developed a first algorithm for test-case generation, which ensures complete transition coverage of a high-level model, and translates the test-cases into traces of a refined model, so that the tests can be run on the "real" system.
 
Optimisations, to reduce the length and number of test cases, as well as to minimise race conditions, have been implemented.
 
Optimisations, to reduce the length and number of test cases, as well as to minimise race conditions, have been implemented.
  
=== Scalability and Validation ===
+
== Scalability, Application by Siemens and Validation ==
We tackled a case study in WP2, which centres on the San Juan metro system installed by Siemens. The control software was developed and formally proven with B. However, the development contains certain assumptions about the actual rail network topology which have to be validated separately in order to ensure safe operation. For this task, Siemens has developed custom proof rules for AtelierB. AtelierB, however, was unable to deal with about 80 properties of the deployment (running out of memory). These properties thus had to be validated by hand at great expense (and they need to be revalidated whenever the rail network infrastructure changes). I
+
We tackled a case study in WP2, which centres on the San Juan metro system installed by Siemens. The control software was developed and formally proven with B. However, the development contains certain assumptions about the actual rail network topology which have to be validated separately in order to ensure safe operation. For this task, Siemens has developed custom proof rules for AtelierB. AtelierB, however, was unable to deal with about 80 properties of the deployment (running out of memory). These properties thus had to be validated by hand at great expense (and they need to be revalidated whenever the rail network infrastructure changes).
  
 
The motivation then was to try and use ProB for this task.
 
The motivation then was to try and use ProB for this task.
Line 60: Line 57:
 
The ProB parser and type checker also had to be re-developed to be able to deal with large industrial specifications.
 
The ProB parser and type checker also had to be re-developed to be able to deal with large industrial specifications.
  
The case study was a succcess:ProB was able to validate all of the about 300 properties of the San Juan deployment, detecting exactly the same faults automatically in around 17 minutes that were manually uncovered in about one man-month.  
+
The case study was a success: ProB was able to validate all of the about 300 properties of the San Juan deployment, detecting exactly the same faults automatically in around 17 minutes that were manually uncovered in about one man-month.
 +
 
 +
This leads to the next task: the issue of validating ProB, so that it can be integrated into the SIL4 development chain at Siemens.
  
This leads to the next task: the issue of validating ProB, so that it can be integrated into the SIL4 development chain at Siemens
+
== Proof Directed Model Checking ==
  
=== Proof Directed Model Checking ===
+
In order to improve the performance and scalability of animation and model checking with ProB, we want to make use of the proof information available in an integrated platform such as Rodin.
 +
In particular, we can use the information about which POs have already been discharged to simplify the task of the model checker and to guide the order in which it evaluates states.
  
Jens: can you insert something?
+
Our first implementation has shown significant reduction of the model checking effort for industrial applications. It uses proof information obtained from Rodin to remove invariants that are proven to hold. Actually, if unproven(e) is the unproven part of the invariant for event e, and a state is reachable via events e and f, we only need to check the intersection of unproven(e) and unproven(f) (see [http://www.stups.uni-duesseldorf.de/~leuschel/publication_detail.php?id=253 Jens Bendisposto, Michael Leuschel. Proof Assisted Model Checking for B. ICFEM 2009.]).
  
 +
== SAT/SMT/Kodkod ==
  
=== SAT/SMT/Kodkod ===
+
In this subtask we investigate alternate approaches to validate high-level B models using techniques and tools based on BDDs, SAT-solving and SMT-solving.
 +
The overall motivation is to improve the scalability of the animator and model checker.
  
Daniel: can you insert something
+
When searching for variable values that satisfy a certain predicate, experience showed that ProB performs best when some values are already known, as in finding state transitions where all values of the previous state are known.
 +
When looking for constants that satisfy the axioms or when trying to find counterexamples for proof obligations (see next Disprover paragraph), SAT resp. SMT solving techniques look very promising to support ProB's constraint solving approach.
 +
 
 +
== Disprover ==
 +
 
 +
In order to help the user, we wanted to make it possible to apply ProB to individual proof obligations.
 +
In some cases, this enables proving a sequent by exhaustive case analysis.
 +
Also, if ProB finds a counterexample, the user gets important feedback: the proof obligation cannot be discharged, along with a reason why.
  
 
= Choices / Decisions =
 
= Choices / Decisions =
This paragraph shall summarize the decisions (eg. design decisions) and justify them. Thus, it may present the studied solutions, through their main advantages and inconvenients, to legitimate the final choices.
+
 
 +
One important choice for ProB is to use the same constraint solving kernel and interpreter for B, Event-B and Z. This decision, allows us to maintain a tool capable of animating and model checking these three formalisms. It also gives us a much wider range of test cases. For example, most of the regression tests of ProB come in the form of B machines (Rodin archives are much "cumbersome" as a basis of regression tests: they need to be upgraded and imported into workspaces).
 +
This choice also enabled us to achieve the successful deployment of ProB in WP2, where the use of classical B was mandatory.
 +
 
 +
Concerning the inclusion of SMT, SAT, BDD techniques into ProB, no decision has been taken yet. We are still investigating the possibilities.
 +
 
 +
Concerning B-Motion Studio, we decided not to use Flash (we had an earlier prototype using Flash). We wanted a tool that can be easily used and installed into Rodin. We also wanted a tool that can be used without having to learn a new programming language (Action Script).
  
 
= Available Documentation =
 
= Available Documentation =
Line 83: Line 98:
  
 
== Published Papers ==
 
== Published Papers ==
 +
 +
Below is a list of published papers, along with an abstract.
 +
 
=== Improved Kernel to deal with large sets and relations ===
 
=== Improved Kernel to deal with large sets and relations ===
 
In this part we describe the successful application of the ProB validation tool on an industrial case study. The case study centres on the San Juan metro system installed by Siemens. The control software was developed and formally proven with B. However, the development contains certain assumptions about the actual rail network topology which have to be validated separately in order to ensure safe operation. For this task, Siemens has developed custom proof rules for AtelierB. AtelierB, however, was unable to deal with about 80 properties of the deployment (running out of memory). These properties thus had to be validated by hand at great expense (and they need to be revalidated whenever the rail network infrastructure changes). In this paper we show how we were able to use ProB to validate all of the about 300 properties of the San Juan deployment, detecting exactly the same faults automatically in around 17 minutes that were manually uncovered in about one man-month. This achievement required the extension of the ProB kernel for large sets as well as an improved constraint propagation phase. We also outline some of the effort and features that were required in moving from a tool capable of dealing with medium-sized examples towards a tool able to deal with actual industrial specifications. Notably, a new parser and type checker had to be developed. We also touch upon the issue of validating ProB, so that it can be integrated into the SIL4 development chain at Siemens
 
In this part we describe the successful application of the ProB validation tool on an industrial case study. The case study centres on the San Juan metro system installed by Siemens. The control software was developed and formally proven with B. However, the development contains certain assumptions about the actual rail network topology which have to be validated separately in order to ensure safe operation. For this task, Siemens has developed custom proof rules for AtelierB. AtelierB, however, was unable to deal with about 80 properties of the deployment (running out of memory). These properties thus had to be validated by hand at great expense (and they need to be revalidated whenever the rail network infrastructure changes). In this paper we show how we were able to use ProB to validate all of the about 300 properties of the San Juan deployment, detecting exactly the same faults automatically in around 17 minutes that were manually uncovered in about one man-month. This achievement required the extension of the ProB kernel for large sets as well as an improved constraint propagation phase. We also outline some of the effort and features that were required in moving from a tool capable of dealing with medium-sized examples towards a tool able to deal with actual industrial specifications. Notably, a new parser and type checker had to be developed. We also touch upon the issue of validating ProB, so that it can be integrated into the SIL4 development chain at Siemens
 
  
 
[http://www.stups.uni-duesseldorf.de/~leuschel/publication_detail.php?id=248 Michael Leuschel, Jérôme Falampin, Fabian Fritz, Daniel Plagge. Automated Property Verification for Large Scale B Models, FM'2009.]
 
[http://www.stups.uni-duesseldorf.de/~leuschel/publication_detail.php?id=248 Michael Leuschel, Jérôme Falampin, Fabian Fritz, Daniel Plagge. Automated Property Verification for Large Scale B Models, FM'2009.]
Line 96: Line 113:
 
[http://www.stups.uni-duesseldorf.de/~leuschel/publication_detail.php?id=256 Stefan Hallerstede, Michael Leuschel, Daniel Plagge. Refinement-Animation for Event-B --- Towards a Method of Validation. ABZ'2010 ]
 
[http://www.stups.uni-duesseldorf.de/~leuschel/publication_detail.php?id=256 Stefan Hallerstede, Michael Leuschel, Daniel Plagge. Refinement-Animation for Event-B --- Towards a Method of Validation. ABZ'2010 ]
  
=== Test-Case Generation ===
+
See also [http://www.springerlink.com/content/282p2316x7165588/ Stefan Hallerstede, Michael Leuschel. How to Explain Mistakes. TFM'09.]
Choreography models describe the communication protocols between services. Testing of service choreographies is an important task for the quality assurance of service-based systems as used e.g. in the context of service-oriented architectures (SOA). The formal modeling of service choreographies enables a model-based integration testing (MBIT) approach. We present MBIT methods for our service choreography modeling approach called Message Choreography Models (MCM). For the model-based testing of service choreographies, MCMs are translated into Event-B models and used as input for our test generator which uses the model checker ProB.
+
 
 +
=== Test Case Generation ===
 +
Choreography models describe the communication protocols between services. Testing of service choreographies is an important task for the quality assurance of service-based systems as used e.g. in the context of service-oriented architectures (SOA). The formal modelling of service choreographies enables a model-based integration testing (MBIT) approach. We present MBIT methods for our service choreography modeling approach called Message Choreography Models (MCM). For the model-based testing of service choreographies, MCMs are translated into Event-B models and used as input for our test generator which uses the model checker ProB.
  
 
[http://www.stups.uni-duesseldorf.de/~leuschel/publication_detail.php?id=252 Sebastian Wieczorek, Vitaly Kozyura, Andreas Roth, Michael Leuschel, Jens Bendisposto, Daniel Plagge, Ina Schieferdecker. Applying Model Checking to Generate Model-based Integration Tests from Choreography Models. TESTCOM/FATES 2009.]
 
[http://www.stups.uni-duesseldorf.de/~leuschel/publication_detail.php?id=252 Sebastian Wieczorek, Vitaly Kozyura, Andreas Roth, Michael Leuschel, Jens Bendisposto, Daniel Plagge, Ina Schieferdecker. Applying Model Checking to Generate Model-based Integration Tests from Choreography Models. TESTCOM/FATES 2009.]
 
  
 
=== Proof-Directed Model Checking ===
 
=== Proof-Directed Model Checking ===
With the aid of the ProB Plugin, the Rodin Platform provides an integrated environment for editing, proving, animating and model checking Event-B models. This is of considerable benefit to the modeler, as it allows him to switch between the various tools to validate, debug and improve his or her models. The crucial idea of this paper is that the integrated platform also provides benefits to the tool developer, i.e., it allows easy access to information from other tools. Indeed, there has been considerable interest in combining model checking, proving and testing. In previous work we have already shown how a model checker can be used to complement the Event-B proving environment, by acting as a disprover. In this paper we show how the prover can help improve the efficiency of the animator and model checker.
+
With the aid of the ProB plug-in, the Rodin Platform provides an integrated environment for editing, proving, animating and model checking Event-B models. This is of considerable benefit to the modeller, as it allows him to switch between the various tools to validate, debug and improve his or her models. The crucial idea of this paper is that the integrated platform also provides benefits to the tool developer, i.e. it allows easy access to information from other tools. Indeed, there has been considerable interest in combining model checking, proving and testing. In previous work we have already shown how a model checker can be used to complement the Event-B proving environment, by acting as a disprover. In this paper we show how the prover can help to improve the efficiency of the animator and model checker.
  
 
[http://www.stups.uni-duesseldorf.de/~leuschel/publication_detail.php?id=253 Jens Bendisposto, Michael Leuschel. Proof Assisted Model Checking for B. ICFEM 2009.]
 
[http://www.stups.uni-duesseldorf.de/~leuschel/publication_detail.php?id=253 Jens Bendisposto, Michael Leuschel. Proof Assisted Model Checking for B. ICFEM 2009.]
 +
 +
=== Debugging Tricky Proof Obligations with the ProB Disprover ===
 +
While a large number of proof obligations can be discharged automatically by tools such as the Rodin platform, a considerable number still have to be proven interactively.  In this paper, we describe a disprover plug-in for Rodin that utilizes ProB to automatically find counterexamples for a given problematic proof obligation.  In case the disprover finds a counterexample, the user can directly investigate the source of the problem, as pinpointed by the counterexample. We also discuss under which circumstances our plug-in can be used as a prover, i.e. when the absence of a counterexample actually is a proof of the proof obligation.
 +
 +
[http://www.stups.uni-duesseldorf.de/publications_detail.php?id=219 Olivier Ligot, Jens Bendisposto, Michael Leuschel. Debugging Event-B Models using the ProB Disprover Plug-in.  AFADL 2007.]
  
 
=== Inspection of Alternate Approaches ===
 
=== Inspection of Alternate Approaches ===
  
ProB is a model checker for high-level B and Event-B models based on constraint-solving. In this paper we investigate alternate approaches for validating high-level B models using alternative techniques and tools based on using BDDs, SAT-solving and SMT-solving. In particular, we examine whether ProB can be complemented or even supplanted by using one of the tools BDDBDDB, Kodkod or SAL.
+
ProB is a model checker for high-level B and Event-B models based on constraint-solving. In this paper we investigate alternate approaches for validating high-level B models using techniques and tools based on using BDDs, SAT-solving and SMT-solving. In particular, we examine whether ProB can be complemented or even supplanted by using one of the tools BDDBDDB, Kodkod or SAL.
  
 
[http://www.stups.uni-duesseldorf.de/~leuschel/publication_detail.php?id=249 Daniel Plagge, Michael Leuschel, Ilya Lopatkin, Alexander Romanovsk. SAL, Kodkod, and BDDs for Validation of B Models. Lessons and Outlook. AFM'09.]
 
[http://www.stups.uni-duesseldorf.de/~leuschel/publication_detail.php?id=249 Daniel Plagge, Michael Leuschel, Ilya Lopatkin, Alexander Romanovsk. SAL, Kodkod, and BDDs for Validation of B Models. Lessons and Outlook. AFM'09.]
Line 115: Line 138:
 
=== Validation of ProB ===
 
=== Validation of ProB ===
  
Symmetry reduction is a model checking technique that can help alleviate the problem of state space explosion, by preventing redundant state space exploration. In previous work, we have developed three effective approaches to symmetry reduction for B that have been implemented into the ProB model checker, and we have proved the soundness of our state symmetries. However, it is also important to show our techniques are sound with respect to standard model checking, at the algorithmic level. In this paper, we present a retrospective B development that addresses this issue through a series of B refinements. This work also demonstrates the valuable insights into a system that can be gained through formal modelling.
+
Symmetry reduction is a model checking technique that can help to alleviate the problem of state space explosion, by preventing redundant state space exploration. In previous work, we have developed three effective approaches to symmetry reduction for B that have been implemented into the ProB model checker, and we have proved the soundness of our state symmetries. However, it is also important to show that our techniques are sound with respect to standard model checking, at the algorithmic level. In this paper, we present a retrospective B development that addresses this issue through a series of B refinements. This work also demonstrates the valuable insights into a system that can be gained through formal modelling.
  
 
[http://www.stups.uni-duesseldorf.de/~leuschel/publication_detail.php?id=257 Edd Turner, Michael Butler, Michael Leuschel. A Refinement-Based Correctness Proof of Symmetry Reduced Model Checking. ABZ'2010.]
 
[http://www.stups.uni-duesseldorf.de/~leuschel/publication_detail.php?id=257 Edd Turner, Michael Butler, Michael Leuschel. A Refinement-Based Correctness Proof of Symmetry Reduced Model Checking. ABZ'2010.]
 
  
 
=== B-Motion Studio ===
 
=== B-Motion Studio ===
Line 127: Line 149:
  
 
= Planning =
 
= Planning =
This paragraph shall give a timeline and current status (as of 29 Jan 2010).
+
In future, it is planned to work on the following topics:
  
=== Model-based Testing ===
+
== Model-based Testing ==
  
* Directed Model Checking to achieve coverage (Deploy extension; Flow Graphs)
+
* Directed model checking to achieve coverage (DEPLOY extension; flow graphs).
* Integrate Algorithm into Rodin
+
* Integrate algorithm into Rodin.
* Make Algorithm more generic
+
* Make algorithm more generic.
* Top-down multi-level animation
+
* Top-down multi-level animation.
* Move from prototype to real product
+
* Move from prototype to real product.
  
=== B-Motion Studio ===
+
== B-Motion Studio ==
 +
 
 +
* Experiment with existing Flash animation and B model of ClearSy.
 +
* Improve usability, more widgets.
 +
 
 +
== Validation of ProB ==
  
* Experiment with existing Flash animation and B model of ClearSy
+
* Test coverage analysis for Prolog code.
* Improve usability, more widgets
+
* Validation document to be delivered to Siemens.
  
=== Validation of ProB ===
+
== Scalability ==
  
* Test coverage analysis for Prolog code
+
* More experiments with SAT, SMT, BDD techniques.
* Validation Document to be delivered to Siemens
 
  
=== Scalability ===
+
* Integration of Kodkod into ProB to solve complicated predicates over first order relations and simple sets.
  
* More experiments with SAT, SMT, BDD techniques
+
* Adaption of ProB for the upcoming mathematical extensions. Indeed, for the moment the Rodin user is often required to model basic datatypes (records, sequences,...) or operators (transitive closure) herself. This is a big challenge to the animator, which does not know that the constants and variables of the machine (e.g. injective functions) are "simply" meant to model quite basic datatypes. With the introduction of mathematical extensions for records, transitive closure, ... this hurdle will be overcome.
  
* Integration of Kodkod into ProB for solving complicated predicates over first order relations and simple sets
+
== Usability ==
 +
* Feedback errors found by ProB into the PO view (as red icons).
 +
* Improve disprover, detect when it is a decision procedure.
 +
* Allow the user to easier inspect elements of the animated model: the cause of errors, why events are not enabled, etc.
 +
* Further improvements to the GUI: 2-D Viewer, better multi-level animation view.
  
=== Usability ===
+
[[Category:D23 Deliverable]]
* Feedback errors found by ProB into the PO view (as red icons)
 
* Improve Disprover, detect when it is a decision procedure
 
* Further improvments to GUI: 2-D Viewer, better multi-level animation view
 

Latest revision as of 15:55, 28 January 2010

Overview

This part of the deliverable describes improvements of the ProB animation and model checking plug-in.

The improvements and development of ProB were mainly carried out by University of Düsseldorf, with some support by the University of Southampton. Furthermore, the work was driven by requirements of Siemens and SAP; some tool development was also undertaken by SAP.

New features:

  • Multi-level animation and validation.
  • B-Motion Studio.
  • Disprover Support.
  • First steps towards test-case generation.

Improvements:

  • Scalability improvements driven by Siemens and SAP applications.
  • Using proof information to improve model checking.

Other works:

  • First steps towards validation of ProB for usage by Siemens in SIL-4 chain.
  • Evaluation against SAT/SMT/BDD-based approaches.

Motivations

Multi-level Animation and Validation

Thus far ProB only allowed single-level animation, i.e. the animator would animate a single refinement level in isolation. This meant that ProB was not able to detect a large class of potential errors in the model:

  • A broken gluing invariant.
  • An invalid witness.
  • Violation of guard strengthening.
  • Violation of variant decrease (resp. decrease or stability) for convergent (resp. anticipated) events.

The new validation algorithm now can animate a range of refinements together. The user can decide which levels are to be animated together. As such, all of the above errors can now be detected by ProB. User experience is also improved, as he or she can inspect also the abstract variables. The new algorithm has been successfully applied to various case studies, and thus far up to 14 levels have been animated concurrently without problem.

B-Motion Studio

It is often very important to be able to show a formal model to a domain expert or manager, not versed in formal methods. For example, only a domain expert will be able to detect certain mistakes in the formal model. To enable to easily and quickly build graphical visualisations of Rodin models, we have developed B-Motion Studio. B-Motion Studio comes with a graphical editor to arrange graphical components and link them with the formal model. No new programming language has to be learned: the linking is described in B itself. To run a graphical visualisation, the ProB animator is used.

Test-Case Generation

During deployment in the SAP workpackage it became clear that test-case generation from the Event-B models is required for success. In this task, we have developed a first algorithm for test-case generation, which ensures complete transition coverage of a high-level model, and translates the test-cases into traces of a refined model, so that the tests can be run on the "real" system. Optimisations, to reduce the length and number of test cases, as well as to minimise race conditions, have been implemented.

Scalability, Application by Siemens and Validation

We tackled a case study in WP2, which centres on the San Juan metro system installed by Siemens. The control software was developed and formally proven with B. However, the development contains certain assumptions about the actual rail network topology which have to be validated separately in order to ensure safe operation. For this task, Siemens has developed custom proof rules for AtelierB. AtelierB, however, was unable to deal with about 80 properties of the deployment (running out of memory). These properties thus had to be validated by hand at great expense (and they need to be revalidated whenever the rail network infrastructure changes).

The motivation then was to try and use ProB for this task. This required a considerable amount of work on improving the scalability of the ProB kernel, to be able to deal with large sets and relations. The ProB parser and type checker also had to be re-developed to be able to deal with large industrial specifications.

The case study was a success: ProB was able to validate all of the about 300 properties of the San Juan deployment, detecting exactly the same faults automatically in around 17 minutes that were manually uncovered in about one man-month.

This leads to the next task: the issue of validating ProB, so that it can be integrated into the SIL4 development chain at Siemens.

Proof Directed Model Checking

In order to improve the performance and scalability of animation and model checking with ProB, we want to make use of the proof information available in an integrated platform such as Rodin. In particular, we can use the information about which POs have already been discharged to simplify the task of the model checker and to guide the order in which it evaluates states.

Our first implementation has shown significant reduction of the model checking effort for industrial applications. It uses proof information obtained from Rodin to remove invariants that are proven to hold. Actually, if unproven(e) is the unproven part of the invariant for event e, and a state is reachable via events e and f, we only need to check the intersection of unproven(e) and unproven(f) (see Jens Bendisposto, Michael Leuschel. Proof Assisted Model Checking for B. ICFEM 2009.).

SAT/SMT/Kodkod

In this subtask we investigate alternate approaches to validate high-level B models using techniques and tools based on BDDs, SAT-solving and SMT-solving. The overall motivation is to improve the scalability of the animator and model checker.

When searching for variable values that satisfy a certain predicate, experience showed that ProB performs best when some values are already known, as in finding state transitions where all values of the previous state are known. When looking for constants that satisfy the axioms or when trying to find counterexamples for proof obligations (see next Disprover paragraph), SAT resp. SMT solving techniques look very promising to support ProB's constraint solving approach.

Disprover

In order to help the user, we wanted to make it possible to apply ProB to individual proof obligations. In some cases, this enables proving a sequent by exhaustive case analysis. Also, if ProB finds a counterexample, the user gets important feedback: the proof obligation cannot be discharged, along with a reason why.

Choices / Decisions

One important choice for ProB is to use the same constraint solving kernel and interpreter for B, Event-B and Z. This decision, allows us to maintain a tool capable of animating and model checking these three formalisms. It also gives us a much wider range of test cases. For example, most of the regression tests of ProB come in the form of B machines (Rodin archives are much "cumbersome" as a basis of regression tests: they need to be upgraded and imported into workspaces). This choice also enabled us to achieve the successful deployment of ProB in WP2, where the use of classical B was mandatory.

Concerning the inclusion of SMT, SAT, BDD techniques into ProB, no decision has been taken yet. We are still investigating the possibilities.

Concerning B-Motion Studio, we decided not to use Flash (we had an earlier prototype using Flash). We wanted a tool that can be easily used and installed into Rodin. We also wanted a tool that can be used without having to learn a new programming language (Action Script).

Available Documentation

User Manual

ProB User Manual

Published Papers

Below is a list of published papers, along with an abstract.

Improved Kernel to deal with large sets and relations

In this part we describe the successful application of the ProB validation tool on an industrial case study. The case study centres on the San Juan metro system installed by Siemens. The control software was developed and formally proven with B. However, the development contains certain assumptions about the actual rail network topology which have to be validated separately in order to ensure safe operation. For this task, Siemens has developed custom proof rules for AtelierB. AtelierB, however, was unable to deal with about 80 properties of the deployment (running out of memory). These properties thus had to be validated by hand at great expense (and they need to be revalidated whenever the rail network infrastructure changes). In this paper we show how we were able to use ProB to validate all of the about 300 properties of the San Juan deployment, detecting exactly the same faults automatically in around 17 minutes that were manually uncovered in about one man-month. This achievement required the extension of the ProB kernel for large sets as well as an improved constraint propagation phase. We also outline some of the effort and features that were required in moving from a tool capable of dealing with medium-sized examples towards a tool able to deal with actual industrial specifications. Notably, a new parser and type checker had to be developed. We also touch upon the issue of validating ProB, so that it can be integrated into the SIL4 development chain at Siemens

Michael Leuschel, Jérôme Falampin, Fabian Fritz, Daniel Plagge. Automated Property Verification for Large Scale B Models, FM'2009.

Multi-Level Animation and Validation

We provide a detailed description of refinement in Event-B, both as a contribution in itself and as a foundation for the approach to simultaneous animation of multiple levels of refinement that we propose. We present an algorithm for simultaneous multi-level animation of refinement, and show how it can be used to detect a variety of errors that occur frequently when using refinement. The algorithm has been implemented in ProB and we applied it to several case studies, showing that multi-level animation is tractable also on larger models.

Stefan Hallerstede, Michael Leuschel, Daniel Plagge. Refinement-Animation for Event-B --- Towards a Method of Validation. ABZ'2010

See also Stefan Hallerstede, Michael Leuschel. How to Explain Mistakes. TFM'09.

Test Case Generation

Choreography models describe the communication protocols between services. Testing of service choreographies is an important task for the quality assurance of service-based systems as used e.g. in the context of service-oriented architectures (SOA). The formal modelling of service choreographies enables a model-based integration testing (MBIT) approach. We present MBIT methods for our service choreography modeling approach called Message Choreography Models (MCM). For the model-based testing of service choreographies, MCMs are translated into Event-B models and used as input for our test generator which uses the model checker ProB.

Sebastian Wieczorek, Vitaly Kozyura, Andreas Roth, Michael Leuschel, Jens Bendisposto, Daniel Plagge, Ina Schieferdecker. Applying Model Checking to Generate Model-based Integration Tests from Choreography Models. TESTCOM/FATES 2009.

Proof-Directed Model Checking

With the aid of the ProB plug-in, the Rodin Platform provides an integrated environment for editing, proving, animating and model checking Event-B models. This is of considerable benefit to the modeller, as it allows him to switch between the various tools to validate, debug and improve his or her models. The crucial idea of this paper is that the integrated platform also provides benefits to the tool developer, i.e. it allows easy access to information from other tools. Indeed, there has been considerable interest in combining model checking, proving and testing. In previous work we have already shown how a model checker can be used to complement the Event-B proving environment, by acting as a disprover. In this paper we show how the prover can help to improve the efficiency of the animator and model checker.

Jens Bendisposto, Michael Leuschel. Proof Assisted Model Checking for B. ICFEM 2009.

Debugging Tricky Proof Obligations with the ProB Disprover

While a large number of proof obligations can be discharged automatically by tools such as the Rodin platform, a considerable number still have to be proven interactively. In this paper, we describe a disprover plug-in for Rodin that utilizes ProB to automatically find counterexamples for a given problematic proof obligation. In case the disprover finds a counterexample, the user can directly investigate the source of the problem, as pinpointed by the counterexample. We also discuss under which circumstances our plug-in can be used as a prover, i.e. when the absence of a counterexample actually is a proof of the proof obligation.

Olivier Ligot, Jens Bendisposto, Michael Leuschel. Debugging Event-B Models using the ProB Disprover Plug-in. AFADL 2007.

Inspection of Alternate Approaches

ProB is a model checker for high-level B and Event-B models based on constraint-solving. In this paper we investigate alternate approaches for validating high-level B models using techniques and tools based on using BDDs, SAT-solving and SMT-solving. In particular, we examine whether ProB can be complemented or even supplanted by using one of the tools BDDBDDB, Kodkod or SAL.

Daniel Plagge, Michael Leuschel, Ilya Lopatkin, Alexander Romanovsk. SAL, Kodkod, and BDDs for Validation of B Models. Lessons and Outlook. AFM'09.

Validation of ProB

Symmetry reduction is a model checking technique that can help to alleviate the problem of state space explosion, by preventing redundant state space exploration. In previous work, we have developed three effective approaches to symmetry reduction for B that have been implemented into the ProB model checker, and we have proved the soundness of our state symmetries. However, it is also important to show that our techniques are sound with respect to standard model checking, at the algorithmic level. In this paper, we present a retrospective B development that addresses this issue through a series of B refinements. This work also demonstrates the valuable insights into a system that can be gained through formal modelling.

Edd Turner, Michael Butler, Michael Leuschel. A Refinement-Based Correctness Proof of Symmetry Reduced Model Checking. ABZ'2010.

B-Motion Studio

B-Motion Studio provides a way to quickly generate domain specific visualisations for a formal model, enabling domain experts and managers to understand and validate the model. We also believe that our tool will be of use when teaching formal methods, both during lectures as a way to motivate students to write their own formal models.

Lukas Ladenberger, Jens Bendisposto, Michael Leuschel. Visualising Event-B models with B-Motion Studio. FMICS'2009.

Planning

In future, it is planned to work on the following topics:

Model-based Testing

  • Directed model checking to achieve coverage (DEPLOY extension; flow graphs).
  • Integrate algorithm into Rodin.
  • Make algorithm more generic.
  • Top-down multi-level animation.
  • Move from prototype to real product.

B-Motion Studio

  • Experiment with existing Flash animation and B model of ClearSy.
  • Improve usability, more widgets.

Validation of ProB

  • Test coverage analysis for Prolog code.
  • Validation document to be delivered to Siemens.

Scalability

  • More experiments with SAT, SMT, BDD techniques.
  • Integration of Kodkod into ProB to solve complicated predicates over first order relations and simple sets.
  • Adaption of ProB for the upcoming mathematical extensions. Indeed, for the moment the Rodin user is often required to model basic datatypes (records, sequences,...) or operators (transitive closure) herself. This is a big challenge to the animator, which does not know that the constants and variables of the machine (e.g. injective functions) are "simply" meant to model quite basic datatypes. With the introduction of mathematical extensions for records, transitive closure, ... this hurdle will be overcome.

Usability

  • Feedback errors found by ProB into the PO view (as red icons).
  • Improve disprover, detect when it is a decision procedure.
  • Allow the user to easier inspect elements of the animated model: the cause of errors, why events are not enabled, etc.
  • Further improvements to the GUI: 2-D Viewer, better multi-level animation view.