D23 ProB
Overview
This part of the deliverable describes improvements of the ProB animation and model checking plug-in.
The improvements and development of ProB were mainly carried out by University of Düsseldorf, with some support by the University of Southampton. Furthermore, the work was driven by requirements of Siemens and SAP; some tool development was also undertaken by SAP.
New features:
- Multi-level animation and validation.
- B-Motion Studio.
- Disprover Support.
- First steps towards test-case generation.
Improvements:
- Scalability improvements driven by Siemens and SAP applications.
- Using proof information to improve model checking.
Other works:
- First steps towards validation of ProB for usage by Siemens in SIL-4 chain.
- Evaluation against SAT/SMT/BDD-based approaches.
Motivations
Multi-level Animation and Validation
Thus far ProB only allowed single-level animation, i.e. the animator would animate a single refinement level in isolation. This meant that ProB was not able to detect a large class of potential errors in the model:
- A broken gluing invariant.
- An invalid witness.
- Violation of guard strengthening.
- Violation of variant decrease (resp. decrease or stability) for convergent (resp. anticipated) events.
The new validation algorithm now can animate a range of refinements together. The user can decide which levels are to be animated together. As such, all of the above errors can now be detected by ProB. User experience is also improved, as he or she can inspect also the abstract variables. The new algorithm has been successfully applied to various case studies, and thus far up to 14 levels have been animated concurrently without problem.
B-Motion Studio
It is often very important to be able to show a formal model to a domain expert or manager, not versed in formal methods. For example, only a domain expert will be able to detect certain mistakes in the formal model. To enable to easily and quickly build graphical visualisations of Rodin models, we have developed B-Motion Studio. B-Motion Studio comes with a graphical editor to arrange graphical components and link them with the formal model. No new programming language has to be learned: the linking is described in B itself. To run a graphical visualisation, the ProB animator is used.
Test-Case Generation
During deployment in the SAP workpackage it became clear that test-case generation from the Event-B models is required for success. In this task, we have developed a first algorithm for test-case generation, which ensures complete transition coverage of a high-level model, and translates the test-cases into traces of a refined model, so that the tests can be run on the "real" system. Optimisations, to reduce the length and number of test cases, as well as to minimise race conditions, have been implemented.
Scalability, Application by Siemens and Validation
We tackled a case study in WP2, which centres on the San Juan metro system installed by Siemens. The control software was developed and formally proven with B. However, the development contains certain assumptions about the actual rail network topology which have to be validated separately in order to ensure safe operation. For this task, Siemens has developed custom proof rules for AtelierB. AtelierB, however, was unable to deal with about 80 properties of the deployment (running out of memory). These properties thus had to be validated by hand at great expense (and they need to be revalidated whenever the rail network infrastructure changes).
The motivation then was to try and use ProB for this task. This required a considerable amount of work on improving the scalability of the ProB kernel, to be able to deal with large sets and relations. The ProB parser and type checker also had to be re-developed to be able to deal with large industrial specifications.
The case study was a success: ProB was able to validate all of the about 300 properties of the San Juan deployment, detecting exactly the same faults automatically in around 17 minutes that were manually uncovered in about one man-month.
This leads to the next task: the issue of validating ProB, so that it can be integrated into the SIL4 development chain at Siemens.
Proof Directed Model Checking
In order to improve the performance and scalability of animation and model checking with ProB, we want to make use of the proof information available in an integrated platform such as Rodin. In particular, we can use the information about which POs have already been discharged to simplify the task of the model checker and to guide the order in which it evaluates states.
Our first implementation has shown significant reduction of the model checking effort for industrial applications. It uses proof information obtained from Rodin to remove invariants that are proven to hold. Actually, if unproven(e) is the unproven part of the invariant for event e, and a state is reachable via events e and f, we only need to check the intersection of unproven(e) and unproven(f) (see Jens Bendisposto, Michael Leuschel. Proof Assisted Model Checking for B. ICFEM 2009.).
SAT/SMT/Kodkod
In this subtask we investigate alternate approaches to validate high-level B models using techniques and tools based on BDDs, SAT-solving and SMT-solving. The overall motivation is to improve the scalability of the animator and model checker.
When searching for variable values that satisfy a certain predicate, experience showed that ProB performs best when some values are already known, as in finding state transitions where all values of the previous state are known. When looking for constants that satisfy the axioms or when trying to find counterexamples for proof obligations (see next Disprover paragraph), SAT resp. SMT solving techniques look very promising to support ProB's constraint solving approach.
Disprover
In order to help the user, we wanted to make it possible to apply ProB to individual proof obligations. In some cases, this enables proving a sequent by exhaustive case analysis. Also, if ProB finds a counterexample, the user gets important feedback: the proof obligation cannot be discharged, along with a reason why.
Choices / Decisions
One important choice for ProB is to use the same constraint solving kernel and interpreter for B, Event-B and Z. This decision, allows us to maintain a tool capable of animating and model checking these three formalisms. It also gives us a much wider range of test cases. For example, most of the regression tests of ProB come in the form of B machines (Rodin archives are much "cumbersome" as a basis of regression tests: they need to be upgraded and imported into workspaces). This choice also enabled us to achieve the successful deployment of ProB in WP2, where the use of classical B was mandatory.
Concerning the inclusion of SMT, SAT, BDD techniques into ProB, no decision has been taken yet. We are still investigating the possibilities.
Concerning B-Motion Studio, we decided not to use Flash (we had an earlier prototype using Flash). We wanted a tool that can be easily used and installed into Rodin. We also wanted a tool that can be used without having to learn a new programming language (Action Script).
Available Documentation
User Manual
Published Papers
Below is a list of published papers, along with an abstract.
Improved Kernel to deal with large sets and relations
In this part we describe the successful application of the ProB validation tool on an industrial case study. The case study centres on the San Juan metro system installed by Siemens. The control software was developed and formally proven with B. However, the development contains certain assumptions about the actual rail network topology which have to be validated separately in order to ensure safe operation. For this task, Siemens has developed custom proof rules for AtelierB. AtelierB, however, was unable to deal with about 80 properties of the deployment (running out of memory). These properties thus had to be validated by hand at great expense (and they need to be revalidated whenever the rail network infrastructure changes). In this paper we show how we were able to use ProB to validate all of the about 300 properties of the San Juan deployment, detecting exactly the same faults automatically in around 17 minutes that were manually uncovered in about one man-month. This achievement required the extension of the ProB kernel for large sets as well as an improved constraint propagation phase. We also outline some of the effort and features that were required in moving from a tool capable of dealing with medium-sized examples towards a tool able to deal with actual industrial specifications. Notably, a new parser and type checker had to be developed. We also touch upon the issue of validating ProB, so that it can be integrated into the SIL4 development chain at Siemens
Multi-Level Animation and Validation
We provide a detailed description of refinement in Event-B, both as a contribution in itself and as a foundation for the approach to simultaneous animation of multiple levels of refinement that we propose. We present an algorithm for simultaneous multi-level animation of refinement, and show how it can be used to detect a variety of errors that occur frequently when using refinement. The algorithm has been implemented in ProB and we applied it to several case studies, showing that multi-level animation is tractable also on larger models.
See also Stefan Hallerstede, Michael Leuschel. How to Explain Mistakes. TFM'09.
Test Case Generation
Choreography models describe the communication protocols between services. Testing of service choreographies is an important task for the quality assurance of service-based systems as used e.g. in the context of service-oriented architectures (SOA). The formal modelling of service choreographies enables a model-based integration testing (MBIT) approach. We present MBIT methods for our service choreography modeling approach called Message Choreography Models (MCM). For the model-based testing of service choreographies, MCMs are translated into Event-B models and used as input for our test generator which uses the model checker ProB.
Proof-Directed Model Checking
With the aid of the ProB plug-in, the Rodin Platform provides an integrated environment for editing, proving, animating and model checking Event-B models. This is of considerable benefit to the modeller, as it allows him to switch between the various tools to validate, debug and improve his or her models. The crucial idea of this paper is that the integrated platform also provides benefits to the tool developer, i.e. it allows easy access to information from other tools. Indeed, there has been considerable interest in combining model checking, proving and testing. In previous work we have already shown how a model checker can be used to complement the Event-B proving environment, by acting as a disprover. In this paper we show how the prover can help to improve the efficiency of the animator and model checker.
Jens Bendisposto, Michael Leuschel. Proof Assisted Model Checking for B. ICFEM 2009.
Debugging Tricky Proof Obligations with the ProB Disprover
While a large number of proof obligations can be discharged automatically by tools such as the Rodin platform, a considerable number still have to be proven interactively. In this paper, we describe a disprover plug-in for Rodin that utilizes ProB to automatically find counterexamples for a given problematic proof obligation. In case the disprover finds a counterexample, the user can directly investigate the source of the problem, as pinpointed by the counterexample. We also discuss under which circumstances our plug-in can be used as a prover, i.e. when the absence of a counterexample actually is a proof of the proof obligation.
Inspection of Alternate Approaches
ProB is a model checker for high-level B and Event-B models based on constraint-solving. In this paper we investigate alternate approaches for validating high-level B models using techniques and tools based on using BDDs, SAT-solving and SMT-solving. In particular, we examine whether ProB can be complemented or even supplanted by using one of the tools BDDBDDB, Kodkod or SAL.
Validation of ProB
Symmetry reduction is a model checking technique that can help to alleviate the problem of state space explosion, by preventing redundant state space exploration. In previous work, we have developed three effective approaches to symmetry reduction for B that have been implemented into the ProB model checker, and we have proved the soundness of our state symmetries. However, it is also important to show that our techniques are sound with respect to standard model checking, at the algorithmic level. In this paper, we present a retrospective B development that addresses this issue through a series of B refinements. This work also demonstrates the valuable insights into a system that can be gained through formal modelling.
B-Motion Studio
B-Motion Studio provides a way to quickly generate domain specific visualisations for a formal model, enabling domain experts and managers to understand and validate the model. We also believe that our tool will be of use when teaching formal methods, both during lectures as a way to motivate students to write their own formal models.
Planning
In future, it is planned to work on the following topics:
Model-based Testing
- Directed model checking to achieve coverage (DEPLOY extension; flow graphs).
- Integrate algorithm into Rodin.
- Make algorithm more generic.
- Top-down multi-level animation.
- Move from prototype to real product.
B-Motion Studio
- Experiment with existing Flash animation and B model of ClearSy.
- Improve usability, more widgets.
Validation of ProB
- Test coverage analysis for Prolog code.
- Validation document to be delivered to Siemens.
Scalability
- More experiments with SAT, SMT, BDD techniques.
- Integration of Kodkod into ProB to solve complicated predicates over first order relations and simple sets.
- Adaption of ProB for the upcoming mathematical extensions. Indeed, for the moment the Rodin user is often required to model basic datatypes (records, sequences,...) or operators (transitive closure) herself. This is a big challenge to the animator, which does not know that the constants and variables of the machine (e.g. injective functions) are "simply" meant to model quite basic datatypes. With the introduction of mathematical extensions for records, transitive closure, ... this hurdle will be overcome.
Usability
- Feedback errors found by ProB into the PO view (as red icons).
- Improve disprover, detect when it is a decision procedure.
- Allow the user to easier inspect elements of the animated model: the cause of errors, why events are not enabled, etc.
- Further improvements to the GUI: 2-D Viewer, better multi-level animation view.
