Synchronization Synthesis for Network Programs

Downloads

NOTE: the "preprint" (updated) version contains the same core content, but is un-anonymized, and includes various formatting improvements, typo fixes, and clarifications, based on feedback from the reviewers. Additionally, the performance results appearing there are significantly better than in the accepted version, due to improvements in our interface with the SPIN model checker. Thus, we recommend using the preprint version.

Backup mirrors: paper.pdf, paper.pdf, paper-preprint.pdf, paper-preprint.pdf, sync-synth64.ova, sync-synth32.ova, sync-synth.tgz

AEC Conflicts

  • None

Our Experimental Platform (Host Machine):

  • x86_64 Ubuntu (16.04.1) Linux workstation with 20 GB RAM and quad-core i5-4570 CPU (3.2 GHz)

NOTE: we recommend using a 64bit host machine with at least 2+ processors and 8+ GB RAM.

Installation

There are several options for installation:

I. (Difficulty: easy) Use a provided Virtual Machine (VM) image.

  1. Download a VM image (preferably 64bit). See above for the download links.
  2. Start VirtualBox on your host machine.
  3. Select "File" -> "Import Appliance", and choose the downloaded image.
  4. If possible, give the VM more memory/processors to more closely match our baseline machine.
  5. You may need to turn on virtualization extensions in your BIOS to enable 64bit virtualization.
  6. Upon VM startup, the synth user should be logged in (username: synth, password: cav2017).
  7. Open a terminal and navigate to the ~/tools/sync-synth directory (if this doesn't happen automatically).
  8. You can check that the synthesizer is working properly by typing ./synth help

II. (Difficulty: moderate) Build/install automatically using the provided Ubuntu Linux installer script.

(NOTE: the install script is designed for Ubuntu Linux (we have tested it on Ubuntu 16.04 and 14.04))

  1. Make a tools directory (for these instructions, we assume it is ~/tools).
mkdir -p ~/tools
cd ~/tools
  1. Download the source tarball (see above for the <download-link>), place it in the tools directory, extract it, and run the install script to set up the required dependencies.
wget <download-link>
tar -xvf sync-synth.tgz
cd sync-synth
./install_ubuntu.sh
source ~/.bashrc
  1. Build the synthesizer.
make

You can check that the synthesizer was built correctly by typing ./synth help

III. (Difficulty: hard) Build/install manually on Ubuntu Linux.

  1. Make a tools directory (for these instructions, we assume it is ~/tools).
mkdir -p ~/tools
cd ~/tools
  1. Download the source tarball (see above for the <download-link>), place it in the tools directory, and extract it.
wget <download-link>
tar -xvf sync-synth.tgz
  1. Install all of the prerequisite packages.
sudo apt-get install -y python-networkx graphviz gnuplot git bison flex default-jdk
sudo apt-get install -y python-numpy python-scipy python-matplotlib python-pydot python-pygraphviz
sudo apt-get install -y python-pip
sudo pip install pydotplus
  1. Build and install the Z3 SMT solver's Java bindings.
git clone https://github.com/Z3Prover/z3.git
cd z3
git reset --hard 758c9cd7a0c6346abc11c6dfefbeaf092b23c01d
python scripts/mk_make.py --java
cd build
make
sudo make install
cd ../..
  1. Install the SPIN model checker. Note that the SPIN source code contains hardcoded limits on many buffer sizes, etc., so we increase some of these before compiling.
wget http://spinroot.com/spin/Src/spin646.tar.gz
tar -xvf spin646.tar.gz
cd Spin/Src*
find . -type f -print0 | xargs -0 sed -i "s/\\[2048\\]/[524288]/g"
find . -type f -print0 | xargs -0 sed -i "s/ 2048)/ 524288)/g"
find . -type f -print0 | xargs -0 sed -i "s/malloc(2048)/malloc(524288)/g"
find . -type f -print0 | xargs -0 sed -i "s/2047/524287/g"
find . -type f -print0 | xargs -0 sed -i "s/4096/1048576/g"
find . -type f -print0 | xargs -0 sed -i "s/20000/100000/g"
make
sudo make install
cd ../..
  1. Install ltl2tgba, an LTL-to-Buchi automaton translator. We found this to have better performance than SPIN's built-in translator.
mkdir spot
cd spot
wget http://www.lrde.epita.fr/dload/spot/spot-2.2.2.tar.gz
tar -xvf spot-2.2.2.tar.gz
cd spot-2.2.2/
./configure --disable-python
make
sudo make install
echo "export LD_LIBRARY_PATH=\"$LD_LIBRARY_PATH:/usr/local/lib\"" >> ~/.bashrc
source ~/.bashrc
cd ..
  1. (Optional) Install ltl2ba, an LTL-to-Buchi automaton utility. By default, we do not use this, because we found the previously-mentioned ltl2tgba tool to have better performance. If you want to use this one instead, you must set the flag boolean spinUseSpot = true in Main.java to false before building the synthesizer, but we do not recommend doing this, and we have not tested it extensively.
mkdir ltl2ba
cd ltl2ba
wget http://spinroot.com/spin/Src/ltl2ba.tar.gz
tar -xvf ltl2ba.tar.gz
make
sudo cp ltl2ba /usr/local/bin
cd ..
  1. Install the clang C compiler, version 3.5. We found this to have better performance than GCC in our experiments. If you want to use GCC instead, you must change the line CC=clang in spin/Makefile to CC=gcc before running the synthesizer.
sudo apt-get install -y clang-3.5
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-3.5 350

Typing clang --version should list version 3.5.X, where "X" is some string.

  1. Build the synthesizer.
cd sync-synth
make

You can check that the synthesizer was built correctly by typing ./synth help

Source-Code and Tool Organization

The sync-synth directory (~/tools/sync-synth on the VM) is where the main components are located:

  1. The sync-synth/experiments subdirectory contains the functionality to build and visualize network topologies used in the experiments.

  2. The sync-synth/results subdirectory contains the experimental data (in CSV format) that was obtained on our experimental platform, and used to generate the graphs in the paper.

  3. The sync-synth/spin subdirectory contains the Promela model that is used by SPIN to produce a verifier (LTL model checker). SPIN produces C code from this model, which is then compiled using clang into an executable binary. This binary corresponds to the event-net verifier described in the paper -- the synthesizer calls the verifier repeatedly, each time passing it a new event net via command-line arguments.

  4. Within the sync-synth directory itself:

  • synth - the main script which runs the synthesizer (run ./synth help to see options).
  • Main.java - the core of the synthesizer functionality (CEGIS loop, etc.)
  • PetriNet.java, Place.java, Transition.java - functionality to encode/manipulate event nets
  • Network.java, Edge.java, Loc.java, Node.java - functionality to encode/manipulate network topologies
  • Sys.java - functionality for outputting to logfiles
  • Parser.java - parser for experiment input (*.input) files
  • Experiments.java - functionality for generating input files for the scalability experiments
  • PDFUtils.java - functionality for producing PDF files from *.dot files
  • table.py - script for collecting runtime information from synthesizer log files, and putting it in CSV format
  • *.pg - Gnuplot scripts for plotting CSV files
  • ex*.input - the experiment input files for Examples 1-5
  • inputs.tgz - the experiment input files for the scalability experiments

Basic Usage

  1. Programming with Event Nets.

Event Nets are detailed in Section 3 of the paper. We provide an ASCII input format for writing event nets, as well as for specifying the desired network topology. Let us start by examining a specific example, namely the provided ex01_isolation.input file.

init_marking { P1, P4 }

topology {
  H4   S4:2,
  H3   S3:2,
  H2   S2:2,
  H1   S1:2,
  S1:1 S5:1,
  S2:1 S5:2,
  S3:1 S5:3,
  S4:1 S5:4
}

config P1 (red) {
  S1:2 (true) -> () S1:1,
  S5:1 (true) -> () S5:2,
  S5:1 (true) -> () S5:3,
  S5:1 (true) -> () S5:4
}

config P2 { }

config P3 (orange) {
  S3:2 (true) -> () S3:1,
  S5:3 (true) -> () S5:4,
  S5:3 (true) -> () S5:1,
  S5:3 (true) -> () S5:2
}

config P4 (green) {
  S2:1 (true) -> () S2:2
}

config P5 { }

config P6 (violet) {
  S4:1 (true) -> () S4:2
}

event T1 1 H2 {P1} -> {P2}
event T2 1 H2 {P2} -> {P3}

event T3 2 H2 {P4} -> {P5}
event T4 2 H2 {P5} -> {P6}

ltl reach1 (([] ((loc==H1) -> []!(loc==H4))))
ltl reach2 (([] ((loc==H3) -> []!(loc==H2))))

This code corresponds to the Tenant Isolation in a Datacenter example (Example #1) described throughout the paper, and shown in Figure 1(a-b). Note that in the code, numeric IDs are used for the transitions ("T1", "T2", etc.), while in the paper, the corresponding letters of the alphabet are used ("A", "B", etc.).

  • The init_marking statement specifies which places are initially marked (in this case, places 1 and 4, as seen in Figure 1(b)).

  • The topology statement declares a set of (bidirectional) links of the form (<loc> - <loc>), where <loc> is either a host ID (e.g., "H1"), or a switch/port pair (e.g., "S4:2", where 4 is the switch ID, and 2 is the port ID). The topology in the above example corresponds to Figure 1(a) in the paper.

  • Each place in the event net is specified by a statement of the form config <id> <color> { <rule>, ... }, where <id> is the place ID (such as "P1"), <color> is either blank or a parenthesized background color for the place (e.g., "(red)"), and <rule> is a forwarding rule in this place's configuration.

  • Each rule has the form <loc> (<cond>) -> (<mod>; ...) <loc>. The first location is an input port on a switch, and the second one is the output port (on the same switch) where a packet whose header fields match condition <cond> should be forwarded, after performing an optional sequence of modifications <mod> to the header fields. We provide three header fields, src (source address), dst (destination address), and ty (packet type), allowing conditions such as ty==1 && src==2 and modifications such as ty = 3.

  • Each transition in the event net is specified by a statement of the form event <id> <proc_id> <loc> { <place_id>, ... } -> { <place_id>, ... }, where <id> is the transition ID (e.g., "T1"), <proc_id> is a numeric ID specifying which process the transition belongs to, <loc> is the event's location, and the <place_id> are place IDs. The first set denotes the pre-places, and the second the post-places for the transition, as described in Section 3.

  • An LTL property can be specified with a statement of the form ltl <name> (<formula>), where <name> is an identifier describing the property, and <formula> is an LTL formula, which we will describe next.

  1. Specifying LTL formulas.

The syntax of LTL formulas is described at the end of Section 3 in the preprint version of the paper (Figure 3). As mentioned previously, we provide the three header fields src, dst, and ty. For LTL properties, we also provide the special field loc which denotes the current location of the packet as it moves through the network.

The LTL properties in our encoding follow the syntax of SPIN. In particular, our "G" (globally) operator is denoted [], our "F" (eventually) operator is denoted <>, our "R" (release) operator is denoted V, etc., and boolean combinations have a similar style to that of C, e.g., equality is denoted using ==, etc.

Returning to the above example, let us examine the LTL property reach1. It has the form ([] ((loc==H1) -> []!(loc==H4))). This is our encoding of Property #1 in the "Example - Tenant Isolation in a Datacenter" paragraph of Section 2 in the paper, which is later encoded as LTL Property #1 in Section 5. At a high level, this property says that any packet which is at Host 1 should never be able to reach Host 4.

  1. Running the Synthesizer.

Our synthesizer takes as input an event net, network topology, and LTL properties (in the form of a *.input file, as described above), and produces a new event net which consists of the original event net plus added synchronization constructs (synchronization skeletons) which ensure that it does not violate the LTL properties in any configuration (the synthesis algorithm is described in Section 4 of the paper).

Returning to the ex01_isolation.input example described above, we can run the synthesizer as follows:

./synth run ex01_isolation.input -letters -quiet

This command produces a log file synth1.log with the full output of the tool (including messages from SPIN, etc.). It also produces a directory gen1 which contains the compiled verifier executable (pan), as well as several PDF files which allow the synthesizer's output to be visualized. The -letters option replaces the numeric identifiers of the transitions (e.g., "T1", "T2", etc.) with the corresponding letters of the alphabet ("A", "B", etc.), to allow for more easily-readable output.

In particular, we can type evince gen1/network.pdf to see the basic network topology, which corresponds to Figure 1(a) in the paper. Similarly, we can type evince gen1/petri.pdf to the see the initial event net, corresponding to Figure 1(b) in the paper, and evince gen1/petri_mod.pdf to see the final (synthesized) event net, corresponding to Figure 1(d).

NOTE: the IDs on the places in the added synchronization skeletons (shapes/lines outlined with blue in the emitted PDFs) may not match those in the paper (we made them consecutive in the paper).

The files gen1/petri_mod_*.pdf show the full sequence of event nets generated during the CEGIS loop (gen1/petri_mod.pdf is simply a copy of the last one in the sequence). This particular example is able to be solved in a single step, but we can force the synthesizer to produce more intermediate event nets (specifically, the ones shown in Figure 1), by doing the following:

./synth run ex01_isolation.input -letters -quiet -single -seed 8

This causes the tool to first return the [C,D] counterexample describing the violation of Property 1, and then return the [A,B] counterexample describing the violation of Property 2 (the -single option tells the tool to use counterexamples from at most one property during each CEGIS iteration, and -seed 8 switches from the default Z3 random seed to a seed of 8).

Typing evince gen1/petri_mod_000.pdf now shows the event net in Figure 1(c) of the paper, and evince gen1/petri_mod_001.pdf shows the final event net in Figure 1(d).

In quiet mode (-quiet), the synthesizer's output looks like the following:

Writing PDF: gen1/petri_mod_000.pdf

>> Iteration: { 2 } (00:00:03 elapsed)
-Running Spin on property 'onesafe'...
  Verifier result: 1 (0.324 seconds)
  Counterexample: {[]}
-Running Spin on property 'progr3'...
  Verifier result: 2 (0.303 seconds)
  Counterexample: {[A, B, C, D]}
-Running Spin on property 'reach1'...
  Verifier result: 1 (0.323 seconds)
  Counterexample: {[]}
-Running Spin on property 'reach2'...
  Verifier result: 2 (0.317 seconds)
  Counterexample: {[A, B]}
-Synthesizing...
  Successfully synthesized (0.054 seconds)

This is a single iteration of the CEGIS loop (specifically, the second iteration, i.e., the one operating on Figure 1(c) in the paper). The verifier is first run on the 1-safety (onesafe) property. No counterexample is obtained, meaning this property is satisfied. Next, the verifier is run on the (negation of the) progress (progr3) property, and a counterexample [A, B, C, D] is obtained, which corresponds to a concrete trace of the event net which satisfies the progress property. The reach1 property (Property #1 in the paper) is satisfied (no counterexample), since Figure 1(c) does not allow places 1 and 6 to be marked simultaneously. However, the reach2 property (Property #2 in the paper) fails (counterexample [A, B]), because if transition A fires, followed by transition B, then places 3 and 4 are marked simultaneously.

Claims and Assumptions Made in the Paper

The three research questions at the beginning of Section 5 in the paper enumerate the items we seek to demonstrate via the experimental results. Specifically:

  1. We can use our approach to model a variety of real-world network programs.

We show this by using event nets to capture the basic elements of five real-world applications, as described in the paper (in particular, the ones appearing in the "Tenant Isolation in a Datacenter", "Conflicting Controller Modules", "Discovery Forwarding Loop", "Policy Composition", and "Topology Changes during Update" subsections of Section 5).

  1. Our tool is able to fix realistic concurrency-related bugs in network programs.

We show this by using our tool to synthesize event nets that fix the bugs in the five example applications.

  1. The performance of our tool is reasonable when applied to real networks.

We show this by measuring the runtime of the synthesizer operating on the various example programs, and by choosing one of the examples (Example #1) and measuring synthesizer performance as we scale up the size of the network to 1000+ nodes.

As mentioned in the paper, we assume that the programmer does not use the next-time (X) operator in the LTL formulas, since this restriction allows us to take advantage of performance improvements offered by partial-order reduction in SPIN. Additionally, we have the implicit assumption that there is a mechanism for implementing our event nets efficiently in a real SDN -- however, this is out-of-scope for the paper, and we do not address it further in the artifact.

Reproducing Results from the Paper

  1. Running the synthesizer on the Examples 1-5 discussed in Section 5 of the paper.

We have included the five examples: ex01_isolation.input (Tenant Isolation in a Datacenter), ex02_conflict.input (Conflicting Controller Modules), ex03_loop.input (Discovery Forwarding Loop), ex04_composition.input (Policy Composition), and ex05_exclusive.input (Topology Changes during Update). The synthesizer can be run on each of these individually as discussed in the above Basic Usage section.

The following can be used to run the synthesizer on all of them:

make examples

This takes about 18 seconds on our experimental platform, and produces examples.csv, which corresponds to Figure 8 in the preprint paper (first 5 rows of the Figure 5 table in the accepted paper). It also produces *.network.pdf which correspond to Figures 1(a) and 9-12(b) in the preprint paper (Figures 1(a) and 4(e-h) in the accepted paper). Similarly, it produces *.petri_mod.pdf, which correspond to Figures 1(d) and 9-12(a) in the preprint paper (Figures 1(d) and 4(a-d) in the accepted paper).

NOTE: the Z3 SMT solver appears to exhibit some nondeterminism, meaning that the generated examples.csv may not exactly match the results in the paper.

  1. Plotting cached results (i.e. data used in the paper).

The results directory contains the exact CSV data files used to produce the Figures in the paper (these were generated by running the experiments on our experimental platform). We include functionality to simply reproduce the PDF plots from this data.

The following command plots the data from the original (accepted) paper. The resulting file scalability01-lines-sw.pdf corresponds to Figure 5(a), and scalability01-lines-ft.pdf corresponds to Figure 5(b). Note that the plot style may look slightly different than in the paper (since we are using Gnuplot instead of LaTeX/TikZ).

make original-plots

The following command plots the data from the preprint (updated) paper. The resulting file scalability01-ft.pdf corresponds to Figure 13, scalability01-sw.pdf corresponds to Figure 15(a), and scalability01-zoo.pdf corresponds to Figure 15(b).

make updated-plots
  1. Generating the network topologies (and accompanying figures).

The experiments directory contains utilities for building/obtaining the real-world network topologies described in the paper. The following produces all of the topologies (encoded as GML files) in experiments/models. It also produces topo_fat.pdf, topo_small.pdf, and topo_zoo.pdf, corresponding to Figures 14(a-c) in the preprint paper.

make topos
experiments/fattree.py 3 2
experiments/smallworld.py 10 4 0.9
experiments/plot.py experiments/models/zoo/Nsfnet.gml
  1. Generating the input files for scalability experiments.

Once the topologies are available (see previous step), we can generate an input file for each topology, as described in the Scalability Experiments paragraph of Section 5 in the paper. The following simply unpacks the pre-generated input files from the included tarball inputs.tgz:

make setup-inputs

To force our tool to re-build all the input files from the topologies, you can run make rebuild-inputs. However, we do NOT recommend doing this full rebuild of the input files -- it takes about 35 minutes on our experimental platform, and you MUST be on a VM/machine with around 8+ GB RAM and Java JDK 1.8+ installed!

  1. Running the scalability experiments.

Once the scalability input files (inputs/*.input and inputs/zoo/*.input) are present, we can run the synthesizer on all of them. The following takes about 10 minutes on our experimental platform, and produces the data file all.csv:

make run-exp

We can then produce the graphs from this data:

make plots

This produces the following plots from the preprint paper: scalability01-ft.pdf corresponds to Figure 13, scalability01-sw.pdf corresponds to Figure 15(a), and scalability01-zoo.pdf corresponds to Figure 15(b).

It also produces the following plots from original (accepted) paper: scalability01-lines-sw.pdf corresponds to Figure 5(a), and scalability01-lines-ft.pdf corresponds to Figure 5(b). Note that we are now using more/larger topologies than at the time of paper submission, so these two graphs will look slightly different than in the accepted paper.

  1. Cleaning up.

While the following functionality is not strictly necessary, it may be useful in certain cases.

To remove the currently-generated plots (CSV, PDF, log, dot files, etc.), you can use:

make tidy

To remove all generated content and return the build to the initial state, you can use:

make clean