Synchronization Synthesis for Network Programs
Downloads
- Accepted Paper: paper.pdf
- "Preprint" (Updated) Version of Paper: paper-preprint.pdf
- Virtual Machine (VM) Image (4 GB): sync-synth64.ova, sync-synth32.ova (username:
synth
, password:cav2017
) - Source Tarball (3 MB): sync-synth.tgz
NOTE: the "preprint" (updated) version contains the same core content, but is un-anonymized, and includes various formatting improvements, typo fixes, and clarifications, based on feedback from the reviewers. Additionally, the performance results appearing there are significantly better than in the accepted version, due to improvements in our interface with the SPIN model checker. Thus, we recommend using the preprint version.
Backup mirrors: paper.pdf, paper.pdf, paper-preprint.pdf, paper-preprint.pdf, sync-synth64.ova, sync-synth32.ova, sync-synth.tgz
AEC Conflicts
- None
Our Experimental Platform (Host Machine):
- x86_64 Ubuntu (16.04.1) Linux workstation with 20 GB RAM and quad-core i5-4570 CPU (3.2 GHz)
NOTE: we recommend using a 64bit host machine with at least 2+ processors and 8+ GB RAM.
Installation
There are several options for installation:
I. (Difficulty: easy) Use a provided Virtual Machine (VM) image.
- Download a VM image (preferably 64bit). See above for the download links.
- Start VirtualBox on your host machine.
- Select "File" -> "Import Appliance", and choose the downloaded image.
- If possible, give the VM more memory/processors to more closely match our baseline machine.
- You may need to turn on virtualization extensions in your BIOS to enable 64bit virtualization.
- Upon VM startup, the
synth
user should be logged in (username:synth
, password:cav2017
). - Open a terminal and navigate to the
~/tools/sync-synth
directory (if this doesn't happen automatically). - You can check that the synthesizer is working properly by typing
./synth help
II. (Difficulty: moderate) Build/install automatically using the provided Ubuntu Linux installer script.
(NOTE: the install script is designed for Ubuntu Linux (we have tested it on Ubuntu 16.04 and 14.04))
- Make a
tools
directory (for these instructions, we assume it is~/tools
).
mkdir -p ~/tools
cd ~/tools
- Download the source tarball (see above for the
<download-link>
), place it in thetools
directory, extract it, and run the install script to set up the required dependencies.
wget <download-link>
tar -xvf sync-synth.tgz
cd sync-synth
./install_ubuntu.sh
source ~/.bashrc
- Build the synthesizer.
make
You can check that the synthesizer was built correctly by typing ./synth help
III. (Difficulty: hard) Build/install manually on Ubuntu Linux.
- Make a
tools
directory (for these instructions, we assume it is~/tools
).
mkdir -p ~/tools
cd ~/tools
- Download the source tarball (see above for the
<download-link>
), place it in thetools
directory, and extract it.
wget <download-link>
tar -xvf sync-synth.tgz
- Install all of the prerequisite packages.
sudo apt-get install -y python-networkx graphviz gnuplot git bison flex default-jdk
sudo apt-get install -y python-numpy python-scipy python-matplotlib python-pydot python-pygraphviz
sudo apt-get install -y python-pip
sudo pip install pydotplus
- Build and install the Z3 SMT solver's Java bindings.
git clone https://github.com/Z3Prover/z3.git
cd z3
git reset --hard 758c9cd7a0c6346abc11c6dfefbeaf092b23c01d
python scripts/mk_make.py --java
cd build
make
sudo make install
cd ../..
- Install the SPIN model checker. Note that the SPIN source code contains hardcoded limits on many buffer sizes, etc., so we increase some of these before compiling.
wget http://spinroot.com/spin/Src/spin646.tar.gz
tar -xvf spin646.tar.gz
cd Spin/Src*
find . -type f -print0 | xargs -0 sed -i "s/\\[2048\\]/[524288]/g"
find . -type f -print0 | xargs -0 sed -i "s/ 2048)/ 524288)/g"
find . -type f -print0 | xargs -0 sed -i "s/malloc(2048)/malloc(524288)/g"
find . -type f -print0 | xargs -0 sed -i "s/2047/524287/g"
find . -type f -print0 | xargs -0 sed -i "s/4096/1048576/g"
find . -type f -print0 | xargs -0 sed -i "s/20000/100000/g"
make
sudo make install
cd ../..
- Install
ltl2tgba
, an LTL-to-Buchi automaton translator. We found this to have better performance than SPIN's built-in translator.
mkdir spot
cd spot
wget http://www.lrde.epita.fr/dload/spot/spot-2.2.2.tar.gz
tar -xvf spot-2.2.2.tar.gz
cd spot-2.2.2/
./configure --disable-python
make
sudo make install
echo "export LD_LIBRARY_PATH=\"$LD_LIBRARY_PATH:/usr/local/lib\"" >> ~/.bashrc
source ~/.bashrc
cd ..
- (Optional) Install
ltl2ba
, an LTL-to-Buchi automaton utility. By default, we do not use this, because we found the previously-mentionedltl2tgba
tool to have better performance. If you want to use this one instead, you must set the flagboolean spinUseSpot = true
inMain.java
tofalse
before building the synthesizer, but we do not recommend doing this, and we have not tested it extensively.
mkdir ltl2ba
cd ltl2ba
wget http://spinroot.com/spin/Src/ltl2ba.tar.gz
tar -xvf ltl2ba.tar.gz
make
sudo cp ltl2ba /usr/local/bin
cd ..
- Install the
clang
C compiler, version 3.5. We found this to have better performance than GCC in our experiments. If you want to use GCC instead, you must change the lineCC=clang
inspin/Makefile
toCC=gcc
before running the synthesizer.
sudo apt-get install -y clang-3.5
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-3.5 350
Typing clang --version
should list version 3.5.X
, where "X" is some string.
- Build the synthesizer.
cd sync-synth
make
You can check that the synthesizer was built correctly by typing ./synth help
Source-Code and Tool Organization
The sync-synth
directory (~/tools/sync-synth
on the VM) is where the main components
are located:
-
The
sync-synth/experiments
subdirectory contains the functionality to build and visualize network topologies used in the experiments. -
The
sync-synth/results
subdirectory contains the experimental data (in CSV format) that was obtained on our experimental platform, and used to generate the graphs in the paper. -
The
sync-synth/spin
subdirectory contains the Promela model that is used by SPIN to produce a verifier (LTL model checker). SPIN produces C code from this model, which is then compiled usingclang
into an executable binary. This binary corresponds to the event-net verifier described in the paper -- the synthesizer calls the verifier repeatedly, each time passing it a new event net via command-line arguments. -
Within the
sync-synth
directory itself:
-
synth
- the main script which runs the synthesizer (run./synth help
to see options). -
Main.java
- the core of the synthesizer functionality (CEGIS loop, etc.) -
PetriNet.java
,Place.java
,Transition.java
- functionality to encode/manipulate event nets -
Network.java
,Edge.java
,Loc.java
,Node.java
- functionality to encode/manipulate network topologies -
Sys.java
- functionality for outputting to logfiles -
Parser.java
- parser for experiment input (*.input
) files -
Experiments.java
- functionality for generating input files for the scalability experiments -
PDFUtils.java
- functionality for producing PDF files from*.dot
files -
table.py
- script for collecting runtime information from synthesizer log files, and putting it in CSV format -
*.pg
- Gnuplot scripts for plotting CSV files -
ex*.input
- the experiment input files for Examples 1-5 -
inputs.tgz
- the experiment input files for the scalability experiments
Basic Usage
- Programming with Event Nets.
Event Nets are detailed in Section 3 of the paper.
We provide an ASCII input format for writing event nets, as well as for
specifying the desired network topology. Let us start by examining a
specific example, namely the provided ex01_isolation.input
file.
init_marking { P1, P4 }
topology {
H4 S4:2,
H3 S3:2,
H2 S2:2,
H1 S1:2,
S1:1 S5:1,
S2:1 S5:2,
S3:1 S5:3,
S4:1 S5:4
}
config P1 (red) {
S1:2 (true) -> () S1:1,
S5:1 (true) -> () S5:2,
S5:1 (true) -> () S5:3,
S5:1 (true) -> () S5:4
}
config P2 { }
config P3 (orange) {
S3:2 (true) -> () S3:1,
S5:3 (true) -> () S5:4,
S5:3 (true) -> () S5:1,
S5:3 (true) -> () S5:2
}
config P4 (green) {
S2:1 (true) -> () S2:2
}
config P5 { }
config P6 (violet) {
S4:1 (true) -> () S4:2
}
event T1 1 H2 {P1} -> {P2}
event T2 1 H2 {P2} -> {P3}
event T3 2 H2 {P4} -> {P5}
event T4 2 H2 {P5} -> {P6}
ltl reach1 (([] ((loc==H1) -> []!(loc==H4))))
ltl reach2 (([] ((loc==H3) -> []!(loc==H2))))
This code corresponds to the Tenant Isolation in a Datacenter example (Example #1) described throughout the paper, and shown in Figure 1(a-b). Note that in the code, numeric IDs are used for the transitions ("T1", "T2", etc.), while in the paper, the corresponding letters of the alphabet are used ("A", "B", etc.).
-
The
init_marking
statement specifies which places are initially marked (in this case, places 1 and 4, as seen in Figure 1(b)). -
The
topology
statement declares a set of (bidirectional) links of the form(<loc> - <loc>)
, where<loc>
is either a host ID (e.g., "H1"), or a switch/port pair (e.g., "S4:2", where 4 is the switch ID, and 2 is the port ID). The topology in the above example corresponds to Figure 1(a) in the paper. -
Each place in the event net is specified by a statement of the form
config <id> <color> { <rule>, ... }
, where<id>
is the place ID (such as "P1"),<color>
is either blank or a parenthesized background color for the place (e.g., "(red)"), and<rule>
is a forwarding rule in this place's configuration. -
Each rule has the form
<loc> (<cond>) -> (<mod>; ...) <loc>
. The first location is an input port on a switch, and the second one is the output port (on the same switch) where a packet whose header fields match condition<cond>
should be forwarded, after performing an optional sequence of modifications<mod>
to the header fields. We provide three header fields, src (source address), dst (destination address), and ty (packet type), allowing conditions such asty==1 && src==2
and modifications such asty = 3
. -
Each transition in the event net is specified by a statement of the form
event <id> <proc_id> <loc> { <place_id>, ... } -> { <place_id>, ... }
, where<id>
is the transition ID (e.g., "T1"),<proc_id>
is a numeric ID specifying which process the transition belongs to,<loc>
is the event's location, and the<place_id>
are place IDs. The first set denotes the pre-places, and the second the post-places for the transition, as described in Section 3. -
An LTL property can be specified with a statement of the form
ltl <name> (<formula>)
, where<name>
is an identifier describing the property, and<formula>
is an LTL formula, which we will describe next.
- Specifying LTL formulas.
The syntax of LTL formulas is described at the end of Section 3 in the preprint version of the paper (Figure 3). As mentioned previously, we provide the three header fields src, dst, and ty. For LTL properties, we also provide the special field loc which denotes the current location of the packet as it moves through the network.
The LTL properties in our encoding follow the syntax of SPIN.
In particular,
our "G" (globally) operator is denoted []
,
our "F" (eventually) operator is denoted <>
,
our "R" (release) operator is denoted V
, etc.,
and boolean combinations have a similar style to that of C,
e.g., equality is denoted using ==
, etc.
Returning to the above example, let us examine the LTL property reach1
.
It has the form ([] ((loc==H1) -> []!(loc==H4)))
.
This is our encoding of Property #1 in the
"Example - Tenant Isolation in a Datacenter" paragraph of Section 2
in the paper, which is later encoded as LTL Property #1 in Section 5.
At a high level, this property says that any packet which is at Host 1
should never be able to reach Host 4.
- Running the Synthesizer.
Our synthesizer takes as input an event net, network
topology, and LTL properties (in the form of a *.input
file, as described above),
and produces
a new event net which consists of the original event net plus added synchronization
constructs (synchronization skeletons) which ensure that it does not violate the
LTL properties in any configuration (the synthesis algorithm is described in
Section 4 of the paper).
Returning to the ex01_isolation.input
example described above, we can run the
synthesizer as follows:
./synth run ex01_isolation.input -letters -quiet
This command produces a log file synth1.log
with the full output of the tool (including
messages from SPIN, etc.). It also produces a directory gen1
which contains the
compiled verifier executable (pan
), as well as several PDF files which allow the
synthesizer's output to be visualized.
The -letters
option replaces the numeric identifiers of the transitions (e.g., "T1", "T2", etc.)
with the corresponding letters of the alphabet ("A", "B", etc.), to allow for more easily-readable
output.
In particular, we can type evince gen1/network.pdf
to see the basic network topology,
which corresponds to Figure 1(a) in the paper. Similarly, we can type
evince gen1/petri.pdf
to the see the initial event net, corresponding to Figure 1(b)
in the paper, and evince gen1/petri_mod.pdf
to see the final (synthesized)
event net, corresponding to Figure 1(d).
NOTE: the IDs on the places in the added synchronization skeletons (shapes/lines outlined with blue in the emitted PDFs) may not match those in the paper (we made them consecutive in the paper).
The files gen1/petri_mod_*.pdf
show the full sequence of event nets generated during
the CEGIS loop (gen1/petri_mod.pdf
is simply a copy of the last one in the sequence).
This particular example is able to be solved in a single step, but we can force the
synthesizer to produce more intermediate event nets (specifically, the ones shown in
Figure 1), by doing the following:
./synth run ex01_isolation.input -letters -quiet -single -seed 8
This causes the tool to first return the [C,D] counterexample describing the violation of
Property 1, and then return the [A,B] counterexample describing the violation of Property 2
(the -single
option tells the tool to use counterexamples from at most one property
during each CEGIS iteration, and -seed 8
switches from the default Z3 random seed to a seed of 8
).
Typing evince gen1/petri_mod_000.pdf
now shows the event net in Figure 1(c) of the paper,
and evince gen1/petri_mod_001.pdf
shows the final event net in Figure 1(d).
In quiet mode (-quiet
), the synthesizer's output looks like the following:
Writing PDF: gen1/petri_mod_000.pdf
>> Iteration: { 2 } (00:00:03 elapsed)
-Running Spin on property 'onesafe'...
Verifier result: 1 (0.324 seconds)
Counterexample: {[]}
-Running Spin on property 'progr3'...
Verifier result: 2 (0.303 seconds)
Counterexample: {[A, B, C, D]}
-Running Spin on property 'reach1'...
Verifier result: 1 (0.323 seconds)
Counterexample: {[]}
-Running Spin on property 'reach2'...
Verifier result: 2 (0.317 seconds)
Counterexample: {[A, B]}
-Synthesizing...
Successfully synthesized (0.054 seconds)
This is a single iteration of the CEGIS loop (specifically, the second iteration,
i.e., the one operating on Figure 1(c) in the paper).
The verifier is first run on the 1-safety (onesafe
) property. No counterexample is
obtained, meaning this property is satisfied.
Next, the verifier is run on the (negation of the) progress (progr3
) property, and
a counterexample [A, B, C, D]
is obtained, which corresponds to a concrete trace
of the event net which satisfies the progress property.
The reach1
property (Property #1 in the paper) is satisfied (no counterexample), since
Figure 1(c) does not allow places 1 and 6 to be marked simultaneously.
However, the reach2
property (Property #2 in the paper) fails (counterexample [A, B]
),
because if transition A fires, followed by transition B, then places 3 and 4
are marked simultaneously.
Claims and Assumptions Made in the Paper
The three research questions at the beginning of Section 5 in the paper enumerate the items we seek to demonstrate via the experimental results. Specifically:
- We can use our approach to model a variety of real-world network programs.
We show this by using event nets to capture the basic elements of five real-world applications, as described in the paper (in particular, the ones appearing in the "Tenant Isolation in a Datacenter", "Conflicting Controller Modules", "Discovery Forwarding Loop", "Policy Composition", and "Topology Changes during Update" subsections of Section 5).
- Our tool is able to fix realistic concurrency-related bugs in network programs.
We show this by using our tool to synthesize event nets that fix the bugs in the five example applications.
- The performance of our tool is reasonable when applied to real networks.
We show this by measuring the runtime of the synthesizer operating on the various example programs, and by choosing one of the examples (Example #1) and measuring synthesizer performance as we scale up the size of the network to 1000+ nodes.
As mentioned in the paper, we assume that the programmer does not use the
next-time (X
) operator in the LTL formulas, since this restriction allows us to take advantage
of performance improvements offered by partial-order reduction in SPIN.
Additionally, we have the implicit assumption that there is a mechanism for implementing our event
nets efficiently in a real SDN -- however, this is out-of-scope for the paper, and we
do not address it further in the artifact.
Reproducing Results from the Paper
- Running the synthesizer on the Examples 1-5 discussed in Section 5 of the paper.
We have included the five examples: ex01_isolation.input
(Tenant Isolation in a Datacenter),
ex02_conflict.input
(Conflicting Controller Modules), ex03_loop.input
(Discovery Forwarding
Loop), ex04_composition.input
(Policy Composition), and ex05_exclusive.input
(Topology Changes
during Update). The synthesizer can be run on each of these individually as discussed in the above
Basic Usage section.
The following can be used to run the synthesizer on all of them:
make examples
This takes about 18 seconds on our experimental platform, and
produces examples.csv
, which corresponds to Figure 8 in the preprint paper (first 5 rows of the
Figure 5 table in the accepted paper).
It also produces *.network.pdf
which correspond to Figures 1(a) and 9-12(b) in the
preprint paper (Figures 1(a) and 4(e-h) in the accepted paper).
Similarly, it produces *.petri_mod.pdf
, which correspond to Figures 1(d) and 9-12(a) in the preprint
paper (Figures 1(d) and 4(a-d) in the accepted paper).
NOTE: the Z3 SMT solver appears to exhibit some nondeterminism, meaning that the generated
examples.csv
may not exactly match the results in the paper.
- Plotting cached results (i.e. data used in the paper).
The results
directory contains the exact CSV data files used to produce the Figures in the paper
(these were generated by running the experiments on our experimental platform).
We include functionality to simply reproduce the PDF plots from this data.
The following command plots the data from the original (accepted) paper. The resulting
file scalability01-lines-sw.pdf
corresponds to Figure 5(a),
and scalability01-lines-ft.pdf
corresponds to Figure 5(b). Note that the plot
style may look slightly different than in the paper (since we are using Gnuplot instead
of LaTeX/TikZ).
make original-plots
The following command plots the data from the preprint (updated) paper. The resulting file
scalability01-ft.pdf
corresponds to Figure 13,
scalability01-sw.pdf
corresponds to Figure 15(a), and
scalability01-zoo.pdf
corresponds to Figure 15(b).
make updated-plots
- Generating the network topologies (and accompanying figures).
The experiments
directory contains utilities for building/obtaining the real-world network
topologies described in the paper.
The following produces all of the topologies (encoded as GML files) in experiments/models
.
It also produces topo_fat.pdf
, topo_small.pdf
, and topo_zoo.pdf
, corresponding to
Figures 14(a-c) in the preprint paper.
make topos
experiments/fattree.py 3 2
experiments/smallworld.py 10 4 0.9
experiments/plot.py experiments/models/zoo/Nsfnet.gml
- Generating the input files for scalability experiments.
Once the topologies are available (see previous step), we can generate
an input file for each topology, as described in the Scalability Experiments
paragraph of Section 5 in the paper.
The following simply unpacks the pre-generated input files from the included
tarball inputs.tgz
:
make setup-inputs
To force our tool to re-build all the input files from the topologies, you can
run make rebuild-inputs
. However, we do NOT recommend doing this full
rebuild of the input files -- it takes about 35 minutes on our
experimental platform, and you MUST be on a VM/machine with
around 8+ GB RAM and Java JDK 1.8+ installed!
- Running the scalability experiments.
Once the scalability input files (inputs/*.input
and inputs/zoo/*.input
) are
present, we can run the synthesizer on all of them.
The following takes about 10 minutes on our experimental platform, and produces the data
file all.csv
:
make run-exp
We can then produce the graphs from this data:
make plots
This produces the following plots from the preprint paper:
scalability01-ft.pdf
corresponds to Figure 13,
scalability01-sw.pdf
corresponds to Figure 15(a), and
scalability01-zoo.pdf
corresponds to Figure 15(b).
It also produces the following plots from original (accepted) paper:
scalability01-lines-sw.pdf
corresponds to Figure 5(a),
and scalability01-lines-ft.pdf
corresponds to Figure 5(b).
Note that we are now using more/larger topologies than at the time
of paper submission, so these two graphs will look slightly different
than in the accepted paper.
- Cleaning up.
While the following functionality is not strictly necessary, it may be useful in certain cases.
To remove the currently-generated plots (CSV, PDF, log, dot files, etc.), you can use:
make tidy
To remove all generated content and return the build to the initial state, you can use:
make clean