Reading and writing CSV files
This page discusses features available in Spot's command-line tools to produce an consume CSV files.
Table of Contents
Producing CSV files
All the tools that normally produce formulas (like genltl
, randltl
,
and ltlfilt
) have a –format
option. That can be used to
customize the way output is formatted.
For instance here is how we could use genltl
to generate a CSV file
with three columns: the family name of the formula, its parameter, and
the formula itself.
genltl --and-gf=1..5 --u-left=1..5 --format='%F,%L,%f' > gen.csv
cat gen.csv
and-gf,1,GFp1 and-gf,2,GFp1 & GFp2 and-gf,3,GFp1 & GFp2 & GFp3 and-gf,4,GFp1 & GFp2 & GFp3 & GFp4 and-gf,5,GFp1 & GFp2 & GFp3 & GFp4 & GFp5 u-left,1,p1 u-left,2,p1 U p2 u-left,3,(p1 U p2) U p3 u-left,4,((p1 U p2) U p3) U p4 u-left,5,(((p1 U p2) U p3) U p4) U p5
Tools that produce automata (like ltl2tgba
, or dstar2tgba
) have a
--stats
option that can be used to output various statistics about
the constructed automaton (these statistics are shown instead of
printing the automaton).
For instance, the following command will translate all the previous
formulas, and show the resulting number of states (%s
) and edges
(%e
) of the automaton constructed for each formula.
genltl --and-gf=1..5 --u-left=1..5 | ltl2tgba -F- --stats '%f,%s,%e'
GFp1,1,2 G(Fp1 & Fp2),1,4 G(Fp1 & Fp2 & Fp3),1,8 G(Fp1 & Fp2 & Fp3 & Fp4),1,16 G(Fp1 & Fp2 & Fp3 & Fp4 & Fp5),1,32 p1,2,2 p1 U p2,2,3 (p1 U p2) U p3,4,10 ((p1 U p2) U p3) U p4,8,34 (((p1 U p2) U p3) U p4) U p5,16,116
If the translated formulas may contain commas, or double-quotes, this simple output may prove difficult to process by other tools. For instance consider the translation of the following two formulas:
ltl2tgba -f Xa -f 'G("switch == on" -> F"tab[3,5] < 12")' --stats '%f,%s,%e'
Xa,3,3 G(!"switch == on" | F"tab[3,5] < 12"),2,4
The second line of this input does no conform to RFC 4180 because
non-escaped fields are not allowed to contain comma or double-quotes.
To fix this, use ltl2tgba
's --csv-escape
option: this causes
"%f
" to produce a double-quoted string properly escaped.
ltl2tgba -f Xa -f 'G("switch == on" -> F"tab[3,5] < 12")' --stats '%f,%s,%e' --csv-escape
"Xa",3,3 "G(!""switch == on"" | F""tab[3,5] < 12"")",2,4
The tool ltlcross
has its own --csv=FILENAME
option to format the
statistics it gathers in a CSV file, but you have very little control
hover how this CSV file is formatted (it can only be changed
via option such as --products
or --omit-missing
).
Reading CSV files
All the tools that read formulas from files extend the filename syntax
to support the specification of a CSV column. The notation
filename/COL
denotes the column COL
of that file.
For instance let's consider the file gen.csv
built with the first command of
this page. It contains:
and-gf,1,GFp1 and-gf,2,GFp1 & GFp2 and-gf,3,GFp1 & GFp2 & GFp3 and-gf,4,GFp1 & GFp2 & GFp3 & GFp4 and-gf,5,GFp1 & GFp2 & GFp3 & GFp4 & GFp5 u-left,1,p1 u-left,2,p1 U p2 u-left,3,(p1 U p2) U p3 u-left,4,((p1 U p2) U p3) U p4 u-left,5,(((p1 U p2) U p3) U p4) U p5
We can run ltl2tgba
on the third column to produce
the same output as in a previous example:
ltl2tgba -F gen.csv/3 --stats '%f,%s,%e'
GFp1,1,2 G(Fp1 & Fp2),1,4 G(Fp1 & Fp2 & Fp3),1,8 G(Fp1 & Fp2 & Fp3 & Fp4),1,16 G(Fp1 & Fp2 & Fp3 & Fp4 & Fp5),1,32 p1,2,2 p1 U p2,2,3 (p1 U p2) U p3,4,10 ((p1 U p2) U p3) U p4,8,34 (((p1 U p2) U p3) U p4) U p5,16,116
When ltlfilt
is used on a CSV file, it will preserve the
text before and after the matched formula in the CSV file.
For instance:
ltlfilt -F gen.csv/3 --size-min=8 --relabel=abc
and-gf,3,GFa & GFb & GFc and-gf,4,GFa & GFb & GFc & GFd and-gf,5,GFa & GFb & GFc & GFd & GFe u-left,5,(((a U b) U c) U d) U e
For security, in case a formula may contain double-quotes or
commas, you should use the --csv-escape
option:
ltlfilt -F gen.csv/3 --size-min=8 --relabel=abc --csv-escape
and-gf,3,"GFa & GFb & GFc" and-gf,4,"GFa & GFb & GFc & GFd" and-gf,5,"GFa & GFb & GFc & GFd & GFe" u-left,5,"(((a U b) U c) U d) U e"
The preservation in the output of the text before and after the
selected column can be altered using the --format
option. The %<
escape sequence represent the (comma-separated) data of all the
columns before the selected column, and %>
is the same for the
trailing data. Note that the comma that separate formulas' column
from the other column are excluded and should be added in the format
string.
For instance this moves the first two columns after the formulas.
ltlfilt -F gen.csv/3 --size-min=8 --csv-escape --format='%f,%<'
"GFp1 & GFp2 & GFp3",and-gf,3 "GFp1 & GFp2 & GFp3 & GFp4",and-gf,4 "GFp1 & GFp2 & GFp3 & GFp4 & GFp5",and-gf,5 "(((p1 U p2) U p3) U p4) U p5",u-left,5
Typical uses of ltlfilt
on CSV file include:
- Filtering lines based on an LTL criterion, as above.
- Changing the syntax of LTL formulas. For instance
ltl2tgba
's--stats
option, andltlcross
's--csv
option always output formulas in Spot's format. If that is inappropriate, simply useltlfilt
to rewrite the relevant column in your prefered syntax.
Dealing with header lines
Some CSV contain a header lines that should not be processed.
The CSV file produced by ltlcross
have such a line:
randltl -n 2 a b | ltlfilt --remove-wm | ltlcross --csv=results.csv 'ltl2tgba -s %f >%N' 'ltl3ba -f %s >%N' cat results.csv
"formula","tool","exit_status","exit_code","time","states","edges","transitions","acc","scc","nonacc_scc","terminal_scc","weak_scc","strong_scc","nondet_states","nondet_aut","terminal_aut","weak_aut","strong_aut","product_states","product_transitions","product_scc" "(1)","ltl2tgba -s %f >%N","ok",0,0.025727,1,1,1,1,1,0,1,0,0,0,0,1,0,0,200,3994,1 "(1)","ltl3ba -f %s >%N","ok",0,0.0733247,1,1,1,1,1,0,1,0,0,0,0,1,0,0,200,3994,1 "(0)","ltl2tgba -s %f >%N","ok",0,0.0338184,1,1,0,1,1,1,0,0,0,0,0,1,0,0,1,0,1 "(0)","ltl3ba -f %s >%N","ok",0,0.00404407,1,0,0,1,1,1,0,0,0,0,0,1,0,0,1,0,1 "(!(G((F(b)) | (F(!((b) | (G(b))))))))","ltl2tgba -s %f >%N","ok",0,0.030958,1,1,0,1,1,1,0,0,0,0,0,1,0,0,1,0,1 "(!(G((F(b)) | (F(!((b) | (G(b))))))))","ltl3ba -f %s >%N","ok",0,0.00386703,1,0,0,1,1,1,0,0,0,0,0,1,0,0,1,0,1 "(G((F(b)) | (F(!((b) | (G(b)))))))","ltl2tgba -s %f >%N","ok",0,0.0260381,1,1,1,1,1,0,1,0,0,0,0,1,0,0,200,4083,1 "(G((F(b)) | (F(!((b) | (G(b)))))))","ltl3ba -f %s >%N","ok",0,0.00406728,1,1,1,1,1,0,1,0,0,0,0,1,0,0,200,4083,1
If we run ltlfilt
on the first column, it will process the formula
header as if it was an LTL formula.
ltlfilt -F results.csv/1 --format='%f' --unique
formula 1 0 !G(Fb | F!(b | Gb)) G(Fb | F!(b | Gb))
In such case, the syntax FILENAME/-COL
(with a minus sign before the
column number) can be used to discard the first line of a CSV file.
ltlfilt -F results.csv/-1 --format='%f' --unique
1 0 !G(Fb | F!(b | Gb)) G(Fb | F!(b | Gb))