User Tools

Trace:

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

benchmark [2016/04/13 18:02]
 benchmark [2016/04/13 18:02] (current)
Line 1: Line 1:
 +===== SEQPower Benchmark =====
 +''​spower execute''​ command executes a given configuration file with parameters set to allow for power analysis under multiple scenarios within a single command. The feature also provides us a shortcut to benchmark performance of different rare variant association methods under various parameter settings.
  
 +==== Prepare a Configuration File ====
 +<hidden initialState="​visible"​ -noprint>​
 +<code text spower_benchmark.conf>​
 +[model]
 +model=LOGIT;​LOGIT,​PAR,​LNR
 +type=power
 +[input and output]
 +title=boyko1
 +input=boyko1.gdat
 +resampling=False
 +output=PowerBenchmark
 +[runtime options]
 +verbosity=2
 +seed=0
 +jobs=8
 +[sample population]
 +sample_size=2000;​600,​1000,​2000,​5000,​10000
 +p1=0.5;​0.1,​0.3,​0.5,​0.7,​0.9
 +[power calculation]
 +power=0.8
 +replicates=1000
 +alpha=0.05
 +methods="​CFisher --name CMC" "KBAC -p 2000 --adaptive 0.1" "​WSSRankTest -p 2000 --adaptive 0.1 --name WSS"
 +[genotyping artifact]
 +missing_low_maf=None
 +missing_sites=None;​0.05,​0.2,​0.4,​0.6,​0.8
 +missing_sites_deleterious=None;​0.05,​0.2,​0.4,​0.6,​0.8
 +missing_sites_protective=None;​0.05,​0.2,​0.4,​0.6,​0.8
 +missing_sites_neutral=None;​0.05,​0.2,​0.4,​0.6,​0.8
 +missing_sites_synonymous=None;​0.05,​0.2,​0.4,​0.6,​0.8
 +missing_calls=None;​0.05,​0.2,​0.4,​0.6,​0.8
 +missing_calls_deleterious=None;​0.05,​0.2,​0.4,​0.6,​0.8
 +missing_calls_protective=None;​0.05,​0.2,​0.4,​0.6,​0.8
 +missing_calls_neutral=None;​0.05,​0.2,​0.4,​0.6,​0.8
 +missing_calls_synonymous=None;​0.05,​0.2,​0.4,​0.6,​0.8
 +error_calls=0.01
 +error_calls_deleterious=None
 +error_calls_protective=None
 +error_calls_neutral=None
 +error_calls_synonymous=None
 +[quality control]
 +def_valid_locus=None
 +rare_only=False
 +missing_as_wt=True
 +discard_samples=None
 +discard_variants=None
 +[variants functionality]
 +def_rare=0.01
 +def_neutral=-0.00001 0.00001
 +def_protective=-1 -0.00001
 +proportion_detrimental=None;​None,​0.8,​0.6,​0.4,​0.2
 +proportion_protective=None;​0,​0.25,​0.5,​0.75,​None
 +[phenotype associations]
 +moi=A;​A,​D,​R,​M
 +OR_rare_detrimental=1.5;​1.0,​1.2,​1.8,​2.0,​2.5,​3.0
 +OR_rare_protective=0.5;​0.9,​0.7,​0.5,​0.3,​0.1
 +ORmax_rare_detrimental=None;​2.0,​2.5,​3.0,​3.5,​4.0
 +ORmin_rare_protective=None;​0.95,​0.85,​0.75,​0.65,​0.55
 +OR_common_detrimental=1.0
 +OR_common_protective=1.0
 +baseline_effect=0.01
 +PAR_rare_detrimental=0.0;​0.005,​0.01,​0.015,​0.025,​0.05
 +PAR_rare_protective=0.0
 +PAR_common_detrimental=0.0
 +PAR_common_protective=0.0
 +PAR_variable=False
 +meanshift_rare_detrimental=0.5
 +meanshift_rare_protective=-0.5
 +meanshiftmax_rare_detrimental=None
 +meanshiftmax_rare_protective=None
 +meanshift_common_detrimental=0.0
 +meanshift_common_protective=0.0
 +QT_thresholds=0.5 0.5;0.1 0.9,0.2 0.8,0.3 0.7,0.4 0.6,0.5 0.5
 +[plot]
 +x_axis=sample_size
 +x_axis_alias=
 +x_axis_fontsize=
 +y_axis=power
 +y_axis_range=
 +y_axis_fontsize=
 +stderr=power_std
 +object=method
 +object_alias=
 +object_color=
 +object_shape={{'​CMC'​ : 1, '​BRV'​ : 3, '​WSS'​ : 5, '​KBAC'​ : 6, '​VT'​ : 7, '​c-alpha'​ : 10, '​RareCover'​ : 11, '​SKAT'​ : 15}}
 +object_line={{'​CMC'​ : 1, '​BRV'​ : 6, '​WSS'​ : 1, '​KBAC'​ : 6, '​VT'​ : 1, '​c-alpha'​ : 6, '​RareCover'​ : 1, '​SKAT'​ : 6}}
 +plot_title=
 +plot_width=
 +plot_height=
 +plot_fn=
 +xlab=
 +xlab_fontsize=
 +ylab=
 +ylab_fontsize=
 +stderrlab=
 +objectlab=
 +condition=
 +</​code>​
 +</​hidden>​\\
 +==== Usage ====
 +<code bash>
 +spower execute -h
 +</​code>​\\
 +<hidden initialState="​visible"​ -noprint>​
 +  usage: spower execute [-h] [-s [ARG [ARG ...]]] [-f [ARG [ARG ...]]] [--plot]
 +                        [--dry_run]
 +                        file
 +  ​
 +  positional arguments:
 +    file                  configuration filename to execute
 +  ​
 +  optional arguments:
 +    -h, --help ​           show this help message and exit
 +    -s [ARG [ARG ...]], --sliding [ARG [ARG ...]]
 +                          specify variable parameters
 +    -f [ARG [ARG ...]], --fixed [ARG [ARG ...]]
 +                          specify fixed parameters
 +    --plot ​               generate plot instead of running simulations
 +    --dry_run ​            print generated commands to screen instead of
 +                          executing them
 +</​hidden>​\\
 +=== Execute benchmark ===
 +<code bash>
 +spower execute spower-benchmark.conf --sliding sample_size=2000,​3000,​4000,​5000,​6000 \
 +       ​--fixed model=LOGIT input=KIT.gdat p1=0.5
 +</​code>​\\
 +<hidden initialState="​visible"​ -noprint>​
 +  Running command 1/5 ...
 +  INFO: Loading data from [KIT.gdat] ...
 +  INFO: 1 units found
 +  KIT : 100% |======================================================================================| Time: 0:03:06
 +  INFO: Tuning [PowerBenchmark.SEQPowerDB] ...
 +  INFO: Result saved to [PowerBenchmark.csv/​.loci.csv/​.SEQPowerDB]
 +  Running command 2/5 ...
 +  INFO: Loading data from [KIT.gdat] ...
 +  INFO: 1 units found
 +  KIT : 100% |======================================================================================| Time: 0:05:08
 +  INFO: Tuning [PowerBenchmark.SEQPowerDB] ...
 +  INFO: Result saved to [PowerBenchmark.csv/​.loci.csv/​.SEQPowerDB]
 +  Running command 3/5 ...
 +  INFO: Loading data from [KIT.gdat] ...
 +  INFO: 1 units found
 +  KIT : 100% |======================================================================================| Time: 0:07:32
 +  INFO: Tuning [PowerBenchmark.SEQPowerDB] ...
 +  INFO: Result saved to [PowerBenchmark.csv/​.loci.csv/​.SEQPowerDB]
 +  Running command 4/5 ...
 +  INFO: Loading data from [KIT.gdat] ...
 +  INFO: 1 units found
 +  KIT : 100% |======================================================================================| Time: 0:10:06
 +  INFO: Tuning [PowerBenchmark.SEQPowerDB] ...
 +  INFO: Result saved to [PowerBenchmark.csv/​.loci.csv/​.SEQPowerDB]
 +  Running command 5/5 ...
 +  INFO: Loading data from [KIT.gdat] ...
 +  INFO: 1 units found
 +  KIT : 100% |======================================================================================| Time: 0:14:09
 +  INFO: Tuning [PowerBenchmark.SEQPowerDB] ...
 +  INFO: Result saved to [PowerBenchmark.csv/​.loci.csv/​.SEQPowerDB]
 +</​hidden>​\\
 +=== Generate plot ===
 +<code bash>
 +spower execute spower-benchmark.conf --plot
 +</​code>​\\
 +{{:​plot-example.png?​500.0}}

Page Tools

SEQPower

Site Tools