Skip to content

Use Cases

1. Minimizing runtime of a workload

It is possible to directly optimize the duration of a workload by issuing the following:

$ optimizer-studio ./workload.sh 
Concertio Optimizer Studio version 1.3
Optimizing ./workload.sh 
Optimization target: duration
Starting runtime (no time limit) ...

Optimization time: 01:26:27
Progress: 100%
Best known knob settings so far:
cpu.cache_prefetch: c
sys.vm.overcommit_ratio: 60
dev.raid.speed_limit_max: 250000
kernel.sched_wakeup_granularity_ns: 10000000

In this example, Optimizer Studio will measure the time it takes for ./workload.sh to complete and will attempt to minimize it by changing knob values and re-running. When no knobs.yaml file is specified, the default behavior of Optimizer Studio is to use its embedded knobs.

2. Maximizing reported benchmark performance

It is common to have a workload script run a benchmark and then report its performance. For example, running Phoronix's pts/apache benchmark reports the average number of web requests served per second. It is possible to directly maximize reported benchmark performance using Optimizer Studio. First, the benchmark performance needs to be extracted from ./workload.sh, for example:

$ cat workload.sh
#!/bin/bash
printf "n\n" | \
  phoronix-test-suite run pts/apache | \
  awk '/Average/{ print $2 }' | \
  sed "s,\x1B\[[0-9;]*[a-zA-Z],,g" > /tmp/result

The above will output the number of web requests into /tmp/result. We'll use this output in a custom knobs.yaml:

domain:
  common:
    metrics:
      shell.my_target:
        sample_script: "echo -n $(cat /tmp/result)"
        aggregated: false
    include_metrics: [shell.my_target]
    plugins: [libshellplugin.so]
    target: shell.my_target:max

Optimizer Studio could then be launched as follows:

$ optimizer-studio --knobs=embedded --knobs=knobs.yaml ./workload.sh 

The order of the knob definition files is significant

Optimizer Studio loads knob definition files in the order they appear in the command line. However this doesn't include embedded knobs which are always processed first. This way your knob files have an opportunity to overload any embedded knobs settings.

3. Maximizing performance using asynchronous sampling

Optimizer Studio can improve the performance of applications by maximizing intermediate performance metrics, such as the current rate of retired instructions, the current rate of database queries, or other metrics that describe performance. In this scenario, Optimizer Studio will sample the metrics asynchronously and in parallel to the workload script. This can be accomplished by using an additional knobs.yaml file that contains a different target:

domain:
  common:
    target: performance:max

The above example knobs.yaml file will maximize the number of retired instructions per second performed by the CPU. Then, Optimizer Studio can be called as follows:

$ optimizer-studio --knobs=embedded --knobs=knobs.yaml --sampling-mode=async ./workload.sh 

In this example, the assumption is that the workload is simple and has only one phase of execution.

4. Optimizing a remote system

Optimizer Studio can optimize remote systems by using ssh. This is useful for optimizing BIOS settings on servers, optimizing non-x86 servers, or even optimizing network switches. This can be accomplished as follows:

  • In the workload script, run the workload on the remote device via ssh. Store the result on the local server running Optimizer Studio. The following example shows a workload script named workload.sh that calls a remote script named remote_workload.sh that resides on the remote device:
#!/bin/bash
ssh -i /home/ubuntu/.ssh/id_rsa ubuntu@52.91.47.2 \
    "./remote_workload.sh" > /tmp/result
  • Add a knobs.yaml file in which all knobs' set and get scripts use ssh to set/get knob values on the remote host. For example:
domain:
  common:
    knobs:
      my_remote_knob:
        options: [1, 2, 3]
        get_script: |
          ssh -i /home/ubuntu/.ssh/id_rsa \
              ubuntu@52.91.47.2 "cat /tmp/knob"
        set_script: |
          ssh -i /home/ubuntu/.ssh/id_rsa \
              ubuntu@52.91.47.2 "echo $KNOB_VAL > /tmp/knob"
    metrics:
      shell.my_target:
        sample_script: "echo -n $(cat /tmp/result)"
        aggregated: false
    include_metrics: [shell.my_target]
    plugins: [libshellplugin.so]
    target: shell.my_target:max
  • Don't use the embedded knobs of Optimizer Studio as they will run on the local server and not on the remote device:
sudo optimizer-studio --knobs=knobs.yaml ./workload.sh 

5. Other use cases

Under /opt/concertio-optimizer/studio/examples/ there are additional examples:

$ ls
a_plus_b  
embedded-gcc  
evolution_synthetic  
hackbench_local  
hackbench_remote  
noisy_async  
phoronix  
sum_knob_array  
sum_knob_assoc_array  
workload_in_yaml