Skip to content

System monitor#

You can monitor the CPU, memory, disk, and network usage during the execution of experiments. The tracking is done transparently in a thread, adding the stats to the run field sysmon.

The monitor operates at the run level and also works in distributed environments.

Tip

The statistics are streamed if data streaming is enabled.

Example: Monitoring the system#

Monitoring the system

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
from mltraq import create_experiment, options
from mltraq.steps.sleep import sleep

# Activate the system monitor, specifying the frequency to sample the stats
with options().ctx({"sysmon.disable": False, "sysmon.interval": 0.1}):

    # Create a new experiment
    experiment = create_experiment()

    # Wait enough time to track a few timestamps, and report the first row
    print(
        experiment.execute(sleep(0.3))
        .runs.first()
        .fields.sysmon.df()
        .iloc[0]
    )
Output
idx                                       0
timestamp        2024-04-11 16:44:41.068206
cpu_pct                                43.7
disk_free_gb                          46.13
disk_total_gb                        494.38
disk_used_pct                         90.67
mem_free_gb                           10.32
mem_total_gb                          25.77
mem_used_pct                           60.0
net_recv_kbs                          10.24
net_sent_kbs                          20.48
Name: 0, dtype: object