summaryrefslogtreecommitdiff
path: root/tools/tracing/rtla/src/timerlat_hist.c
AgeCommit message (Collapse)Author
2024-05-16rtla: Fix -t\--trace[=file]John Kacur
The -t option has an optional argument. The usual case is for a short option to be specified without an '=' and for the long version to be specified with an '=' Various forms of this do not work as expected. For example: rtla timerlat hist -T50 -tfile.txt will result in a truncated file name of "ile.txt" Another example is that the long form without the '=' will result in the default file name instead of the requested file name. This patch properly parses the optional argument with and without '=' and with and without spaces for the short form. This patch was also tested using -t and --trace without providing a file name both as the last requested option and with a following long and short option. For example: rtla timerlat hist -T50 -t -u rtla timerlat hist -T50 --trace -u This fix is applied to both timerlat top and hist and to osnoise top and hist. Here is the full testing for rtla timerlat hist. Before applying the patch rtla timerlat hist -T50 -t=file.txt Works as expected, "file.txt" rtla timerlat hist -T50 -tfile.txt Truncated file name "ile.txt" rtla timerlat hist -T50 -t file.txt Default file name instead of file.txt rtla timerlat hist -T50 --trace=file.txt Truncated file name "ile.txt" rtla timerlat hist -T50 --trace file.txt Default file name "timerlat_trace.txt" instead of "file.txt" After applying the patch: rtla timerlat hist -T50 -t=file.txt Works as expected, "file.txt" rtla timerlat hist -T50 -tfile.txt Works as expected, "file.txt" rtla timerlat hist -T50 -t file.txt Works as expected, "file.txt" rtla timerlat hist -T50 --trace=file.txt Works as expected, "file.txt" rtla timerlat hist -T50 --trace file.txt Works as expected, "file.txt" In addition the following tests were performed to make sure that the default file name worked as expected including with trailing options. rtla timerlat hist -T50 -t Works as expected "timerlat_trace.txt" rtla timerlat hist -T50 --trace Works as expected "timerlat_trace.txt" rtla timerlat hist -T50 -t -u Works as expected "timerlat_trace.txt" rtla timerlat hist -T50 --trace -u Works as expected "timerlat_trace.txt" Link: https://lkml.kernel.org/r/20240515183024.59985-1-jkacur@redhat.com Cc: Daniel Bristot de Oliveria <bristot@kernel.org> Signed-off-by: John Kacur <jkacur@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
2024-05-16rtla/timerlat: Fix histogram report when a cpu count is 0John Kacur
On short runs it is possible to get no samples on a cpu, like this: # rtla timerlat hist -u -T50 Index IRQ-001 Thr-001 Usr-001 IRQ-002 Thr-002 Usr-002 2 1 0 0 0 0 0 33 0 1 0 0 0 0 36 0 0 1 0 0 0 49 0 0 0 1 0 0 52 0 0 0 0 1 0 over: 0 0 0 0 0 0 count: 1 1 1 1 1 0 min: 2 33 36 49 52 18446744073709551615 avg: 2 33 36 49 52 - max: 2 33 36 49 52 0 rtla timerlat hit stop tracing IRQ handler delay: (exit from idle) 48.21 us (91.09 %) IRQ latency: 49.11 us Timerlat IRQ duration: 2.17 us (4.09 %) Blocking thread: 1.01 us (1.90 %) swapper/2:0 1.01 us ------------------------------------------------------------------------ Thread latency: 52.93 us (100%) Max timerlat IRQ latency from idle: 49.11 us in cpu 2 Note, the value 18446744073709551615 is the same as ~0. Fix this by reporting no results for the min, avg and max if the count is 0. Link: https://lkml.kernel.org/r/20240510190318.44295-1-jkacur@redhat.com Cc: stable@vger.kernel.org Fixes: 1eeb6328e8b3 ("rtla/timerlat: Add timerlat hist mode") Suggested-by: Daniel Bristot de Oliveria <bristot@kernel.org> Signed-off-by: John Kacur <jkacur@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
2024-05-16rtla: Add --trace-buffer-size optionDaniel Bristot de Oliveira
Add the option allow the users to set a different buffer size for the trace. For example, in large systems, the user might be interested on reducing the trace buffer to avoid large tracing files. The buffer size is specified in kB, and it is only affecting the tracing instance. The function trace_set_buffer_size() appears on libtracefs v1.6, so increase the minimum required version on Makefile.config. Link: https://lkml.kernel.org/r/e7c9ca5b3865f28e131a49ec3b984fadf2d056c6.1715860611.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: John Kacur <jkacur@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
2024-05-15rtla/timerlat: Make user-space threads the defaultDaniel Bristot de Oliveira
After ther -u addition, most of the known users are setting it. And it makes sense, as it adds more information, and inherits the default setup for the threads - e.g., cgroups configs. Thus, if the user-space interface is available, enable -u. Otherwise, use the in-kernel thread. Add the -k option to allow the user to request kernel-threads. Link: https://lkml.kernel.org/r/9241d3089de4091b124f780ed832a0e6646cadaa.1713968967.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Cc: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
2024-05-15rtla: Add the --warm-up optionDaniel Bristot de Oliveira
On many cases, the results right after the startup are different from the rest of the execution, biasing the results. For example, on osnoise, the scheduler might take some time to adapt to the new busy-loop workload. Add the --warm-up <seconds> option, adding a warm-up phase (in seconds) where the workload is set, but the results are discarded. Link: https://lkml.kernel.org/r/e682d5ce5af90f123bd13220f63d5c3d118a92be.1713968967.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Cc: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
2024-05-15rtla/timerlat: Add a summary for hist modeDaniel Bristot de Oliveira
Like on rtla timerlat top, add an overall summary at the bottom of timerlat hist. For instance: # timerlat hist -c 0-1 -d 10s -E 20 # RTLA timerlat histogram # Time unit is microseconds (us) # Duration: 0 00:00:10 Index IRQ-000 Thr-000 IRQ-001 Thr-001 6 1 0 0 0 7 1 0 0 0 8 1 0 1 0 9 7 0 0 0 10 16 0 0 0 11 1 0 3 0 15 0 0 3 0 16 0 0 12 0 17 0 0 28 0 18 0 2 26 0 19 1 1 80 1 over: 9973 9998 9848 10000 count: 10001 10001 10001 10001 min: 6 18 8 19 avg: 185 204 95 113 max: 428 450 341 371 ALL: IRQ Thr count: 20002 20002 min: 6 18 avg: 140 159 max: 428 450 Link: https://lkml.kernel.org/r/a6bc06c798f72127edc57d1f99da8d57e1187cee.1713968967.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Suggested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
2024-03-20tools/rtla: Add -U/--user-load option to timerlatDaniel Bristot de Oliveira
The timerlat tracer provides an interface for any application to wait for the timerlat's periodic wakeup. Currently, rtla timerlat uses it to dispatch its user-space workload (-u option). But as the tracer interface is generic, rtla timerlat can also be used to monitor any workload that uses it. For example, a user might place their own workload to wait on the tracer interface, and monitor the results with rtla timerlat. Add the -U option to rtla timerlat top and hist. With this option, rtla timerlat will not dispatch its workload but only setting up the system, waiting for a user to dispatch its workload. The sample code in this patch is an example of python application that loops in the timerlat tracer fd. To use it, dispatch: # rtla timerlat -U In a terminal, then run the python program on another terminal, specifying the CPU to run it. For example, setting on CPU 1: #./timerlat_load.py 1 Then rtla timerlat will start printing the statistics of the ./timerlat_load.py app. An interesting point is that the "Ret user Timer Latency" value is the overall response time of the load. The sample load does a memory copy to exemplify that. The stop tracing options on rtla timerlat works in this setup as well, including auto analysis. Link: https://lkml.kernel.org/r/36e6bcf18fe15c7601048fd4c65aeb193c502cc8.1707229706.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Cc: Masami Hiramatsu <mhiramat@kernel.org> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
2024-02-12tools/rtla: Exit with EXIT_SUCCESS when help is invokedJohn Kacur
Fix rtla so that the following commands exit with 0 when help is invoked rtla osnoise top -h rtla osnoise hist -h rtla timerlat top -h rtla timerlat hist -h Link: https://lore.kernel.org/linux-trace-devel/20240203001607.69703-1-jkacur@redhat.com Cc: stable@vger.kernel.org Fixes: 1eeb6328e8b3 ("rtla/timerlat: Add timerlat hist mode") Signed-off-by: John Kacur <jkacur@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
2024-02-12tools/rtla: Fix uninitialized bucket/data->bucket_size warningDaniel Bristot de Oliveira
When compiling rtla with clang, I am getting the following warnings: $ make HOSTCC=clang CC=clang LLVM_IAS=1 [..] clang -O -g -DVERSION=\"6.8.0-rc3\" -flto=auto -fexceptions -fstack-protector-strong -fasynchronous-unwind-tables -fstack-clash-protection -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS $(pkg-config --cflags libtracefs) -c -o src/osnoise_hist.o src/osnoise_hist.c src/osnoise_hist.c:138:6: warning: variable 'bucket' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized] 138 | if (data->bucket_size) | ^~~~~~~~~~~~~~~~~ src/osnoise_hist.c:149:6: note: uninitialized use occurs here 149 | if (bucket < entries) | ^~~~~~ src/osnoise_hist.c:138:2: note: remove the 'if' if its condition is always true 138 | if (data->bucket_size) | ^~~~~~~~~~~~~~~~~~~~~~ 139 | bucket = duration / data->bucket_size; src/osnoise_hist.c:132:12: note: initialize the variable 'bucket' to silence this warning 132 | int bucket; | ^ | = 0 1 warning generated. [...] clang -O -g -DVERSION=\"6.8.0-rc3\" -flto=auto -fexceptions -fstack-protector-strong -fasynchronous-unwind-tables -fstack-clash-protection -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS $(pkg-config --cflags libtracefs) -c -o src/timerlat_hist.o src/timerlat_hist.c src/timerlat_hist.c:181:6: warning: variable 'bucket' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized] 181 | if (data->bucket_size) | ^~~~~~~~~~~~~~~~~ src/timerlat_hist.c:204:6: note: uninitialized use occurs here 204 | if (bucket < entries) | ^~~~~~ src/timerlat_hist.c:181:2: note: remove the 'if' if its condition is always true 181 | if (data->bucket_size) | ^~~~~~~~~~~~~~~~~~~~~~ 182 | bucket = latency / data->bucket_size; src/timerlat_hist.c:175:12: note: initialize the variable 'bucket' to silence this warning 175 | int bucket; | ^ | = 0 1 warning generated. This is a legit warning, but data->bucket_size is always > 0 (see timerlat_hist_parse_args()), so the if is not necessary. Remove the unneeded if (data->bucket_size) to avoid the warning. Link: https://lkml.kernel.org/r/6e1b1665cd99042ae705b3e0fc410858c4c42346.1707217097.git.bristot@kernel.org Cc: stable@vger.kernel.org Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nick Desaulniers <ndesaulniers@google.com> Cc: Bill Wendling <morbo@google.com> Cc: Justin Stitt <justinstitt@google.com> Cc: Donald Zickus <dzickus@redhat.com> Fixes: 1eeb6328e8b3 ("rtla/timerlat: Add timerlat hist mode") Fixes: 829a6c0b5698 ("rtla/osnoise: Add the hist mode") Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
2023-06-13rtla/timerlat_hist: Add timerlat user-space supportDaniel Bristot de Oliveira
Add the support for running timerlat threads in user-space. In this mode, enabled with -u/--user-threads, timerlat dispatches user-space processes that will loop in the timerlat_fd, measuring the overhead for going to user-space and then returning to the kernel - in addition to the existing measurements. Here is one example of the tool's output with -u enabled: $ sudo timerlat hist -u -c 1-3 -d 600 # RTLA timerlat histogram # Time unit is microseconds (us) # Duration: 0 00:10:01 Index IRQ-001 Thr-001 Usr-001 IRQ-002 Thr-002 Usr-002 IRQ-003 Thr-003 Usr-003 0 477555 0 0 425287 0 0 474357 0 0 1 122385 7998 0 174616 1921 0 125412 3138 0 2 47 587376 492150 89 594717 447830 147 593463 454872 3 11 2549 101930 7 2682 145580 64 2530 138680 4 3 1954 2833 1 463 4917 11 548 4656 5 0 60 1037 0 138 1117 6 179 1130 6 0 26 1837 0 38 277 1 76 339 7 0 15 143 0 28 147 2 37 156 8 0 10 23 0 11 75 0 12 80 9 0 7 17 0 0 26 0 11 42 10 0 2 11 0 0 18 0 2 20 11 0 0 7 0 1 8 0 2 12 12 0 0 6 0 1 4 0 2 8 13 0 1 3 0 0 0 0 0 1 14 0 1 0 0 0 1 0 0 2 15 0 1 0 0 0 0 0 0 2 16 0 1 2 0 0 0 0 0 0 17 0 0 1 0 0 0 0 0 0 19 0 0 1 0 0 0 0 0 0 over: 0 0 0 0 0 0 0 0 0 count: 600001 600001 600001 600000 600000 600000 600000 600000 600000 min: 0 1 2 0 1 2 0 1 2 avg: 0 1 2 0 2 2 0 2 2 max: 4 16 19 4 12 14 7 12 15 The tuning setup like -p or -C work for the user-space threads as well. Link: https://lkml.kernel.org/r/b6a042d55003c4a67ff7dce28d96044b7044f00d.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-06-13rtla: Start the tracers after creating all instancesDaniel Bristot de Oliveira
Group all start tracing after finishing creating all instances. The tracing instance starts first for the case of hitting a stop tracing while enabling other instances. The trace instance is the one with most valuable information. Link: https://lkml.kernel.org/r/67da7a703a56f75d7cd46568525145a65501a7e8.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-06-13rtla/timerlat_hist: Add auto-analysis supportDaniel Bristot de Oliveira
Add auto-analysis to timerlat hist, including the --no-aa option to reduce overhead and --dump-task. --aa-only was not added as it is already on timerlat top. Link: https://lkml.kernel.org/r/c2693f47ee83e659a7723fed8035f5d2534f528e.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-06-13rtla: Automatically move rtla to a house-keeping cpuDaniel Bristot de Oliveira
When the user sets -c <cpu-list> try to move rtla out of the <cpu-list>, even without an -H option. This is useful to avoid having rtla interfering with the workload. This works by removing <cpu-list> from rtla's current affinity. If rtla fails to move itself away it is not that of a problem as this is an automatic measure. Link: https://lkml.kernel.org/r/c54304d90c777310fb85a3e658d1449173759aab.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-06-13rtla: Change monitored_cpus from char * to cpu_set_tDaniel Bristot de Oliveira
Use a cpumask instead of a char *, reducing memory footprint and code. No functional change, and in preparation for auto house-keeping. Link: https://lkml.kernel.org/r/54c46293261d13cb1042d0314486539eeb45fe5d.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-06-13rtla: Add --house-keeping optionDaniel Bristot de Oliveira
To avoid having rtla interfering with the measurement threads, add an option for the user to set the CPUs in which rtla should run. For instance: # rtla timerlat top -H 0 -c 1-7 Will place rtla in the CPU 0, while running the measurement threads in the CPU 1-7. Link: https://lkml.kernel.org/r/6a6c78a579a96ba8b02ae67ee1e0ba2cb5e03c4a.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Suggested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-06-13rtla: Add -C cgroup supportDaniel Bristot de Oliveira
The -C option sets a cgroup to the tracer's threads. If the -C option is passed without arguments, the tracer's thread will inherit rtla's cgroup. Otherwise, the threads will be placed on the cgroup passed to the option. Link: https://lkml.kernel.org/r/cb051477331d292f17c08bf1d66f0e0384bbe5a5.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-08-10rtla: Fix tracer nameAlexandre Vicenzi
The correct tracer name is timerlat and not timelat. Link: https://lore.kernel.org/linux-trace-devel/20220808180343.22262-1-alexandre.vicenzi@suse.com Signed-off-by: Alexandre Vicenzi <alexandre.vicenzi@suse.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-05-26rtla: Avoid record NULL pointer dereferenceWan Jiabing
Fix the following null/deref_null.cocci errors: ./tools/tracing/rtla/src/osnoise_hist.c:870:31-36: ERROR: record is NULL but dereferenced. ./tools/tracing/rtla/src/osnoise_top.c:650:31-36: ERROR: record is NULL but dereferenced. ./tools/tracing/rtla/src/timerlat_hist.c:905:31-36: ERROR: record is NULL but dereferenced. ./tools/tracing/rtla/src/timerlat_top.c:700:31-36: ERROR: record is NULL but dereferenced. "record" is NULL before calling osnoise_init_trace_tool. Add a tag "out_free" to avoid dereferring a NULL pointer. Link: https://lkml.kernel.org/r/ae0e4500d383db0884eb2820286afe34ca303778.1651247710.git.bristot@kernel.org Link: https://lore.kernel.org/r/20220408151406.34823-1-wanjiabing@vivo.com/ Cc: kael_w@yeah.net Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Fixes: 51d64c3a1819 ("rtla: Add -e/--event support") Acked-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Wan Jiabing <wanjiabing@vivo.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-03-15rtla: Tools main loop cleanupDaniel Bristot de Oliveira
I probably started using "do {} while();", but changed all but osnoise_top to "while(){};" leaving the ; behind. Cleanup the main loop code, making all tools use "while() {}" Changcheng Deng reported this problem, as reported by coccicheck: Fix the following coccicheck review: ./tools/tracing/rtla/src/timerlat_hist.c: 800: 2-3: Unneeded semicolon ./tools/tracing/rtla/src/osnoise_hist.c: 776: 2-3: Unneeded semicolon ./tools/tracing/rtla/src/timerlat_top.c: 596: 2-3: Unneeded semicolon Link: https://lkml.kernel.org/r/3c1642110aa87c396f5da4a037dabc72dbb9c601.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Reported-by: Changcheng Deng <deng.changcheng@zte.com.cn> Reported-by: Zeal Robot <zealci@zte.com.cn> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-03-15rtla/timerlat: Add --dma-latency optionDaniel Bristot de Oliveira
Add the --dma-latency to set /dev/cpu_dma_latency to the specified value, this aims to avoid having exit from idle states latencies that could be influencing the analysis. Link: https://lkml.kernel.org/r/72ddb0d913459f13217086dadafad88a7c46dd28.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-03-15rtla: Check for trace off also in the trace instanceDaniel Bristot de Oliveira
With the addition of --trigger option, it is also possible to stop the trace from the -t tracing instance using the traceoff trigger. Make rtla tools to check if the trace is stopped also in the trace instance, stopping the execution of the tool. Link: https://lkml.kernel.org/r/59fc7c6f23dddd5c8b7ef1782cf3da51ea2ce0f5.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-03-15rtla: Add --filter supportDaniel Bristot de Oliveira
Add --filter option. This option enables a trace event filtering of the previous -e sys:event argument. This option is available for all current tools. Link: https://lkml.kernel.org/r/509d70b6348d3e5bcbf1f07ab725ce08d063149a.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-03-15rtla: Add --trigger supportDaniel Bristot de Oliveira
Add --trigger option. This option enables a trace event trigger to the previous -e sys:event argument, allowing some advanced tracing options. For instance, in a system with CPUs 2:23 isolated, it is possible to get a stack trace of thread wakeup targeting those CPUs while running osnoise with the following command line: # osnoise top -c 2-23 -a 50 -e sched:sched_wakeup --trigger="stacktrace if target_cpu >= 2" This option is available for all current tools. Link: https://lkml.kernel.org/r/07d2983d5f71261d4da89dbaf02efcad100ab8ee.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-03-15rtla: Add -e/--event supportDaniel Bristot de Oliveira
Add -e/--event option. This option enables an event in the trace (-t) session. The argument can be a specific event, e.g., -e sched:sched_switch, or all events of a system group, e.g., -e sched. Multiple -e are allowed. It is only active when -t or -a are set. This option is available for all current tools. Link: https://lkml.kernel.org/r/6a3b753be9b1e811953995f7f21a86918ad13390.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-03-15rtla/timerlat: Add the automatic trace optionDaniel Bristot de Oliveira
Add the -a/--auto <arg in us> option. This option sets some commonly used options while debugging the system. It aims to help users produce reports in the field, reducing the number of arguments passed to the tool in the first approach to a problem. It is equivalent to setting osnoise/stop_tracing_total_us and print_stack with the argument, and saving the trace to timerlat_trace.txt file if the trace is stopped automatically. Link: https://lkml.kernel.org/r/92438f7ef132c731f538cebdf77850300afe04a5.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-02-25rtla/hist: Make -E the short version of --entriesDaniel Bristot de Oliveira
Currently, --entries uses -e as the short version in the hist mode of timerlat and osnoise tools. But as -e is already used to enable events on trace sessions by other tools, thus let's keep it available for the same usage for all rtla tools. Make -E the short version of --entries for hist mode on all tools. Note: rtla was merged in this merge window, so rtla was not released yet. Link: https://lkml.kernel.org/r/5dbf0cbe7364d3a05e708926b41a097c59a02b1e.1645206561.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-02-08rtla: Fix segmentation fault when failing to enable -tDaniel Bristot de Oliveira
rtla osnoise and timerlat are causing a segmentation fault when running with the --trace option on a kernel that does not support multiple instances. For example: [root@f34 rtla]# rtla osnoise top -t failed to enable the tracer osnoise Could not enable osnoiser tracer for tracing Failed to enable the trace instance Segmentation fault (core dumped) This error happens because the exit code of the tools is trying to destroy the trace instance that failed to be created. Make osnoise_destroy_tool() aware of possible NULL osnoise_tool *, and do not attempt to destroy it. This also simplifies the exit code. Link: https://lkml.kernel.org/r/5660a2b6bf66c2655842360f2d7f6b48db5dba23.1644327249.git.bristot@kernel.org Suggested-by: Steven Rostedt <rostedt@goodmis.org> Fixes: 1eceb2fc2ca5 ("rtla/osnoise: Add osnoise top mode") Fixes: 829a6c0b5698 ("rtla/osnoise: Add the hist mode") Fixes: a828cd18bc4a ("rtla: Add timerlat tool and timelart top mode") Fixes: 1eeb6328e8b3 ("rtla/timerlat: Add timerlat hist mode") Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-01-13rtla/timerlat: Add timerlat hist modeDaniel Bristot de Oliveira
The rtla hist hist mode displays a histogram of each tracer event occurrence, both for IRQ and timer latencies. The tool also allows many configurations of the timerlat tracer and the collection of the tracer output. Here is one example of the rtla timerlat hist mode output: ---------- %< ---------- [root@alien ~]# rtla timerlat hist -c 0-3 -d 1M # RTLA timerlat histogram # Time unit is microseconds (us) # Duration: 0 00:01:00 Index IRQ-000 Thr-000 IRQ-001 Thr-001 IRQ-002 Thr-002 IRQ-003 Thr-003 0 58572 0 59373 0 58691 0 58895 0 1 1422 57021 628 57241 1310 56160 1102 56805 2 6 2931 0 2695 0 3567 4 3031 3 1 40 0 53 0 260 0 142 4 0 7 0 5 0 6 0 17 5 0 2 0 5 0 7 0 4 6 0 0 0 2 0 1 0 1 8 0 0 0 0 0 0 0 1 over: 0 0 0 0 0 0 0 0 count: 60001 60001 60001 60001 60001 60001 60001 60001 min: 0 1 0 1 0 1 0 1 avg: 0 1 0 1 0 1 0 1 max: 3 5 1 6 1 6 2 8 ---------- >% ---------- Running - rtla timerlat hist --help provides information about the available options. Link: https://lkml.kernel.org/r/7049ed3c46b7d6aceab18ffe7770003dfc4ddceb.1639158831.git.bristot@kernel.org Cc: Tao Zhou <tao.zhou@linux.dev> Cc: Ingo Molnar <mingo@redhat.com> Cc: Tom Zanussi <zanussi@kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Clark Williams <williams@redhat.com> Cc: John Kacur <jkacur@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: linux-rt-users@vger.kernel.org Cc: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>