summaryrefslogtreecommitdiff
path: root/tools/perf/util/parse-events.c
AgeCommit message (Collapse)Author
2015-10-29perf tools: Compile scriptlets to BPF objects when passing '.c' to --eventWang Nan
This patch provides infrastructure for passing source files to --event directly using: # perf record --event bpf-file.c command This patch does following works: 1) Allow passing '.c' file to '--event'. parse_events_load_bpf() is expanded to allow caller tell it whether the passed file is source file or object. 2) llvm__compile_bpf() is called to compile the '.c' file, the result is saved into memory. Use bpf_object__open_buffer() to load the in-memory object. Introduces a bpf-script-example.c so we can manually test it: # perf record --clang-opt "-DLINUX_VERSION_CODE=0x40200" --event ./bpf-script-example.c sleep 1 Note that '--clang-opt' must put before '--event'. Futher patches will merge it into a testcase so can be tested automatically. Signed-off-by: Wang Nan <wangnan0@huawei.com> Acked-by: Alexei Starovoitov <ast@plumgrid.com> Cc: Brendan Gregg <brendan.d.gregg@gmail.com> Cc: Daniel Borkmann <daniel@iogearbox.net> Cc: David Ahern <dsahern@gmail.com> Cc: He Kuang <hekuang@huawei.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Kaixu Xia <xiakaixu@huawei.com> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Zefan Li <lizefan@huawei.com> Cc: pi3orama@163.com Link: http://lkml.kernel.org/r/1444826502-49291-10-git-send-email-wangnan0@huawei.com Signed-off-by: He Kuang <hekuang@huawei.com> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-10-29perf bpf: Attach eBPF filter to perf eventWang Nan
This is the final patch which makes basic BPF filter work. After applying this patch, users are allowed to use BPF filter like: # perf record --event ./hello_world.o ls A bpf_fd field is appended to 'struct evsel', and setup during the callback function add_bpf_event() for each 'probe_trace_event'. PERF_EVENT_IOC_SET_BPF ioctl is used to attach eBPF program to a newly created perf event. The file descriptor of the eBPF program is passed to perf record using previous patches, and stored into evsel->bpf_fd. It is possible that different perf event are created for one kprobe events for different CPUs. In this case, when trying to call the ioctl, EEXIST will be return. This patch doesn't treat it as an error. Committer note: The bpf proggie used so far: __attribute__((section("fork=_do_fork"), used)) int fork(void *ctx) { return 0; } char _license[] __attribute__((section("license"), used)) = "GPL"; int _version __attribute__((section("version"), used)) = 0x40300; failed to produce any samples, even with forks happening and it being running in system wide mode. That is because now the filter is being associated, and the code above always returns zero, meaning that all forks will be probed but filtered away ;-/ Change it to 'return 1;' instead and after that: # trace --no-syscalls --event /tmp/foo.o 0.000 perf_bpf_probe:fork:(ffffffff8109be30)) 2.333 perf_bpf_probe:fork:(ffffffff8109be30)) 3.725 perf_bpf_probe:fork:(ffffffff8109be30)) 4.550 perf_bpf_probe:fork:(ffffffff8109be30)) ^C# And it works with all tools, including 'perf trace'. Signed-off-by: Wang Nan <wangnan0@huawei.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Alexei Starovoitov <ast@plumgrid.com> Cc: Brendan Gregg <brendan.d.gregg@gmail.com> Cc: Daniel Borkmann <daniel@iogearbox.net> Cc: David Ahern <dsahern@gmail.com> Cc: He Kuang <hekuang@huawei.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Kaixu Xia <xiakaixu@huawei.com> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Zefan Li <lizefan@huawei.com> Cc: pi3orama@163.com Link: http://lkml.kernel.org/r/1444826502-49291-8-git-send-email-wangnan0@huawei.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-10-28perf bpf: Collect perf_evsel in BPF object filesWang Nan
This patch creates a 'struct perf_evsel' for every probe in a BPF object file(s) and fills 'struct evlist' with them. The previously introduced dummy event is now removed. After this patch, the following command: # perf record --event filter.o ls Can trace on each of the probes defined in filter.o. The core of this patch is bpf__foreach_tev(), which calls a callback function for each 'struct probe_trace_event' event for a bpf program with each associated file descriptors. The add_bpf_event() callback creates evsels by calling parse_events_add_tracepoint(). Since bpf-loader.c will not be built if libbpf is turned off, an empty bpf__foreach_tev() is defined in bpf-loader.h to avoid build errors. Committer notes: Before: # /tmp/oldperf record --event /tmp/foo.o -a usleep 1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.198 MB perf.data ] # perf evlist /tmp/foo.o # perf evlist -v /tmp/foo.o: type: 1, size: 112, config: 0x9, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|CPU|PERIOD, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, task: 1, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1 I.e. we create just the PERF_TYPE_SOFTWARE (type: 1), PERF_COUNT_SW_DUMMY(config 0x9) event, now, with this patch: # perf record --event /tmp/foo.o -a usleep 1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.210 MB perf.data ] # perf evlist -v perf_bpf_probe:fork: type: 2, size: 112, config: 0x6bd, { sample_period, sample_freq }: 1, sample_type: IP|TID|TIME|CPU|PERIOD|RAW, disabled: 1, inherit: 1, mmap: 1, comm: 1, task: 1, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1 # We now have a PERF_TYPE_SOFTWARE (type: 1), but the config states 0x6bd, which is how, after setting up the event via the kprobes interface, the 'perf_bpf_probe:fork' event is accessible via the perf_event_open syscall. This is all transient, as soon as the 'perf record' session ends, these probes will go away. To see how it looks like, lets try doing a neverending session, one that expects a control+C to end: # perf record --event /tmp/foo.o -a So, with that in place, we can use 'perf probe' to see what is in place: # perf probe -l perf_bpf_probe:fork (on _do_fork@acme/git/linux/kernel/fork.c) We also can use debugfs: [root@felicio ~]# cat /sys/kernel/debug/tracing/kprobe_events p:perf_bpf_probe/fork _text+638512 Ok, now lets stop and see if we got some forks: [root@felicio linux]# perf record --event /tmp/foo.o -a ^C[ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.325 MB perf.data (111 samples) ] [root@felicio linux]# perf script sshd 1271 [003] 81797.507678: perf_bpf_probe:fork: (ffffffff8109be30) sshd 18309 [000] 81797.524917: perf_bpf_probe:fork: (ffffffff8109be30) sshd 18309 [001] 81799.381603: perf_bpf_probe:fork: (ffffffff8109be30) sshd 18309 [001] 81799.408635: perf_bpf_probe:fork: (ffffffff8109be30) <SNIP> Sure enough, we have 111 forks :-) Callchains seems to work as well: # perf report --stdio --no-child # To display the perf.data header info, please use --header/--header-only options. # # Total Lost Samples: 0 # # Samples: 562 of event 'perf_bpf_probe:fork' # Event count (approx.): 562 # # Overhead Command Shared Object Symbol # ........ ........ ................ ............ # 44.66% sh [kernel.vmlinux] [k] _do_fork | ---_do_fork entry_SYSCALL_64_fastpath __libc_fork make_child 26.16% make [kernel.vmlinux] [k] _do_fork <SNIP> # Signed-off-by: Wang Nan <wangnan0@huawei.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Alexei Starovoitov <ast@plumgrid.com> Cc: Brendan Gregg <brendan.d.gregg@gmail.com> Cc: Daniel Borkmann <daniel@iogearbox.net> Cc: David Ahern <dsahern@gmail.com> Cc: He Kuang <hekuang@huawei.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Kaixu Xia <xiakaixu@huawei.com> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Zefan Li <lizefan@huawei.com> Cc: pi3orama@163.com Link: http://lkml.kernel.org/r/1444826502-49291-7-git-send-email-wangnan0@huawei.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-10-28perf tools: Load eBPF object into kernelWang Nan
This patch utilizes bpf_object__load() provided by libbpf to load all objects into kernel. Committer notes: Testing it: When using an incorrect kernel version number, i.e., having this in your eBPF proggie: int _version __attribute__((section("version"), used)) = 0x40100; For a 4.3.0-rc6+ kernel, say, this happens and needs checking at event parsing time, to provide a better error report to the user: # perf record --event /tmp/foo.o sleep 1 libbpf: load bpf program failed: Invalid argument libbpf: -- BEGIN DUMP LOG --- libbpf: libbpf: -- END LOG -- libbpf: failed to load program 'fork=_do_fork' libbpf: failed to load object '/tmp/foo.o' event syntax error: '/tmp/foo.o' \___ Invalid argument: Are you root and runing a CONFIG_BPF_SYSCALL kernel? (add -v to see detail) Run 'perf list' for a list of valid events Usage: perf record [<options>] [<command>] or: perf record [<options>] -- <command> [<options>] -e, --event <event> event selector. use 'perf list' to list available events If we instead make it match, i.e. use 0x40300 on this v4.3.0-rc6+ kernel, the whole process goes thru: # perf record --event /tmp/foo.o -a usleep 1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.202 MB perf.data ] # perf evlist -v /tmp/foo.o: type: 1, size: 112, config: 0x9, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|CPU|PERIOD, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, task: 1, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1 # Signed-off-by: Wang Nan <wangnan0@huawei.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Alexei Starovoitov <ast@plumgrid.com> Cc: Brendan Gregg <brendan.d.gregg@gmail.com> Cc: Daniel Borkmann <daniel@iogearbox.net> Cc: David Ahern <dsahern@gmail.com> Cc: He Kuang <hekuang@huawei.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Kaixu Xia <xiakaixu@huawei.com> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Zefan Li <lizefan@huawei.com> Cc: pi3orama@163.com Link: http://lkml.kernel.org/r/1444826502-49291-6-git-send-email-wangnan0@huawei.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-10-28perf tools: Create probe points for BPF programsWang Nan
This patch introduces bpf__{un,}probe() functions to enable callers to create kprobe points based on section names a BPF program. It parses the section names in the program and creates corresponding 'struct perf_probe_event' structures. The parse_perf_probe_command() function is used to do the main parsing work. The resuling 'struct perf_probe_event' is stored into program private data for further using. By utilizing the new probing API, this patch creates probe points during event parsing. To ensure probe points be removed correctly, register an atexit hook so even perf quit through exit() bpf__clear() is still called, so probing points are cleared. Note that bpf_clear() should be registered before bpf__probe() is called, so failure of bpf__probe() can still trigger bpf__clear() to remove probe points which are already probed. strerror style error reporting scaffold is created by this patch. bpf__strerror_probe() is the first error reporting function in bpf-loader.c. Committer note: Trying it: To build a test eBPF object file: I am testing using a script I built from the 'perf test -v LLVM' output: $ cat ~/bin/hello-ebpf export KERNEL_INC_OPTIONS="-nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/4.8.3/include -I/home/acme/git/linux/arch/x86/include -Iarch/x86/include/generated/uapi -Iarch/x86/include/generated -I/home/acme/git/linux/include -Iinclude -I/home/acme/git/linux/arch/x86/include/uapi -Iarch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -Iinclude/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h" export WORKING_DIR=/lib/modules/4.2.0/build export CLANG_SOURCE=- export CLANG_OPTIONS=-xc OBJ=/tmp/foo.o rm -f $OBJ echo '__attribute__((section("fork=do_fork"), used)) int fork(void *ctx) {return 0;} char _license[] __attribute__((section("license"), used)) = "GPL";int _version __attribute__((section("version"), used)) = 0x40100;' | \ clang -D__KERNEL__ $CLANG_OPTIONS $KERNEL_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c "$CLANG_SOURCE" -target bpf -O2 -o /tmp/foo.o && file $OBJ --- First asking to put a probe in a function not present in the kernel (misses the initial _): $ perf record --event /tmp/foo.o sleep 1 Probe point 'do_fork' not found. event syntax error: '/tmp/foo.o' \___ You need to check probing points in BPF file (add -v to see detail) Run 'perf list' for a list of valid events Usage: perf record [<options>] [<command>] or: perf record [<options>] -- <command> [<options>] -e, --event <event> event selector. use 'perf list' to list available events $ --- Now, with "__attribute__((section("fork=_do_fork"), used)): $ grep _do_fork /proc/kallsyms ffffffff81099ab0 T _do_fork $ perf record --event /tmp/foo.o sleep 1 Failed to open kprobe_events: Permission denied event syntax error: '/tmp/foo.o' \___ Permission denied --- Cool, we need to provide some better hints, "kprobe_events" is too low level, one doesn't strictly need to know the precise details of how these things are put in place, so something that shows the command needed to fix the permissions would be more helpful. Lets try as root instead: # perf record --event /tmp/foo.o sleep 1 Lowering default frequency rate to 1000. Please consider tweaking /proc/sys/kernel/perf_event_max_sample_rate. [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.013 MB perf.data ] # perf evlist /tmp/foo.o [root@felicio ~]# perf evlist -v /tmp/foo.o: type: 1, size: 112, config: 0x9, { sample_period, sample_freq }: 1000, sample_type: IP|TID|TIME|PERIOD, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, enable_on_exec: 1, task: 1, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1 --- Signed-off-by: Wang Nan <wangnan0@huawei.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Alexei Starovoitov <ast@plumgrid.com> Cc: Brendan Gregg <brendan.d.gregg@gmail.com> Cc: Daniel Borkmann <daniel@iogearbox.net> Cc: David Ahern <dsahern@gmail.com> Cc: He Kuang <hekuang@huawei.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Kaixu Xia <xiakaixu@huawei.com> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Zefan Li <lizefan@huawei.com> Cc: pi3orama@163.com Link: http://lkml.kernel.org/r/1444826502-49291-5-git-send-email-wangnan0@huawei.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-10-28perf tools: Enable passing bpf object file to --eventWang Nan
By introducing new rules in tools/perf/util/parse-events.[ly], this patch enables 'perf record --event bpf_file.o' to select events by an eBPF object file. It calls parse_events_load_bpf() to load that file, which uses bpf__prepare_load() and finally calls bpf_object__open() for the object files. After applying this patch, commands like: # perf record --event foo.o sleep become possible. However, at this point it is unable to link any useful things onto the evsel list because the creating of probe points and BPF program attaching have not been implemented. Before real events are possible to be extracted, to avoid perf report error because of empty evsel list, this patch link a dummy evsel. The dummy event related code will be removed when probing and extracting code is ready. Commiter notes: Using it: $ ls -la foo.o ls: cannot access foo.o: No such file or directory $ perf record --event foo.o sleep libbpf: failed to open foo.o: No such file or directory event syntax error: 'foo.o' \___ BPF object file 'foo.o' is invalid (add -v to see detail) Run 'perf list' for a list of valid events Usage: perf record [<options>] [<command>] or: perf record [<options>] -- <command> [<options>] -e, --event <event> event selector. use 'perf list' to list available events $ $ file /tmp/build/perf/perf.o /tmp/build/perf/perf.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped $ perf record --event /tmp/build/perf/perf.o sleep libbpf: /tmp/build/perf/perf.o is not an eBPF object file event syntax error: '/tmp/build/perf/perf.o' \___ BPF object file '/tmp/build/perf/perf.o' is invalid (add -v to see detail) Run 'perf list' for a list of valid events Usage: perf record [<options>] [<command>] or: perf record [<options>] -- <command> [<options>] -e, --event <event> event selector. use 'perf list' to list available events $ $ file /tmp/foo.o /tmp/foo.o: ELF 64-bit LSB relocatable, no machine, version 1 (SYSV), not stripped $ perf record --event /tmp/foo.o sleep 1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.013 MB perf.data ] $ perf evlist /tmp/foo.o $ perf evlist -v /tmp/foo.o: type: 1, size: 112, config: 0x9, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|PERIOD, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, enable_on_exec: 1, task: 1, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1 $ So, type 1 is PERF_TYPE_SOFTWARE, config 0x9 is PERF_COUNT_SW_DUMMY, ok. $ perf report --stdio Error: The perf.data file has no samples! # To display the perf.data header info, please use --header/--header-only options. # $ Signed-off-by: Wang Nan <wangnan0@huawei.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Alexei Starovoitov <ast@plumgrid.com> Cc: Brendan Gregg <brendan.d.gregg@gmail.com> Cc: Daniel Borkmann <daniel@iogearbox.net> Cc: David Ahern <dsahern@gmail.com> Cc: He Kuang <hekuang@huawei.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Kaixu Xia <xiakaixu@huawei.com> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Zefan Li <lizefan@huawei.com> Cc: pi3orama@163.com Link: http://lkml.kernel.org/r/1444826502-49291-4-git-send-email-wangnan0@huawei.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-10-28perf tools: Enable pre-event inherit setting by config termsWang Nan
This patch allows perf record setting event's attr.inherit bit by config terms like: # perf record -e cycles/no-inherit/ ... # perf record -e cycles/inherit/ ... So user can control inherit bit for each event separately. In following example, a.out fork()s in main then do some complex CPU intensive computations in both of its children. Basic result with and without inherit: # perf record -e cycles -e instructions ./a.out [ perf record: Woken up 9 times to write data ] [ perf record: Captured and wrote 2.205 MB perf.data (47920 samples) ] # perf report --stdio # ... # Samples: 23K of event 'cycles' # Event count (approx.): 23641752891 ... # Samples: 24K of event 'instructions' # Event count (approx.): 30428312415 # perf record -i -e cycles -e instructions ./a.out [ perf record: Woken up 5 times to write data ] [ perf record: Captured and wrote 1.111 MB perf.data (24019 samples) ] ... # Samples: 12K of event 'cycles' # Event count (approx.): 11699501775 ... # Samples: 12K of event 'instructions' # Event count (approx.): 15058023559 Cancel inherit for one event when globally enable: # perf record -e cycles/no-inherit/ -e instructions ./a.out [ perf record: Woken up 7 times to write data ] [ perf record: Captured and wrote 1.660 MB perf.data (36004 samples) ] ... # Samples: 12K of event 'cycles/no-inherit/' # Event count (approx.): 11895759282 ... # Samples: 24K of event 'instructions' # Event count (approx.): 30668000441 Enable inherit for one event when globally disable: # perf record -i -e cycles/inherit/ -e instructions ./a.out [ perf record: Woken up 7 times to write data ] [ perf record: Captured and wrote 1.654 MB perf.data (35868 samples) ] ... # Samples: 23K of event 'cycles/inherit/' # Event count (approx.): 23285400229 ... # Samples: 11K of event 'instructions' # Event count (approx.): 14969050259 Committer note: One can check if the bit was set, in addition to seeing the result in the perf.data file size as above by doing one of: # perf record -e cycles -e instructions -a usleep 1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.911 MB perf.data (63 samples) ] # perf evlist -v cycles: size: 112, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|ID|CPU|PERIOD, read_format: ID, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, task: 1, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1 instructions: size: 112, config: 0x1, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|ID|CPU|PERIOD, read_format: ID, disabled: 1, inherit: 1, freq: 1, sample_id_all: 1, exclude_guest: 1 # So, the inherit bit was set in both, now, if we disable it globally using --no-inherit: # perf record --no-inherit -e cycles -e instructions -a usleep 1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.910 MB perf.data (56 samples) ] # perf evlist -v cycles: size: 112, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|ID|CPU|PERIOD, read_format: ID, disabled: 1, mmap: 1, comm: 1, freq: 1, task: 1, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1 instructions: size: 112, config: 0x1, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|ID|CPU|PERIOD, read_format: ID, disabled: 1, freq: 1, sample_id_all: 1, exclude_guest: 1 No inherit bit set, then disabling it and setting just on the cycles event: # perf record --no-inherit -e cycles/inherit/ -e instructions -a usleep 1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.909 MB perf.data (48 samples) ] # perf evlist -v cycles/inherit/: size: 112, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|ID|CPU|PERIOD, read_format: ID, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, task: 1, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1 instructions: size: 112, config: 0x1, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|ID|CPU|PERIOD, read_format: ID, disabled: 1, freq: 1, sample_id_all: 1, exclude_guest: 1 # We can see it as well in by using a more verbose level of debug messages in the tool that sets up the perf_event_attr, 'perf record' in this case: [root@zoo ~]# perf record -vv --no-inherit -e cycles/inherit/ -e instructions -a usleep 1 ------------------------------------------------------------ perf_event_attr: size 112 { sample_period, sample_freq } 4000 sample_type IP|TID|TIME|ID|CPU|PERIOD read_format ID disabled 1 inherit 1 mmap 1 comm 1 freq 1 task 1 sample_id_all 1 exclude_guest 1 mmap2 1 comm_exec 1 ------------------------------------------------------------ sys_perf_event_open: pid -1 cpu 0 group_fd -1 flags 0x8 sys_perf_event_open: pid -1 cpu 1 group_fd -1 flags 0x8 sys_perf_event_open: pid -1 cpu 2 group_fd -1 flags 0x8 sys_perf_event_open: pid -1 cpu 3 group_fd -1 flags 0x8 ------------------------------------------------------------ perf_event_attr: size 112 config 0x1 { sample_period, sample_freq } 4000 sample_type IP|TID|TIME|ID|CPU|PERIOD read_format ID disabled 1 freq 1 sample_id_all 1 exclude_guest 1 ------------------------------------------------------------ sys_perf_event_open: pid -1 cpu 0 group_fd -1 flags 0x8 <SNIP> Signed-off-by: Wang Nan <wangnan0@huawei.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Alexei Starovoitov <ast@plumgrid.com> Cc: Brendan Gregg <brendan.d.gregg@gmail.com> Cc: David S. Miller <davem@davemloft.net> Cc: Li Zefan <lizefan@huawei.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Zefan Li <lizefan@huawei.com> Cc: pi3orama@163.com Link: http://lkml.kernel.org/r/1446029705-199659-2-git-send-email-wangnan0@huawei.com [ s/u64/bool/ for the perf_evsel_config_term inherit field - jolsa] Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-10-05perf tools: Fail properly in case pattern matching fails to find tracepointJiri Olsa
Currently we dont fail properly when pattern matching fails to find any tracepoint. Current behaviour: $ perf record -e 'sched:krava*' sleep 1 WARNING: event parser found nothinginvalid or unsupported event: 'sched:krava*' Run 'perf list' for a list of valid events usage: perf record [<options>] [<command>] or: perf record [<options>] -- <command> [<options>] This patch change: $ perf record -e 'sched:krava*' sleep 1 event syntax error: 'sched:krava*' \___ unknown tracepoint Error: File /sys/kernel/debug/tracing/events/sched/krava* not found. Hint: Perhaps this kernel misses some CONFIG_ setting to enable this feature?. Run 'perf list' for a list of valid events usage: perf record [<options>] [<command>] or: perf record [<options>] -- <command> [<options>] Reported-by: Daniel Bristot de Oliveira <bristot@redhat.com> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: David Ahern <dsahern@gmail.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1444073477-3181-1-git-send-email-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-10-05perf tools: Introduce 'P' modifier to request max precisionJiri Olsa
The 'P' will cause the event to get maximum possible detected precise level. Following record: $ perf record -e cycles:P ... will detect maximum precise level for 'cycles' event and use it. Commiter note: Testing it: $ perf record -e cycles:P usleep 1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.013 MB perf.data (9 samples) ] $ perf evlist cycles:P $ perf evlist -v cycles:P: size: 112, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|PERIOD, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, enable_on_exec: 1, task: 1, precise_ip: 2, sample_id_all: 1, mmap2: 1, comm_exec: 1 $ Signed-off-by: Jiri Olsa <jolsa@kernel.org> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Kan Liang <kan.liang@intel.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1444068369-20978-6-git-send-email-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-30perf list: Remove blank lines, headers when piping outputArnaldo Carvalho de Melo
So that one can, for instance, use it with wc -l: # perf list *:*write* | wc -l 60 Or to look for the "bio" tracepoints, without 'perf list' headers: # perf list *:*bio* | head block:block_bio_backmerge [Tracepoint event] block:block_bio_bounce [Tracepoint event] block:block_bio_complete [Tracepoint event] block:block_bio_frontmerge [Tracepoint event] block:block_bio_queue [Tracepoint event] block:block_bio_remap [Tracepoint event] # Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Cc: Wang Nan <wangnan0@huawei.com> Link: http://lkml.kernel.org/n/tip-ts7sc0x8u4io4cifzkup4j44@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-30perf tools: Fix shadowed declaration in parse-events.cJiri Olsa
The error variable breaks build on CentOS 6.7, due to a collision with a global error symbol: CC util/parse-events.o cc1: warnings being treated as errors util/parse-events.c:419: error: declaration of ‘error’ shadows a global declaration util/util.h:135: error: shadowed declaration is here util/parse-events.c: In function ‘add_tracepoint_multi_event’: ... Using different argument names instead to fix it. Reported-by: Vinson Lee <vlee@twopensource.com> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: David Ahern <dsahern@gmail.com> Cc: He Kuang <hekuang@huawei.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: linux-tip-commits@vger.kernel.org Cc: Matt Fleming <matt@codeblueprint.co.uk> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Raphael Beamonte <raphael.beamonte@gmail.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/20150929150531.GI27383@krava.redhat.com [ Fix one more case, at line 770 ] Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-28perf tools: Enable event_config terms to tracepoint eventsHe Kuang
This patch enables config terms for tracepoint perf events. Valid terms for tracepoint events are 'call-graph' and 'stack-size', so we can use different callgraph settings for each event and eliminate unnecessary overhead. Here is an example for using different call-graph config for each tracepoint. $ perf record -e syscalls:sys_enter_write/call-graph=fp/ -e syscalls:sys_exit_write/call-graph=no/ dd if=/dev/zero of=test bs=4k count=10 $ perf report --stdio # # Total Lost Samples: 0 # # Samples: 13 of event 'syscalls:sys_enter_write' # Event count (approx.): 13 # # Children Self Command Shared Object Symbol # ........ ........ ....... .................. ...................... # 76.92% 76.92% dd libpthread-2.20.so [.] __write_nocancel | ---__write_nocancel 23.08% 23.08% dd libc-2.20.so [.] write | ---write | |--33.33%-- 0x2031342820736574 | |--33.33%-- 0xa6e69207364726f | --33.33%-- 0x34202c7320393039 ... # Samples: 13 of event 'syscalls:sys_exit_write' # Event count (approx.): 13 # # Children Self Command Shared Object Symbol # ........ ........ ....... .................. ...................... # 76.92% 76.92% dd libpthread-2.20.so [.] __write_nocancel 23.08% 23.08% dd libc-2.20.so [.] write 7.69% 0.00% dd [unknown] [.] 0x0a6e69207364726f 7.69% 0.00% dd [unknown] [.] 0x2031342820736574 7.69% 0.00% dd [unknown] [.] 0x34202c7320393039 Signed-off-by: He Kuang <hekuang@huawei.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Kan Liang <kan.liang@intel.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Wang Nan <wangnan0@huawei.com> Cc: pi3orama@163.com Link: http://lkml.kernel.org/r/1443412336-120050-4-git-send-email-hekuang@huawei.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-28perf tools: Show proper error message for wrong terms of hw/sw eventsHe Kuang
Show proper error message and show valid terms when wrong config terms is specified for hw/sw type perf events. This patch makes the original error format function formats_error_string() more generic, which only outputs the static config terms for hw/sw perf events, and prepends pmu formats for pmu events. Before this patch: $ perf record -e 'cpu-clock/freqx=200/' -a sleep 1 invalid or unsupported event: 'cpu-clock/freqx=200/' Run 'perf list' for a list of valid events usage: perf record [<options>] [<command>] or: perf record [<options>] -- <command> [<options>] -e, --event <event> event selector. use 'perf list' to list available events After this patch: $ perf record -e 'cpu-clock/freqx=200/' -a sleep 1 event syntax error: 'cpu-clock/freqx=200/' \___ unknown term valid terms: config,config1,config2,name,period,freq,branch_type,time,call-graph,stack-size Run 'perf list' for a list of valid events usage: perf record [<options>] [<command>] or: perf record [<options>] -- <command> [<options>] -e, --event <event> event selector. use 'perf list' to list available events Signed-off-by: He Kuang <hekuang@huawei.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Kan Liang <kan.liang@intel.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Wang Nan <wangnan0@huawei.com> Cc: pi3orama@163.com Link: http://lkml.kernel.org/r/1443412336-120050-2-git-send-email-hekuang@huawei.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-28perf tools: Adds the config_term callback for different type eventsHe Kuang
Currently, function config_term() is used for checking config terms of all types of events, while unknown terms is not reported as an error because pmu events have valid terms in sysfs. But this is wrong when unknown terms are specificed to hw/sw events. This patch Adds the config_term callback so we can use separate check routines for each type of events. Signed-off-by: He Kuang <hekuang@huawei.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Kan Liang <kan.liang@intel.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Wang Nan <wangnan0@huawei.com> Cc: pi3orama@163.com Link: http://lkml.kernel.org/r/1443412336-120050-1-git-send-email-hekuang@huawei.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-21perf tools: Don't assume that the parser returns non empty evsel listWang Nan
Don't blindly retrieve and use a last element in the lists returned by parse_events__scanner(), as it may have collected no entries, i.e. return an empty list. Signed-off-by: Wang Nan <wangnan0@huawei.com> Cc: Alexei Starovoitov <ast@plumgrid.com> Cc: Brendan Gregg <brendan.d.gregg@gmail.com> Cc: Daniel Borkmann <daniel@iogearbox.net> Cc: David Ahern <dsahern@gmail.com> Cc: He Kuang <hekuang@huawei.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Kaixu Xia <xiakaixu@huawei.com> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Zefan Li <lizefan@huawei.com> Cc: pi3orama@163.com Link: http://lkml.kernel.org/r/1441523623-152703-2-git-send-email-wangnan0@huawei.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-16Merge branch 'perf/urgent' into perf/core, to resolve a conflictIngo Molnar
Conflicts: tools/perf/ui/browsers/hists.c Signed-off-by: Ingo Molnar <mingo@kernel.org>
2015-09-15perf evsel: Add own_cpus memberAdrian Hunter
perf_evlist__propagate_maps() cannot easily tell if an evsel has its own cpu map. To make that simpler, keep a copy of the PMU cpu map and adjust the propagation logic accordingly. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Cc: Kan Liang <kan.liang@intel.com> Link: http://lkml.kernel.org/r/1441699142-18905-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-15perf evlist: Fix splice_list_tail() not setting evlistAdrian Hunter
Commit d49e46950772 ("perf evsel: Add a backpointer to the evlist a evsel is in") updated perf_evlist__add() but not perf_evlist__splice_list_tail(). This illustrates that it is better if perf_evlist__splice_list_tail() calls perf_evlist__add() instead of duplicating the logic, so do that. This will also simplify a subsequent fix for propagating maps. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Cc: Kan Liang <kan.liang@intel.com> Link: http://lkml.kernel.org/r/1441699142-18905-6-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-15perf tools: Enhance parsing events tracepoint error outputJiri Olsa
Enhancing parsing events tracepoint error output. Adding more verbose output when the tracepoint is not found or the tracing event path cannot be access. $ sudo perf record -e sched:sched_krava ls event syntax error: 'sched:sched_krava' \___ unknown tracepoint Error: File /sys/kernel/debug/tracing//tracing/events/sched/sched_krava not found. Hint: Perhaps this kernel misses some CONFIG_ setting to enable this feature?. Run 'perf list' for a list of valid events ... $ perf record -e sched:sched_krava ls event syntax error: 'sched:sched_krava' \___ can't access trace events Error: No permissions to read /sys/kernel/debug/tracing//tracing/events/sched/sched_krava Hint: Try 'sudo mount -o remount,mode=755 /sys/kernel/debug' Run 'perf list' for a list of valid events ... Signed-off-by: Jiri Olsa <jolsa@kernel.org> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Raphael Beamonte <raphael.beamonte@gmail.com> Cc: David Ahern <dsahern@gmail.com> Cc: Matt Fleming <matt@codeblueprint.co.uk> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1441615087-13886-6-git-send-email-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-15perf evsel: Propagate error info from tp_formatJiri Olsa
Propagate error info from tp_format via ERR_PTR to get it all the way down to the parse-event.c tracepoint adding routines. Following functions now return pointer with encoded error: - tp_format - trace_event__tp_format - perf_evsel__newtp_idx - perf_evsel__newtp This affects several other places in perf, that cannot use pointer check anymore, but must utilize the err.h interface, when getting error information from above functions list. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Cc: David Ahern <dsahern@gmail.com> Cc: Matt Fleming <matt@codeblueprint.co.uk> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Raphael Beamonte <raphael.beamonte@gmail.com> Link: http://lkml.kernel.org/r/1441615087-13886-5-git-send-email-jolsa@kernel.org [ Add two missing ERR_PTR() and one IS_ERR() ] Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-15perf tools: Propagate error info for the tracepoint parsingJiri Olsa
Pass 'struct parse_events_error *error' to the parse-event.c tracepoint adding path. It will be filled with error data in following patches. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Reviewed-by: Raphael Beamonte <raphael.beamonte@gmail.com> Reviewed-by: Matt Fleming <matt@codeblueprint.co.uk> Cc: David Ahern <dsahern@gmail.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1441615087-13886-4-git-send-email-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-04tools lib api fs: Move tracing_path interface into api/fs/tracing_path.cJiri Olsa
Moving tracing_path interface into api/fs/tracing_path.c out of util.c. It seems generic enough to be used by others, and I couldn't think of better place. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Reviewed-by: Matt Fleming <matt.fleming@intel.com> Reviewed-by: Raphael Beamonte <raphael.beamonte@gmail.com> Cc: David Ahern <dsahern@gmail.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Steven Rostedt <rostedt@goodmis.org> Link: http://lkml.kernel.org/r/1441180605-24737-5-git-send-email-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-12perf callchain: Per-event type selection supportKan Liang
This patchkit adds the ability to set callgraph mode (fp, dwarf, lbr) per event. This in term can reduce sampling overhead and the size of the perf.data. Here is an example. perf record -e 'cpu/cpu-cycles,period=1000,call-graph=fp,time=1/,cpu/instructions,call-graph=lbr/' sleep 1 perf evlist -v cpu/cpu-cycles,period=1000,call-graph=fp,time=1/: type: 4, size: 112, config: 0x3c, { sample_period, sample_freq }: 1000, sample_type: IP|TID|TIME|CALLCHAIN|PERIOD|IDENTIFIER, read_format: ID, disabled: 1, inherit: 1, mmap: 1, comm: 1, enable_on_exec: 1, task: 1, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1 cpu/instructions,call-graph=lbr/: type: 4, size: 112, config: 0xc0, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|CALLCHAIN|PERIOD|BRANCH_STACK|IDENTIFIER, read_format: ID, disabled: 1, inherit: 1, freq: 1, enable_on_exec: 1, sample_id_all: 1, exclude_guest: 1 Signed-off-by: Kan Liang <kan.liang@intel.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Andi Kleen <ak@linux.intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Namhyung Kim <namhyung@kernel.org> Link: http://lkml.kernel.org/r/1439289050-40510-1-git-send-email-kan.liang@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-10perf record: Support per-event freq termNamhyung Kim
Now perf can set per-event value of time and (sampling) period. But I guess most users like me just want to set frequency rather than period. So add the 'freq' term in the event parser. Signed-off-by: Namhyung Kim <namhyung@kernel.org> Acked-by: Jiri Olsa <jolsa@kernel.org> Cc: David Ahern <dsahern@gmail.com> Cc: Kan Liang <kan.liang@intel.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1439102724-14079-1-git-send-email-namhyung@kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-06perf tools: Extend the event parser maximum error indexAdrian Hunter
Extend the event parser maximum error index from 10 to 13. That allows PMU config terms of up to 10 characters to display un-truncated in the error message. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/1437150840-31811-17-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-05perf tools: Per-event time supportKan Liang
This patchkit adds the ability to turn off time stamps per event. One usaful case for partial time is to work with per-event callgraph to enable "PEBS threshold > 1" (https://lkml.org/lkml/2015/5/10/196), which can significantly reduce the sampling overhead. The event samples with time stamps off will not be ordered. Signed-off-by: Kan Liang <kan.liang@intel.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Cc: Andi Kleen <ak@linux.intel.com> Cc: Namhyung Kim <namhyung@kernel.org> Link: http://lkml.kernel.org/r/1438677022-34296-2-git-send-email-kan.liang@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-29perf tools: Force period term to overload global settingsJiri Olsa
Currently the command line option settings beats the per event period settings: With no global settings, we get per-event configuration: $ perf record -e 'cpu/instructions,period=20000/' sleep 1 $ perf evlist -v ... { sample_period, sample_freq }: 20000 ... With 'c' option period setup, we get 'c' option value: $ perf record -e 'cpu/instructions,period=20000/' -c 1000 sleep 1 $ perf evlist -v ... { sample_period, sample_freq }: 1000 ... This patch makes the per-event settings overload the global 'c' option setup: $ perf record -e 'cpu/instructions,period=20000/' -c 1000 sleep 1 $ perf evlist -v ... { sample_period, sample_freq }: 20000 ... I think the making the per-event settings to overload any other config makes more sense than current state. However it breaks the current 'period' term handling, which might cause some noise.. so let's see ;-). Also fixing parse event tests with the new behaviour. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Andi Kleen <ak@linux.intel.com> Cc: Kan Liang <kan.liang@intel.com> Cc: Namhyung Kim <namhyung@kernel.org> Link: http://lkml.kernel.org/r/1438162936-59698-3-git-send-email-kan.liang@intel.com Signed-off-by: Kan Liang <kan.liang@intel.com> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-29perf tools: Add support for event post configurationJiri Olsa
Add support to overload any global settings for event and force user specified term value. It will be useful for new time and backtrace terms. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Cc: Andi Kleen <ak@linux.intel.com> Cc: Kan Liang <kan.liang@intel.com> Cc: Namhyung Kim <namhyung@kernel.org> Link: http://lkml.kernel.org/r/1438162936-59698-2-git-send-email-kan.liang@intel.com Signed-off-by: Kan Liang <kan.liang@intel.com> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-20perf record: Allow filtering perf's pid via --exclude-perfWang Nan
This patch allows 'perf record' to exclude events issued by perf itself by '--exclude-perf' option. Before this patch, when doing something like: # perf record -a -e syscalls:sys_enter_write <cmd> One could easily get result like this: # /tmp/perf report --stdio ... # Overhead Command Shared Object Symbol # ........ ....... .................. .................... # 99.99% perf libpthread-2.18.so [.] __write_nocancel 0.01% ls libc-2.18.so [.] write 0.01% sshd libc-2.18.so [.] write ... Where most events are generated by perf itself. A shell trick can be done to filter perf itself out: # cat << EOF > ./tmp > #!/bin/sh > exec perf record -e ... --filter="common_pid != \$\$" -a sleep 10 > EOF # chmod a+x ./tmp # ./tmp However, doing so is user unfriendly. This patch extracts evsel iteration framework introduced by patch 'perf record: Apply filter to all events in a glob matching' into foreach_evsel_in_last_glob(), and makes exclude_perf() function append new filter expression to each evsel selected by a '-e' selector. To avoid losing filters if user pass '--filter' after '--exclude-perf', this patch uses perf_evsel__append_filter() in both case, instead of perf_evsel__set_filter() which removes old filter. As a side effect, now it is possible to use multiple '--filter' option for one selector. They are combinded with '&&'. Signed-off-by: Wang Nan <wangnan0@huawei.com> Cc: Andi Kleen <ak@linux.intel.com> Cc: Brendan Gregg <brendan.d.gregg@gmail.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Zefan Li <lizefan@huawei.com> Cc: pi3orama@163.com Link: http://lkml.kernel.org/r/1436513770-8896-2-git-send-email-wangnan0@huawei.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-20perf record: Apply filter to all events in a glob matchingWang Nan
There is an old problem in perf's filter applying which first posted at Sep. 2014 at https://lkml.org/lkml/2014/9/9/944 that, if passing multiple events in a glob matching expression in cmdline then add '--filter' after them, the filter will be applied on only the last one. For example: # dd if=/dev/zero of=/dev/null & [1] 464 # perf record -a -e 'syscalls:sys_*_read' --filter 'common_pid != 464' sleep 0.1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.239 MB perf.data (2094 samples) ] # perf report --stdio | tee ... # Samples: 2K of event 'syscalls:sys_enter_read' # Event count (approx.): 2092 ... # Samples: 2 of event 'syscalls:sys_exit_read' # Event count (approx.): 2 ... In this example, filter only applied on 'syscalls:sys_exit_read', and there's no way to set filter for ''syscalls:sys_enter_read'. This patch adds a 'cmdline_group_boundary' for 'struct evsel', and apply filter on all events between two boundary marks. After applying this patch: # perf record -a -e 'syscalls:sys_*_read' --filter 'common_pid != 464' sleep 0.1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.031 MB perf.data (3 samples) ] # perf report --stdio | tee ... # Samples: 1 of event 'syscalls:sys_enter_read' # Event count (approx.): 1 ... # Samples: 2 of event 'syscalls:sys_exit_read' # Event count (approx.): 2 ... Signed-off-by: Wang Nan <wangnan0@huawei.com> Reported-by: Brendan Gregg <brendan.d.gregg@gmail.com> Cc: Andi Kleen <ak@linux.intel.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Zefan Li <lizefan@huawei.com> Cc: pi3orama@163.com Link: http://lkml.kernel.org/r/1436513770-8896-1-git-send-email-wangnan0@huawei.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-06perf evsel: Introduce set_filter methodArnaldo Carvalho de Melo
Replaces existing filter string with the one provided. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-jst49z83li0yx3g18o54u51a@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-06-25perf tools: Add reference counting for cpu_map objectJiri Olsa
Adding refference counting for cpu_map object, so it could be easily shared among other objects. Using cpu_map__put instead cpu_map__delete and making cpu_map__delete static. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Andi Kleen <ak@linux.intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1435012588-9007-4-git-send-email-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-05-27perf tools: Fix parse_events_error dereferencesAdrian Hunter
Parse errors can be reported in struct parse_events_error but the pointer passed is optional and can be NULL. Ensure it is not NULL before dereferencing it. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Link: http://lkml.kernel.org/r/1432040746-1755-4-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-05-27perf tools: Fix function declarations needed by parse-events.yAdrian Hunter
Patch "perf tools: Add location to pmu event terms" moved declarations for parse_events_term__num() and parse_events_term__str() so that they were no longer visible in parse-events.y. That can result in segfaults as the arguments no longer need match the function prototype. Move the declarations back, changing YYLTYPE pointers to pointers-to-void because YYLTYPE is not generated until parse-events.y is processed. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Link: http://lkml.kernel.org/r/1432040746-1755-2-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-29perf tools: Add symbolic events support for parse_events_errorJiri Olsa
Allowing symbolic events processing to report back error. $ perf record -e 'cycles/period=krava/' ls event syntax error: '../period=krava/' \___ expected numeric value $ perf record -e 'cycles/name=1/' ls event syntax error: '..es/name=1/' \___ expected string value Signed-off-by: Jiri Olsa <jolsa@kernel.org> Cc: David Ahern <dsahern@gmail.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1429729824-13932-10-git-send-email-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-29perf tools: Add static terms support for parse_events_errorJiri Olsa
Allowing static terms like 'name,period,config,config1..' processing to report back error. $ perf record -e 'cpu/event=1,name=1/' ls event syntax error: '..=1,name=1/' \___ expected string value $ perf record -e 'cpu/event=1,period=krava/' ls event syntax error: '..,period=krava/' \___ expected numeric value $ perf record -e 'cpu/config=krava1/' ls event syntax error: '../config=krava1/' \___ expected numeric value Signed-off-by: Jiri Olsa <jolsa@kernel.org> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: David Ahern <dsahern@gmail.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1429729824-13932-8-git-send-email-jolsa@kernel.org [ Renamed 'error' variables to 'err', not to clash with util.h error() ] Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-29perf tools: Add term support for parse_events_errorJiri Olsa
Allowing event's term processing to report back error, like: $ perf record -e 'cpu/even=0x1/' ls event syntax error: 'cpu/even=0x1/' \___ unknown term valid terms: pc,any,inv,edge,cmask,event,in_tx,ldlat,umask,in_tx_cp,offcore_rsp,config,config1,config2,name,period,branch_type Signed-off-by: Jiri Olsa <jolsa@kernel.org> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: David Ahern <dsahern@gmail.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1429729824-13932-7-git-send-email-jolsa@kernel.org [ Renamed 'error' variables to 'err', not to clash with util.h error() ] Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-29perf tools: Add location to pmu event termsJiri Olsa
Saving the terms location within term struct, so it could be used later for report. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Cc: David Ahern <dsahern@gmail.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1429729824-13932-6-git-send-email-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-29perf tools: Change parse_events_add_pmu interfaceJiri Olsa
Changing parse_events_add_pmu interface to allow propagating of the parse_events_error info. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Cc: David Ahern <dsahern@gmail.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1429729824-13932-5-git-send-email-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-29perf tools: Always bail out when config_attr function failsJiri Olsa
Not sure why we allowed the fail state, but it's wrong. Wrong type for 'name' term can cause segfault, and there's probably more fun hidden. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Cc: David Ahern <dsahern@gmail.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1429729824-13932-4-git-send-email-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-29perf tools: Add parse_events_error interfaceJiri Olsa
Adding support to return error information from parse_events function. Following struct will be populated by parse_events function on return: struct parse_events_error { int idx; char *str; char *help; }; where 'idx' is the position in the string where the parsing failed, 'str' contains dynamically allocated error string describing the error and 'help' is optional help string. The change contains reporting function, which currently does not display anything. The code changes to supply error data for specific event types are coming in next patches. However this is what the expected output is: $ sudo perf record -e 'sched:krava' ls event syntax error: 'sched:krava' \___ unknown tracepoint ... $ perf record -e 'cpu/even=0x1/' ls event syntax error: 'cpu/even=0x1/' \___ unknown term valid terms: pc,any,inv,edge,cmask,event,in_tx,ldlat,umask,in_tx_cp,offcore_rsp,config,config1,config2,name,period,branch_type ... $ perf record -e cycles,cache-mises ls event syntax error: '..es,cache-mises' \___ parser error ... The output functions cut the beginning of the event string so the error starts up to 10th character and cut the end of the string of it crosses the terminal width. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Cc: David Ahern <dsahern@gmail.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1429729824-13932-2-git-send-email-jolsa@kernel.org [ Renamed 'error' variables to 'err', not to clash with util.h error() ] Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-08perf tools: Add 'I' event modifier for exclude_idle bitJiri Olsa
Adding 'I' event modifier to have complete set of modifiers for perf_event_attr:exclude_* bits. Any event specified with 'I' modifier will have the perf_event_attr:exclude_idle bit set. $ perf record -e cycles:I -vv ls 2>&1 | grep exclude_idle exclude_hv 0 exclude_idle 1 Adding automated tests. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Cc: Andi Kleen <andi@firstfloor.org> Cc: David Ahern <dsahern@gmail.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: William Cohen <wcohen@redhat.com> Link: http://lkml.kernel.org/r/1428441919-23099-2-git-send-email-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-27perf list: Clean up the printing functions of hardware/software eventsYunlong Song
Do not need print_events_type or __print_events_type for listing hw/sw events, let print_symbol_events do its job instead. Moreover, print_symbol_events can also handle event_glob and name_only. Signed-off-by: Yunlong Song <yunlong.song@huawei.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Wang Nan <wangnan0@huawei.com> Link: http://lkml.kernel.org/r/1425032491-20224-4-git-send-email-yunlong.song@huawei.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-27perf list: Sort the output of 'perf list' to view more clearlyYunlong Song
Sort the output according to ASCII character list (using strcmp), which supports both number sequence and alphabet sequence. Example: Before this patch: $ perf list List of pre-defined events (to be used in -e): cpu-cycles OR cycles [Hardware event] instructions [Hardware event] cache-references [Hardware event] cache-misses [Hardware event] branch-instructions OR branches [Hardware event] branch-misses [Hardware event] bus-cycles [Hardware event] ... ... jbd2:jbd2_start_commit [Tracepoint event] jbd2:jbd2_commit_locking [Tracepoint event] jbd2:jbd2_run_stats [Tracepoint event] block:block_rq_issue [Tracepoint event] block:block_bio_complete [Tracepoint event] block:block_bio_backmerge [Tracepoint event] block:block_getrq [Tracepoint event] ... ... After this patch: $ perf list List of pre-defined events (to be used in -e): branch-instructions OR branches [Hardware event] branch-misses [Hardware event] bus-cycles [Hardware event] cache-misses [Hardware event] cache-references [Hardware event] cpu-cycles OR cycles [Hardware event] instructions [Hardware event] ... ... block:block_bio_backmerge [Tracepoint event] block:block_bio_complete [Tracepoint event] block:block_getrq [Tracepoint event] block:block_rq_issue [Tracepoint event] jbd2:jbd2_commit_locking [Tracepoint event] jbd2:jbd2_run_stats [Tracepoint event] jbd2:jbd2_start_commit [Tracepoint event] ... ... Signed-off-by: Yunlong Song <yunlong.song@huawei.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Wang Nan <wangnan0@huawei.com> Link: http://lkml.kernel.org/r/1425032491-20224-2-git-send-email-yunlong.song@huawei.com [ Don't forget closedir({sys,evt}_dir) when handling errors ] Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-13perf list: Place the header text in its right positionYunlong Song
The hearer text 'List of pre-defined events (to be used in -e):' is placed in an improper function, which causes an abnormal output, e.g. 'perf list hw' shows no guiding text at all, and 'perf list hw L1-dcache*' shows the guiding text incorrectly in the middle of the output. Example Before this patch: $ perf list hw L1-dcache* branch-instructions OR branches [Hardware event] branch-misses [Hardware event] bus-cycles [Hardware event] cache-misses [Hardware event] cache-references [Hardware event] cpu-cycles OR cycles [Hardware event] instructions [Hardware event] stalled-cycles-backend OR idle-cycles-backend [Hardware event] stalled-cycles-frontend OR idle-cycles-frontend [Hardware event] List of pre-defined events (to be used in -e): <-- incorrect position L1-dcache-load-misses [Hardware cache event] L1-dcache-loads [Hardware cache event] L1-dcache-prefetch-misses [Hardware cache event] L1-dcache-prefetches [Hardware cache event] L1-dcache-store-misses [Hardware cache event] L1-dcache-stores [Hardware cache event] After this patch: $ perf list hw L1-dcache* List of pre-defined events (to be used in -e): <-- correct position branch-instructions OR branches [Hardware event] branch-misses [Hardware event] bus-cycles [Hardware event] cache-misses [Hardware event] cache-references [Hardware event] cpu-cycles OR cycles [Hardware event] instructions [Hardware event] stalled-cycles-backend OR idle-cycles-backend [Hardware event] stalled-cycles-frontend OR idle-cycles-frontend [Hardware event] L1-dcache-load-misses [Hardware cache event] L1-dcache-loads [Hardware cache event] L1-dcache-prefetch-misses [Hardware cache event] L1-dcache-prefetches [Hardware cache event] L1-dcache-store-misses [Hardware cache event] L1-dcache-stores [Hardware cache event] Signed-off-by: Yunlong Song <yunlong.song@huawei.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Paul Mackerras <paulus@samba.org> Cc: Wang Nan <wangnan0@huawei.com> Link: http://lkml.kernel.org/r/1423833115-11199-8-git-send-email-yunlong.song@huawei.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-07perf tools: Do not check debugfs MAGIC for tracing filesSteven Rostedt (Red Hat)
It's rather strange to be checking the debugfs MAGIC number for the tracing directory. A system admin may want to have a custom set of events to trace and it should be allowed to let the admin make a temp file (even for tracing virtual boxes, this is useful). Also with the coming tracefs, the files may not even be under debugfs, so checking the debugfs MAGIC number is pointless. Signed-off-by: Steven Rostedt <rostedt@goodmis.org> Acked-by: Jiri Olsa <jolsa@kernel.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Ingo Molnar <mingo@kernel.org> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Namhyung Kim <namhyung@kernel.org> Link: http://lkml.kernel.org/r/20150202193552.546175764@goodmis.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-01-28Merge branch 'perf/hw_breakpoints' into perf/coreIngo Molnar
The new hw_breakpoint bits are now ready for v3.20, merge them into the main branch, to avoid conflicts. Conflicts: tools/perf/Documentation/perf-record.txt Signed-off-by: Ingo Molnar <mingo@kernel.org>
2015-01-21perf tools: Remove EOL whitespacesArnaldo Carvalho de Melo
Janitorial stuff: boredom moment. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-u70i7shys3kths4hzru72bha@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-12-03perf tools: allow user to specify hardware breakpoint bp_lenJacob Shin
Currently bp_len is given a default value of 4. Allow user to override it: $ perf stat -e mem:0x1000/8 ^ bp_len If no value is given, it will default to 4 as it did before. Signed-off-by: Jacob Shin <jacob.w.shin@gmail.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Reviewed-by: Oleg Nesterov <oleg@redhat.com> Cc: Arnaldo Carvalho de Melo <acme@ghostprotocols.net> Cc: Ingo Molnar <mingo@kernel.org> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: xiakaixu <xiakaixu@huawei.com> Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
2014-11-24perf tools: Add snapshot format file parsingJiri Olsa
The .snapshot file indicates that the provided event value is a snapshot value and we have to bypass the delta computation logic. Adding support to check up this file and set event flag accordingly. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Cc: Andi Kleen <ak@linux.intel.com> Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Matt Fleming <matt.fleming@intel.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1416562275-12404-10-git-send-email-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>