| Age | Commit message (Collapse) | Author |
|---|
|
This patch adds several new tunables to control the behavior of
elision on supported platforms[1]. Since elision now depends
on tunables, we should always *compile* with elision enabled,
and leave the code disabled, but available for runtime
selection. This gives us *much* better compile-time testing of
the existing code to avoid bit-rot[2].
Tested on ppc, ppc64, ppc64le, s390x and x86_64.
[1] This part of the patch was initially proposed by
Paul Murphy but was "staled" because the framework have changed
since the patch was originally proposed:
https://patchwork.sourceware.org/patch/10342/
[2] This part of the patch was inititally proposed as a RFC by
Carlos O'Donnell. Make sense to me integrate this on the patch:
https://sourceware.org/ml/libc-alpha/2017-05/msg00335.html
* elf/dl-tunables.list: Add elision parameters.
* manual/tunables.texi: Add entries about elision tunable.
* sysdeps/unix/sysv/linux/powerpc/elision-conf.c:
Add callback functions to dynamically enable/disable elision.
Add multiple callbacks functions to set elision parameters.
Deleted __libc_enable_secure check.
* sysdeps/unix/sysv/linux/s390/elision-conf.c: Likewise.
* sysdeps/unix/sysv/linux/x86/elision-conf.c: Likewise.
* configure: Regenerated.
* configure.ac: Option enable_lock_elision was deleted.
* config.h.in: ENABLE_LOCK_ELISION flag was deleted.
* config.make.in: Remove references to enable_lock_elision.
* manual/install.texi: Elision configure option was removed.
* INSTALL: Regenerated to remove enable_lock_elision.
* nptl/Makefile:
Disable elision so it can verify error case for destroying a mutex.
* sysdeps/powerpc/nptl/elide.h:
Cleanup ENABLE_LOCK_ELISION check.
Deleted macros for the case when ENABLE_LOCK_ELISION was not defined.
* sysdeps/s390/configure: Regenerated.
* sysdeps/s390/configure.ac: Remove references to enable_lock_elision..
* nptl/tst-mutex8.c:
Deleted all #ifndef ENABLE_LOCK_ELISION from the test.
* sysdeps/powerpc/powerpc32/sysdep.h:
Deleted all ENABLE_LOCK_ELISION checks.
* sysdeps/powerpc/powerpc64/sysdep.h: Likewise.
* sysdeps/powerpc/sysdep.h: Likewise.
* sysdeps/s390/nptl/bits/pthreadtypes-arch.h: Likewise.
* sysdeps/unix/sysv/linux/powerpc/force-elision.h: Likewise.
* sysdeps/unix/sysv/linux/s390/elision-conf.h: Likewise.
* sysdeps/unix/sysv/linux/s390/force-elision.h: Likewise.
* sysdeps/unix/sysv/linux/s390/lowlevellock.h: Likewise.
* sysdeps/unix/sysv/linux/s390/Makefile: Remove references to
enable-lock-elision.
Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.vnet.ibm.com>
|
|
A note header has 3 4-bytes fields, followed by note name and note
descriptor. According to gABI, in a note entry, the note name field,
not note name size, is padded for the note descriptor. And the note
descriptor field, not note descriptor size, is padded for the next
note entry. Notes are aligned to 4 bytes in 32-bit objects and 8 bytes
in 64-bit objects.
For all GNU notes, the name is "GNU" which is 4 bytes. They have the
same format in the first 16 bytes in both 32-bit and 64-bit objects.
They differ by note descriptor size and note type. So far, .note.ABI-tag
and .note.gnu.build-id notes are always aligned to 4 bytes. The exsting
codes compute the note size by aligning the note name size and note
descriptor size to 4 bytes. It happens to produce the same value as
the actual note size by luck since the name size is 4 and offset of the
note descriptor is 16. But it will produce the wrong size when note
alignment is 8 bytes in 64-bit objects.
This patch defines ELF_NOTE_DESC_OFFSET and ELF_NOTE_NEXT_OFFSET to
properly compute offsets of note descriptor and next note. It uses
alignment of PT_NOTE segment to support both 4-byte and 8-byte note
alignments in 64-bit objects. To handle PT_NOTE segments with
incorrect alignment, which may lead to an infinite loop, if segment
alignment is less than 4, we treate alignment as 4 bytes since some
note segments have 0 or 1 byte alignment.
[BZ #22370]
* elf/dl-hwcaps.c (ROUND): Removed.
(_dl_important_hwcaps): Replace ROUND with ELF_NOTE_DESC_OFFSET
and ELF_NOTE_NEXT_OFFSET.
* elf/dl-load.c (ROUND): Removed.
(open_verify): Replace ROUND with ELF_NOTE_NEXT_OFFSET.
* elf/readelflib.c (ROUND): Removed.
(process_elf_file): Replace ROUND with ELF_NOTE_NEXT_OFFSET.
* include/elf.h [!_ISOMAC]: Include <libc-pointer-arith.h>.
[!_ISOMAC] (ELF_NOTE_DESC_OFFSET): New.
[!_ISOMAC] (ELF_NOTE_NEXT_OFFSET): Likewise.
|
|
Combine the four places where link maps are sorted into a single function.
This also moves the logic to skip the first map (representing the main
binary) to the callers.
|
|
To support Intel Control-flow Enforcement Technology (CET) run-time
control:
1. An architecture specific field in the writable ld.so namespace is
needed to indicate if CET features are enabled at run-time.
2. An architecture specific field in struct link_map is needed if
CET features are enabled in an ELF module.
This patch adds dl-procruntime.c to the writable ld.so namespace and
link_map.h to struct link_map.
Tested with build-many-glibcs.py.
* elf/dl-support.c: Include <dl-procruntime.c>.
* include/link.h: Include <link_map.h>.
* sysdeps/generic/dl-procruntime.c: New file.
* sysdeps/generic/link_map.h: Likewise.
* sysdeps/generic/ldsodefs.h: Include <dl-procruntime.c> in
the writable ld.so namespace.
|
|
These static functions are not needed if a target does not do lazy
tlsdesc initialization.
* elf/tlsdeschtab.h (_dl_tls_resolve_early_return_p): Mark unused.
(_dl_tlsdesc_wake_up_held_fixups): Likewise.
|
|
References remain in io/fts.c, io/ftw.c, posix/glob.c,
sysdeps/posix/getcwd.c. These files are (potentially) externally
shared.
|
|
Use $(LDFLAGS-$(@F)) with tst-tls1-static-non-pie may not be sufficient
when static PIE is built by default. Use $(DEFAULT-LDFLAGS-$(@F)) in
+link-static-before-libc to make sure that tst-tls1-static-non-pie is
always built as non-PIE static executable and make sure that crt1.o is
used with tst-tls1-static-non-pie.
* Makeconfig (+link-static-before-libc): Use
$(DEFAULT-LDFLAGS-$(@F)).
* elf/Makefile (CRT-tst-tls1-static-non-pie): New.
(LDFLAGS-tst-tls1-static-non-pie): Renamed to ...
(DEFAULT-LDFLAGS-tst-tls1-static-non-pie): This.
|
|
Verify that crt1.o can be used with main () in a shared object.
* elf/Makefile (tests): Add tst-main1.
(modules-names): Add tst-main1mod.
($(objpfx)tst-main1): New.
(CRT-tst-main1): Likewise.
(LDFLAGS-tst-main1): Likewise.
(LDLIBS-tst-main1): Likewise.
(tst-main1mod.so-no-z-defs): Likewise.
* elf/tst-main1.c: New file.
* elf/tst-main1mod.c: Likewise.
|
|
(&_dl_main_map) is used instead of (&bootstrap_map) to bootstrap static
PIE. Define BOOTSTRAP_MAP with (&_dl_main_map) to avoid hardcode to
(&bootstrap_map).
* elf/rtld.c (BOOTSTRAP_MAP): New.
(RESOLVE_MAP): Replace (&bootstrap_map) with BOOTSTRAP_MAP.
* sysdeps/hppa/dl-machine.h (ELF_MACHINE_BEFORE_RTLD_RELOC):
Likewise.
* sysdeps/ia64/dl-machine.h (ELF_MACHINE_BEFORE_RTLD_RELOC):
Likewise.
* sysdeps/mips/dl-machine.h (ELF_MACHINE_BEFORE_RTLD_RELOC):
Likewise.
|
|
__dl_iterate_phdr is hidden and should be accessed directly within
libc.so and libc.a without using GOT nor PLT.
[BZ #18822]
* elf/dl-iteratephdr.c (hidden_proto (__dl_iterate_phdr)): Moved
to ...
* include/link.h (hidden_proto (__dl_iterate_phdr)): Here.
|
|
tst-tls1-static-non-pie is built with $(no-pie-ldflag) to make it a
non-PIE static executable, regardless if --enable-static-pie is used
to configure glibc.
* elf/Makefile (tests-static-internal): Add
tst-tls1-static-non-pie.
(LDFLAGS-tst-tls1-static-non-pie): New.
* elf/tst-tls1-static-non-pie.c: New file.
|
|
Current implementation of tunables does not set arena_max and arena_test
values. Any value provided by glibc.malloc.arena_max and
glibc.malloc.arena_test parameters is ignored.
These tunables have minval value set to 1 (see elf/dl-tunables.list file)
and undefined maxval value. In that case default value (which is 0. see
scripts/gen-tunables.awk) is being used to set maxval.
For instance, generated tunable_list[] entry for arena_max is:
(gdb) p *cur
$1 = {name = 0x7ffff7df6217 "glibc.malloc.arena_max",
type = {type_code = TUNABLE_TYPE_SIZE_T, min = 1, max = 0},
val = {numval = 0, strval = 0x0}, initialized = false,
security_level = TUNABLE_SECLEVEL_SXID_IGNORE,
env_alias = 0x7ffff7df622e "MALLOC_ARENA_MAX"}
As a result, any value of glibc.malloc.arena_max is ignored by
TUNABLE_SET_VAL_IF_VALID_RANGE macro
__type min = (__cur)->type.min; <- initialized to 1
__type max = (__cur)->type.max; <- initialized to 0!
if (min == max) <- false
{
min = __default_min;
max = __default_max;
}
if ((__type) (__val) >= min && (__type) (val) <= max) <- false
{
(__cur)->val.numval = val;
(__cur)->initialized = true;
}
Assigning correct min/max values at a build time fixes a problem.
Plus, a bit of optimization: Setting of default min/max values for the
given type at a run time might be eliminated.
* elf/dl-tunables.c (do_tunable_update_val): Range checking fix.
* scripts/gen-tunables.awk: Set unspecified minval and/or maxval
values to correct default value for given type.
|
|
ELF objects generated with "objcopy --only-keep-debug" have
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
DYNAMIC 0x0+e28 0x0+200e40 0x0+200e40 0x0+ 0x0+1a0 RW 0x8
with 0 file size. ld.so should skip such PT_DYNAMIC segments.
Without a PT_DYNAMIC segment the loading of the shared object will
fail, and therefore ldd on such objects will also fail instead of
crashing. This provides better diagnostics for tooling that is
attempting to inspect the invalid shared objects which may just
contain debug information.
[BZ #22101]
* elf/Makefile (tests): Add tst-debug1.
($(objpfx)tst-debug1): New.
($(objpfx)tst-debug1.out): Likewise.
($(objpfx)tst-debug1mod1.so): Likewise.
* elf/dl-load.c (_dl_map_object_from_fd): Skip PT_DYNAMIC segment
with p_filesz == 0.
* elf/tst-debug1.c: New file.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
|
|
* elf/rtld-Rules: Fix $(error) use.
|
|
|
|
|
|
Some programs have more than one source files. These non-lib modules
should not be compiled with -DMODULE_NAME=libc. This patch puts these
non-lib modules in $(others-extras) and adds $(others-extras) to
all-nonlib.
[BZ #21864]
* Makerules (all-nonlib): Add $(others-extras).
* catgets/Makefile (others-extras): New.
* elf/Makefile (others-extras): Likewise.
* nss/Makefile (others-extras): Likewise.
|
|
Since __libc_multiple_libcs is defined as hidden symbol in init-first.c,
it should be always marked with attribute_hidden.
[BZ #18822]
* csu/libc-start.c (__libc_multiple_libcs): Removed.
* elf/dl-open.c: Include <libc-internal.h>.
(__libc_multiple_libcs): Removed.
* elf/dl-sysdep.c: Include <libc-internal.h> instead of
<hp-timing.h>.
* include/libc-internal.h (__libc_multiple_libcs): New.
* misc/sbrk.c: Include <libc-internal.h>.
(__libc_multiple_libcs): Removed.
|
|
Add macros used in GNU .note.gnu.property notes (NT_GNU_PROPERTY_TYPE_0).
* elf/elf.h (NT_GNU_PROPERTY_TYPE_0): New.
(NOTE_GNU_PROPERTY_SECTION_NAME): Likewise.
(GNU_PROPERTY_STACK_SIZE): Likewie.
(GNU_PROPERTY_NO_COPY_ON_PROTECTED): Likewie.
(GNU_PROPERTY_LOPROC): Likewise.
(GNU_PROPERTY_HIPROC): Likewise.
(GNU_PROPERTY_LOUSER): Likewise.
(GNU_PROPERTY_HIUSER): Likewise.
(GNU_PROPERTY_X86_ISA_1_USED): Likwise.
(GNU_PROPERTY_X86_ISA_1_NEEDED): Likwise.
(GNU_PROPERTY_X86_FEATURE_1_AND): Likwise.
(GNU_PROPERTY_X86_ISA_1_486): Likwise.
(GNU_PROPERTY_X86_ISA_1_586): Likwise.
(GNU_PROPERTY_X86_ISA_1_686): Likwise.
(GNU_PROPERTY_X86_ISA_1_SSE): Likwise.
(GNU_PROPERTY_X86_ISA_1_SSE2): Likwise.
(GNU_PROPERTY_X86_ISA_1_SSE3): Likwise.
(GNU_PROPERTY_X86_ISA_1_SSSE3): Likwise.
(GNU_PROPERTY_X86_ISA_1_SSE4_1): Likwise.
(GNU_PROPERTY_X86_ISA_1_SSE4_2): Likwise.
(GNU_PROPERTY_X86_ISA_1_AVX): Likwise.
(GNU_PROPERTY_X86_ISA_1_AVX2): Likwise.
(GNU_PROPERTY_X86_ISA_1_AVX512F): Likwise.
(GNU_PROPERTY_X86_ISA_1_AVX512CD): Likwise.
(GNU_PROPERTY_X86_ISA_1_AVX512ER): Likwise.
(GNU_PROPERTY_X86_ISA_1_AVX512PF): Likwise.
(GNU_PROPERTY_X86_ISA_1_AVX512VL): Likwise.
(GNU_PROPERTY_X86_ISA_1_AVX512DQ): Likwise.
(GNU_PROPERTY_X86_ISA_1_AVX512BW): Likwise.
(GNU_PROPERTY_X86_FEATURE_1_IBT): Likwise.
(GNU_PROPERTY_X86_FEATURE_1_SHSTK): Likwise.
|
|
|
|
Add powerpc specific note sections available since Linux v4.8.
* elf/elf.h A (NT_PPC_TAR): New macro.
(NT_PPC_PPR): Likewise.
(NT_PPC_DSCR): Likewise.
(NT_PPC_EBB): Likewise.
(NT_PPC_PMU): Likewise.
(NT_PPC_TM_CGPR): Likewise.
(NT_PPC_TM_CFPR): Likewise.
(NT_PPC_TM_CVMX): Likewise.
(NT_PPC_TM_CVSX): Likewise.
(NT_PPC_TM_SPR): Likewise.
(NT_PPC_TM_CTAR): Likewise.
(NT_PPC_TM_CPPR): Likewise.
(NT_PPC_TM_CDSCR): Likewise.
|
|
The function is called from the i386 startup code, which needs minor
adjustments due to the changed ABI.
|
|
The i386 startup code needs adjusting because it calls the function
and the ABI has changed.
|
|
Assembler code passes the address of _dl_fini to __libc_start_main,
whose function pointer argument lacks the attribute. This means
that calls could use the wrong ABI. Fortunately, for zero-parameter
void-returning functions, internal_function does not change ABI
on i386 (the only architecture which uses internal_function), so
this inconsistency was harmless (which is why it had not been
noticed so far).
|
|
These functions are invoked from other DSOs and should therefore
use the standard calling convention.
|
|
These are called across DSO boundaries and therefore should use
the ABI calling convention.
|
|
This commit separates allocating and raising exceptions. This
simplifies catching and re-raising them because it is no longer
necessary to make a temporary, on-stack copy of the exception message.
|
|
On Linux/i386, there are 3 ways to make a system call:
1. call *%gs:SYSINFO_OFFSET. This requires TLS initialization.
2. call *_dl_sysinfo. This requires relocation of _dl_sysinfo.
3. int $0x80. This is slower than #2 and #3, but works everywhere.
When an object file is compiled with PIC, #1 is prefered since it is
faster than #3 and doesn't require relocation of _dl_sysinfo. For
dynamic executables, ld.so initializes TLS. However, for static
executables, before TLS is initialized by __libc_setup_tls, #3 should
be used for system calls.
This patch adds <startup.h> which defines _startup_fatal and defaults
it to __libc_fatal. It replaces __libc_fatal with _startup_fatal in
static executables where it is called before __libc_setup_tls is called.
This header file is included in all files containing functions which are
called before __libc_setup_tls is called. On Linux/i386, when PIE is
enabled by default, _startup_fatal is turned into ABORT_INSTRUCTION and
I386_USE_SYSENTER is defined to 0 so that "int $0x80" is used for system
calls before __libc_setup_tls is called.
Tested on i686 and x86-64. Without this patch, all statically-linked
tests will fail on i686 when the compiler defaults to -fPIE.
[BZ #21913]
* csu/libc-tls.c: Include <startup.h> first.
(__libc_setup_tls): Call _startup_fatal instead of __libc_fatal.
* elf/dl-tunables.c: Include <startup.h> first.
* include/libc-symbols.h (BUILD_PIE_DEFAULT): New.
* sysdeps/generic/startup.h: New file.
* sysdeps/unix/sysv/linux/i386/startup.h: Likewise.
* sysdeps/unix/sysv/linux/i386/brk.c [BUILD_PIE_DEFAULT != 0]
(I386_USE_SYSENTER): New. Defined to 0.
|
|
|
|
tst-prelink.c checks for conflict with GLOB_DAT relocation against stdio.
On i386, there is no GLOB_DAT relocation against stdio with PIE. We
should compile tst-prelink.c without PIE.
[BZ #21815]
* elf/Makefile (CFLAGS-tst-prelink.c): New.
(LDFLAGS-tst-prelink): Likewise.
|
|
Gold doesn't support protected data symbol:
configure:5672: checking linker support for protected data symbol
configure:5682: gcc -fuse-ld=gold -nostdlib -nostartfiles -fno-stack-protector -fPIC -shared conftest.c -o conftest.so
configure:5685: $? = 0
configure:5692: gcc -fuse-ld=gold -nostdlib -nostartfiles -fno-stack-protector conftest.c -o conftest conftest.so
/usr/local/bin/ld.gold: error: /tmp/ccXWoofs.o: cannot make copy relocation for protected symbol 'bar', defined in conftest.so
collect2: error: ld returned 1 exit status
Run vismain only if linker supports protected data symbol.
* elf/Makefile (tests): Add vismain only if
$(have-protected-data) == yes.
(tests-pie): Likewise.
|
|
The function maybe_enable_malloc_check, which is called by
__tunables_init, calls __access_noerrno. It isn't problem when
symbol is is in ld.so, which has a special version of __access_noerrno
without stack protector. But when glibc is built with stack protector,
maybe_enable_malloc_check in libc.a can't call the regular version of
__access_noerrno with stack protector.
This patch changes how Linux defines the __access_noerrno to be an
inline call instead and thus preventing defining different build
rules for ld/static and shared.
H.J. Lu <hongjiu.lu@intel.com>
Adhemerval Zanella <adhemerval.zanella@linaro.org>
[BZ #21744]
* elf/dl-tunables.c: Include not-errno.h header.
* include/unistd.h (__access_noerrno): Remove definition.
* sysdeps/unix/sysv/linux/access.c (__access_noerrno): Likewise.
* sysdeps/generic/not-errno.h: New file.
* sysdeps/unix/sysv/linux/not-errno.h: Likewise.
|
|
The patch proposed by Peter Bergner [1] to libgcc in order to fix
[BZ #21707] adds a dependency on a symbol provided by the loader,
forcing the loader to be linked to tests after libgcc was linked.
It also requires to read the thread pointer during IRELA relocations.
Tested on powerpc, powerpc64, powerpc64le, s390x and x86_64.
[1] https://sourceware.org/ml/libc-alpha/2017-06/msg01383.html
[BZ #21707]
* csu/libc-start.c (LIBC_START_MAIN): Perform IREL{,A}
relocations before or after initializing the TCB on statically
linked executables. That's a per-architecture definition.
* elf/rtld.c (dl_main): Add a comment about thread-local
variables initialization.
* sysdeps/generic/libc-start.h: New file. Define
ARCH_APPLY_IREL and ARCH_SETUP_IREL.
* sysdeps/powerpc/Makefile:
[$(subdir) = elf && $(multi-arch) != no] (tests-static-internal): Add tst-tlsifunc-static.
[$(subdir) = elf && $(multi-arch) != no && $(build-shared) == yes]
(tests-internal): Add tst-tlsifunc.
* sysdeps/powerpc/tst-tlsifunc.c: New file.
* sysdeps/powerpc/tst-tlsifunc-static.c: Likewise.
* sysdeps/powerpc/powerpc64le/Makefile (f128-loader-link): New
variable.
[$(subdir) = math] (test-float128% test-ifloat128%): Force
linking to the loader after linking to libgcc.
[$(subdir) = wcsmbs || $(subdir) = stdlib] (bug-strtod bug-strtod2)
(bug-strtod2 tst-strtod-round tst-wcstod-round tst-strtod6 tst-strrom)
(tst-strfrom-locale strfrom-skeleton): Likewise.
* sysdeps/unix/sysv/linux/powerpc/libc-start.h: New file. Define
ARCH_APPLY_IREL and ARCH_SETUP_IREL.
|
|
Since _dl_resolve_conflicts is only used in elf/rtld.c, don't include
it in libc.a.
[BZ #21742]
* elf/Makefile (dl-routines): Move dl-conflict to ...
(rtld-routines): Here.
|
|
* config.make.in: Enable experimental malloc option.
* configure.ac: Likewise.
* configure: Regenerate.
* manual/install.texi: Document it.
* INSTALL: Regenerate.
* malloc/Makefile: Likewise.
* malloc/malloc.c: Add per-thread cache (tcache).
(tcache_put): New.
(tcache_get): New.
(tcache_thread_freeres): New.
(tcache_init): New.
(__libc_malloc): Use cached chunks if available.
(__libc_free): Initialize tcache if needed.
(__libc_realloc): Likewise.
(__libc_calloc): Likewise.
(_int_malloc): Prefill tcache when appropriate.
(_int_free): Likewise.
(do_set_tcache_max): New.
(do_set_tcache_count): New.
(do_set_tcache_unsorted_limit): New.
* manual/probes.texi: Document new probes.
* malloc/arena.c: Add new tcache tunables.
* elf/dl-tunables.list: Likewise.
* manual/tunables.texi: Document them.
* NEWS: Mention the per-thread cache.
|
|
Add a new tunable (glibc.tune.cpu) to override CPU identification on
aarch64. This is useful in two cases: one where it is desirable to
pretend to be another CPU for purposes of testing or because routines
written for that CPU are beneficial for specific workloads and second
where the underlying kernel does not support emulation of MRS to get
the MIDR of the CPU.
* elf/dl-tunables.h (tunable_is_name): Move from...
* elf/dl-tunables.c (is_name): ... here.
(parse_tunables, __tunables_init): Adjust.
* manual/tunables.texi: Document glibc.tune.cpu.
* sysdeps/aarch64/dl-tunables.list: New file.
* sysdeps/unix/sysv/linux/aarch64/cpu-features.c (struct
cpu_list): New type.
(cpu_list): New list of CPU names and their MIDR.
(get_midr_from_mcpu): New function.
(init_cpu_features): Override MIDR if necessary.
|
|
Building the testsuite with current GCC mainline fails with:
loadtest.c: In function 'main':
loadtest.c:76:3: error: macro expands to multiple statements [-Werror=multistatement-macros]
for (map = MAPS; map != NULL; map = map->l_next) \
^
loadtest.c:165:2: note: in expansion of macro 'OUT'
OUT;
^~~
loadtest.c:164:7: note: some parts of macro expansion are not guarded by this 'if' clause
if (debug)
^~
This seems like a genuine bug, although fairly harmless; it means the
fflush call in the OUT macro is unconditional instead of being inside
the conditional as presumably intended. This patch makes this macro
use do { } while (0) to avoid the problem.
Tested for x86_64 (testsuite), and with build-many-glibcs.py for
aarch64-linux-gnu with GCC mainline.
* elf/loadtest.c (OUT): Define using do { } while (0).
|
|
Rename glibc.tune.ifunc to glibc.tune.hwcaps and move it to
sysdeps/x86/dl-tunables.list since it is x86 specicifc. Also
change type of data_cache_size, data_cache_size and
non_temporal_threshold to unsigned long int to match size_t.
Remove usage DEFAULT_STRLEN from cpu-tunables.c.
* elf/dl-tunables.list (glibc.tune.ifunc): Removed.
* sysdeps/x86/dl-tunables.list (glibc.tune.hwcaps): New.
Remove security_level on all fields.
* manual/tunables.texi: Replace ifunc with hwcaps.
* sysdeps/x86/cpu-features.c (TUNABLE_CALLBACK (set_ifunc)):
Renamed to ..
(TUNABLE_CALLBACK (set_hwcaps)): This.
(init_cpu_features): Updated.
* sysdeps/x86/cpu-features.h (cpu_features): Change type of
data_cache_size, data_cache_size and non_temporal_threshold to
unsigned long int.
* sysdeps/x86/cpu-tunables.c (DEFAULT_STRLEN): Removed.
(TUNABLE_CALLBACK (set_ifunc)): Renamed to ...
(TUNABLE_CALLBACK (set_hwcaps)): This. Update comments. Don't
use DEFAULT_STRLEN.
|
|
* elf/dl-tunables.list: Move x86 specific tunables to ...
* sysdeps/x86/dl-tunables.list: Here. New file.
|
|
The current IFUNC selection is based on microbenchmarks in glibc. It
should give the best performance for most workloads. But other choices
may have better performance for a particular workload or on the hardware
which wasn't available at the selection was made. The environment
variable, GLIBC_TUNABLES=glibc.tune.ifunc=-xxx,yyy,-zzz...., can be used
to enable CPU/ARCH feature yyy, disable CPU/ARCH feature yyy and zzz,
where the feature name is case-sensitive and has to match the ones in
cpu-features.h. It can be used by glibc developers to override the
IFUNC selection to tune for a new processor or improve performance for
a particular workload. It isn't intended for normal end users.
NOTE: the IFUNC selection may change over time. Please check all
multiarch implementations when experimenting.
Also, GLIBC_TUNABLES=glibc.tune.x86_non_temporal_threshold=NUMBER is
provided to set threshold to use non temporal store to NUMBER,
GLIBC_TUNABLES=glibc.tune.x86_data_cache_size=NUMBER to set data cache
size, GLIBC_TUNABLES=glibc.tune.x86_shared_cache_size=NUMBER to set
shared cache size.
* elf/dl-tunables.list (tune): Add ifunc,
x86_non_temporal_threshold,
x86_data_cache_size and x86_shared_cache_size.
* manual/tunables.texi: Document glibc.tune.ifunc,
glibc.tune.x86_data_cache_size, glibc.tune.x86_shared_cache_size
and glibc.tune.x86_non_temporal_threshold.
* sysdeps/unix/sysv/linux/x86/dl-sysdep.c: New file.
* sysdeps/x86/cpu-tunables.c: Likewise.
* sysdeps/x86/cacheinfo.c
(init_cacheinfo): Check and get data cache size, shared cache
size and non temporal threshold from cpu_features.
* sysdeps/x86/cpu-features.c [HAVE_TUNABLES] (TUNABLE_NAMESPACE):
New.
[HAVE_TUNABLES] Include <unistd.h>.
[HAVE_TUNABLES] Include <elf/dl-tunables.h>.
[HAVE_TUNABLES] (TUNABLE_CALLBACK (set_ifunc)): Likewise.
[HAVE_TUNABLES] (init_cpu_features): Use TUNABLE_GET to set
IFUNC selection, data cache size, shared cache size and non
temporal threshold.
* sysdeps/x86/cpu-features.h (cpu_features): Add data_cache_size,
shared_cache_size and non_temporal_threshold.
|
|
Also only process the last LD_AUDIT entry.
|
|
|
|
LD_LIBRARY_PATH can only be used to reorder system search paths, which
is not useful functionality.
This makes an exploitable unbounded alloca in _dl_init_paths unreachable
for AT_SECURE=1 programs.
|
|
Since _dl_out_of_memory is static in elf/dl-error-skeleton.c:
static const char _dl_out_of_memory[] = "out of memory";
remove _dl_out_of_memory from elf/Versions.
* elf/Versions (ld): Remove _dl_out_of_memory.
|
|
ELFv2 functions with localentry:0 are those with a single entry point,
ie. global entry == local entry, that have no requirement on r2 or
r12 and guarantee r2 is unchanged on return. Such an external
function can be called via the PLT without saving r2 or restoring it
on return, avoiding a common load-hit-store for small functions.
This patch implements the ld.so changes necessary for this
optimization. ld.so needs to check that an optimized plt call
sequence is in fact calling a function implemented with localentry:0,
end emit a fatal error otherwise.
The elf/testobj6.c change is to stop "error while loading shared
libraries: expected localentry:0 `preload'" when running
elf/preloadtest, which we'd get otherwise.
* elf/elf.h (PPC64_OPT_LOCALENTRY): Define.
* sysdeps/alpha/dl-machine.h (elf_machine_fixup_plt): Add
refsym and sym parameters. Adjust callers.
* sysdeps/aarch64/dl-machine.h (elf_machine_fixup_plt): Likewise.
* sysdeps/arm/dl-machine.h (elf_machine_fixup_plt): Likewise.
* sysdeps/generic/dl-machine.h (elf_machine_fixup_plt): Likewise.
* sysdeps/hppa/dl-machine.h (elf_machine_fixup_plt): Likewise.
* sysdeps/i386/dl-machine.h (elf_machine_fixup_plt): Likewise.
* sysdeps/ia64/dl-machine.h (elf_machine_fixup_plt): Likewise.
* sysdeps/m68k/dl-machine.h (elf_machine_fixup_plt): Likewise.
* sysdeps/microblaze/dl-machine.h (elf_machine_fixup_plt): Likewise.
* sysdeps/mips/dl-machine.h (elf_machine_fixup_plt): Likewise.
* sysdeps/nios2/dl-machine.h (elf_machine_fixup_plt): Likewise.
* sysdeps/powerpc/powerpc32/dl-machine.h (elf_machine_fixup_plt):
Likewise.
* sysdeps/s390/s390-32/dl-machine.h (elf_machine_fixup_plt): Likewise.
* sysdeps/s390/s390-64/dl-machine.h (elf_machine_fixup_plt): Likewise.
* sysdeps/sh/dl-machine.h (elf_machine_fixup_plt): Likewise.
* sysdeps/sparc/sparc32/dl-machine.h (elf_machine_fixup_plt): Likewise.
* sysdeps/sparc/sparc64/dl-machine.h (elf_machine_fixup_plt): Likewise.
* sysdeps/tile/dl-machine.h (elf_machine_fixup_plt): Likewise.
* sysdeps/x86_64/dl-machine.h (elf_machine_fixup_plt): Likewise.
* sysdeps/powerpc/powerpc64/dl-machine.c (_dl_error_localentry): New.
(_dl_reloc_overflow): Increase buffser size. Formatting.
* sysdeps/powerpc/powerpc64/dl-machine.h (ppc64_local_entry_offset):
Delete reloc param, add refsym and sym. Check optimized plt
call stubs for localentry:0 functions. Adjust callers.
(elf_machine_fixup_plt, elf_machine_plt_conflict): Add refsym
and sym parameters. Adjust callers.
(_dl_reloc_overflow): Move attribute.
(_dl_error_localentry): Declare.
* elf/dl-runtime.c (_dl_fixup): Save original sym. Pass
refsym and sym to elf_machine_fixup_plt.
* elf/testobj6.c (preload): Call printf.
|
|
There are 2 minimal strtoul implementations in ld.so:
1. __strtoul_internal in elf/dl-minimal.c.
2. tunables_strtoul in elf/dl-tunables.c.
This patch adds _dl_strtoul to replace them. Tested builds with and
without --enable-tunables.
[BZ #21528]
* elf/dl-minimal.c (__strtoul_internal): Removed.
(strtoul): Likewise.
* elf/dl-misc.c (_dl_strtoul): New function.
* elf/dl-tunables.c (tunables_strtoul): Removed.
(tunable_initialize): Replace tunables_strtoul with _dl_strtoul.
* elf/rtld.c (process_envvars): Likewise.
* sysdeps/unix/sysv/linux/dl-librecon.h (_dl_osversion_init):
Likewise.
* sysdeps/generic/ldsodefs.h (_dl_strtoul): New prototype.
|
|
The LD_HWCAP_MASK environment variable was ignored in static binaries,
which is inconsistent with the behaviour of dynamically linked
binaries. This seems to have been because of the inability of
ld_hwcap_mask being read early enough to influence anything but now
that it is in tunables, the mask is usable in static binaries as well.
This feature is important for aarch64, which relies on HWCAP_CPUID
being masked out to disable multiarch. A sanity test on x86_64 shows
that there are no failures. Likewise for aarch64.
* elf/dl-hwcaps.h [HAVE_TUNABLES]: Always read hwcap_mask.
* sysdeps/sparc/sparc32/dl-machine.h [HAVE_TUNABLES]:
Likewise.
* sysdeps/x86/cpu-features.c (init_cpu_features): Always set
up hwcap and hwcap_mask.
|
|
Drop _dl_hwcap_mask when building with tunables. This completes the
transition of hwcap_mask reading from _dl_hwcap_mask to tunables.
* elf/dl-hwcaps.h: New file.
* elf/dl-hwcaps.c: Include it.
(_dl_important_hwcaps)[HAVE_TUNABLES]: Read and update
glibc.tune.hwcap_mask.
* elf/dl-cache.c: Include dl-hwcaps.h.
(_dl_load_cache_lookup)[HAVE_TUNABLES]: Read
glibc.tune.hwcap_mask.
* sysdeps/sparc/sparc32/dl-machine.h: Likewise.
* elf/dl-support.c (_dl_hwcap2)[HAVE_TUNABLES]: Drop
_dl_hwcap_mask.
* elf/rtld.c (rtld_global_ro)[HAVE_TUNABLES]: Drop
_dl_hwcap_mask.
(process_envvars)[HAVE_TUNABLES]: Likewise.
* sysdeps/generic/ldsodefs.h (rtld_global_ro)[HAVE_TUNABLES]:
Likewise.
* sysdeps/x86/cpu-features.c (init_cpu_features): Don't
initialize dl_hwcap_mask when tunables are enabled.
|
|
Add LD_HWCAP_MASK to tunables in preparation of it being removed from
rtld.c. This allows us to read LD_HWCAP_MASK much earlier so that it
can influence IFUNC resolution in aarch64.
This patch does not actually do anything other than read the
LD_HWCAP_MASK variable and add the tunables way to set the
LD_HWCAP_MASK, i.e. via the glibc.tune.hwcap_mask tunable. In a
follow-up patch, the _dl_hwcap_mask will be replaced with
glibc.tune.hwcap_mask to complete the transition.
* elf/dl-tunables.list: Add glibc.tune.hwcap_mask.
* scripts/gen-tunables.awk: Include dl-procinfo.h.
* manual/tunables.texi: Document glibc.tune.hwcap_mask.
|
|
The TUNABLE_SET_VALUE and family of macros (and my later attempt to
add a TUNABLE_GET) never quite went together very well because the
overall interface was not clearly defined. This patch is an attempt
to do just that.
This patch consolidates the API to two simple sets of macros,
TUNABLE_GET* and TUNABLE_SET*. If TUNABLE_NAMESPACE is defined,
TUNABLE_GET takes just the tunable name, type and a (optionally NULL)
callback function to get the value of the tunable. The callback
function, if non-NULL, is called if the tunable was externally set
(i.e. via GLIBC_TUNABLES or any future mechanism). For example:
val = TUNABLE_GET (check, int32_t, check_callback)
returns the value of the glibc.malloc.check tunable (assuming
TUNABLE_NAMESPACE is set to malloc) as an int32_t into VAL after
calling check_callback.
Likewise, TUNABLE_SET can be used to set the value of the tunable,
although this is currently possible only in the dynamic linker before
it relocates itself. For example:
TUNABLE_SET (check, int32_t, 2)
will set glibc.malloc.check to 2. Of course, this is not possible
since we set (or read) glibc.malloc.check long after it is relocated.
To access or set a tunable outside of TUNABLE_NAMESPACE, use the
TUNABLE_GET_FULL and TUNABLE_SET_FULL macros, which have the following
prototype:
TUNABLE_GET_FULL (glibc, tune, hwcap_mask, uint64_t, NULL)
TUNABLE_SET_FULL (glibc, tune, hwcap_mask, uint64_t, 0xffff)
In future the tunable list may get split into mutable and immutable
tunables where mutable tunables can be modified by the library and
userspace after relocation as well and TUNABLE_SET will be more useful
than it currently is. However whenever we actually do that split, we
will have to ensure that the mutable tunables are protected with
locks.
* elf/Versions (__tunable_set_val): Rename to __tunable_get_val.
* elf/dl-tunables.c: Likewise.
(do_tunable_update_val): New function.
(__tunable_set_val): New function.
(__tunable_get_val): Call CB only if the tunable was externally
initialized.
(tunables_strtoul): Replace strval with initialized.
* elf/dl-tunables.h (strval): Replace with a bool initialized.
(TUNABLE_ENUM_NAME, TUNABLE_ENUM_NAME1): Adjust names to
prevent collision.
(__tunable_set_val): New function.
(TUNABLE_GET, TUNABLE_GET_FULL): New macros.
(TUNABLE_SET, TUNABLE_SET_FULL): Likewise.
(TUNABLE_SET_VAL): Remove.
(TUNABLE_SET_VAL_WITH_CALLBACK): Likewise.
* README.tunables: Document the new macros.
* malloc/arena.c (ptmalloc_init): Adjust.
|
