diff options
| author | Samuel Thibault <samuel.thibault@ens-lyon.org> | 2016-10-09 19:04:57 +0200 |
|---|---|---|
| committer | Samuel Thibault <samuel.thibault@ens-lyon.org> | 2016-10-09 19:04:57 +0200 |
| commit | 7bb5f8a836b916d6ebf7b6921b136e99cea2442d (patch) | |
| tree | 28a7ed786dae726ad14f100e8626eee872b1ba11 /sysdeps/s390 | |
| parent | f76453c31593957fec1a99b986bfa5506618b79c (diff) | |
| parent | ab30899d880f9741a409cbc0d7a28399bdac21bf (diff) | |
Merge tag 'glibc-2.23' into baseline
The GNU C Library
=================
The GNU C Library version 2.23 is now available.
The GNU C Library is used as *the* C library in the GNU system and
in GNU/Linux systems, as well as many other systems that use Linux
as the kernel.
The GNU C Library is primarily designed to be a portable
and high performance C library. It follows all relevant
standards including ISO C11 and POSIX.1-2008. It is also
internationalized and has one of the most complete
internationalization interfaces known.
The GNU C Library webpage is at http://www.gnu.org/software/libc/
Packages for the 2.23 release may be downloaded from:
http://ftpmirror.gnu.org/libc/
http://ftp.gnu.org/gnu/libc/
The mirror list is at http://www.gnu.org/order/ftp.html
NEWS for version 2.23
=====================
* Unicode 8.0.0 Support: Character encoding, character type info, and
transliteration tables are all updated to Unicode 8.0.0, using new
and/or improved generator scripts contributed by Mike FABIAN (Red Hat).
These updates cause user visible changes, such as the fixes for bugs
89, 16061, and 18568.
* sched_setaffinity, pthread_setaffinity_np no longer attempt to guess the
kernel-internal CPU set size. This means that requests that change the
CPU affinity which failed before (for example, an all-ones CPU mask) will
now succeed. Applications that need to determine the effective CPU
affinities need to call sched_getaffinity or pthread_getaffinity_np after
setting it because the kernel can adjust it (and the previous size check
would not detect this in the majority of cases).
* The fts.h header can now be used with -D_FILE_OFFSET_BITS=64. With LFS
the following new symbols are used: fts64_children, fts64_close,
fts64_open, fts64_read and fts64_set.
* getaddrinfo now detects certain invalid responses on an internal netlink
socket. If such responses are received, an affected process will
terminate with an error message of "Unexpected error <number> on netlink
descriptor <number>" or "Unexpected netlink response of size <number> on
descriptor <number>". The most likely cause for these errors is a
multi-threaded application which erroneously closes and reuses the netlink
file descriptor while it is used by getaddrinfo.
* A defect in the malloc implementation, present since glibc 2.15 (2012) or
glibc 2.10 via --enable-experimental-malloc (2009), could result in the
unnecessary serialization of memory allocation requests across threads.
The defect is now corrected. Users should see a substantial increase in
the concurent throughput of allocation requests for applications which
trigger this bug. Affected applications typically create create and
destroy threads frequently. (Bug 19048 was reported and analyzed by
Ericsson.)
* There is now a --disable-timezone-tools configure option for disabling the
building and installing of the timezone related utilities (zic, zdump, and
tzselect). This is useful for people who build the timezone data and code
independent of the GNU C Library.
* The obsolete header <regexp.h> has been removed. Programs that require
this header must be updated to use <regex.h> instead.
* The obsolete functions bdflush, create_module, get_kernel_syms,
query_module and uselib are no longer available to newly linked binaries;
the header <sys/kdaemon.h> has been removed. These functions and header
were specific to systems using the Linux kernel and could not usefully be
used with the GNU C Library on systems with version 2.6 or later of the
Linux kernel.
* Optimized string, wcsmbs and memory functions for IBM z13.
Implemented by Stefan Liebler.
* Newly linked programs that define a variable called signgam will no longer
have it set by the lgamma, lgammaf and lgammal functions. Programs that
require signgam to be set by those functions must ensure that they use the
variable provided by the GNU C Library and declared in <math.h>, without
defining their own copy.
* The minimum GCC version that can be used to build this version of the GNU
C Library is GCC 4.7. Older GCC versions, and non-GNU compilers, can
still be used to compile programs using the GNU C Library.
Security related changes:
* An out-of-bounds value in a broken-out struct tm argument to strftime no
longer causes a crash. Reported by Adam Nielsen. (CVE-2015-8776)
* The LD_POINTER_GUARD environment variable can no longer be used to disable
the pointer guard feature. It is always enabled. Previously,
LD_POINTER_GUARD could be used to disable security hardening in binaries
running in privileged AT_SECURE mode. Reported by Hector Marco-Gisbert.
(CVE-2015-8777)
* An integer overflow in hcreate and hcreate_r could lead to an
out-of-bounds memory access. Reported by Szabolcs Nagy. (CVE-2015-8778)
* The catopen function no longer has unbounded stack usage. Reported by
Max. (CVE-2015-8779)
* The nan, nanf and nanl functions no longer have unbounded stack usage
depending on the length of the string passed as an argument to the
functions. Reported by Joseph Myers. (CVE-2014-9761)
* A stack-based buffer overflow was found in libresolv when invoked from
libnss_dns, allowing specially crafted DNS responses to seize control
of execution flow in the DNS client. The buffer overflow occurs in
the functions send_dg (send datagram) and send_vc (send TCP) for the
NSS module libnss_dns.so.2 when calling getaddrinfo with AF_UNSPEC
family. The use of AF_UNSPEC triggers the low-level resolver code to
send out two parallel queries for A and AAAA. A mismanagement of the
buffers used for those queries could result in the response of a query
writing beyond the alloca allocated buffer created by
_nss_dns_gethostbyname4_r. Buffer management is simplified to remove
the overflow. Thanks to the Google Security Team and Red Hat for
reporting the security impact of this issue, and Robert Holiday of
Ciena for reporting the related bug 18665. (CVE-2015-7547)
The following bugs are resolved with this release:
[89] localedata: Locales nb_NO and nn_NO should transliterate æøå
[887] math: Math library function "logb" and "nextafter" inconsistent
[2542] math: Incorrect return from float gamma (-0X1.FA471547C2FE5P+1)
[2543] math: Incorrect return from float gamma (-0X1.9260DCP+1)
[2558] math: Incorrect return from double gamma (-0X1.FA471547C2FE5P+1)
[2898] libc: [improve] warning: the use of `mktemp' is dangerous, better
use `mkstemp'
[4404] localedata: German translation of "Alarm clock" is misleading
[6799] math: nextafter() and nexttoward() doen't set errno on
overflow/underflow errors
[6803] math: scalb(), scalbln(), scalbn() do not set errno on
overflow/underflow
[10432] nis: _nss_nis_setnetgrent assertion failure
[11460] libc: fts has no LFS support
[12926] network: getaddrinfo()/make_request() may spin forever
[13065] nptl: Race condition in pthread barriers
[13690] nptl: pthread_mutex_unlock potentially cause invalid access
[14341] dynamic-link: Dynamic linker crash when DT_JMPREL and DT_REL{,A}
are not contiguous
[14551] math: [ldbl-128ibm] strtold overflow handling for IBM long double
[14912] libc: Rename non-installed bits/*.h headers
[15002] libc: Avoid undefined behavior in posix_fallocate overflow check
[15367] math: Let gcc use __builtin_isinf
[15384] math: One constant fewer in ieee754/dbl-64/wordsize-64/s_finite.c
[15421] math: lgamma wrongly sets signgam for ISO C
[15470] math: [arm] On ARM llrintl() and llroundl() do not raise
FE_INVALID with argument out of range
[15491] math: [i386/x86_64] x86 nearbyint implementations wrongly clear
all exceptions
[15786] dynamic-link: ifunc resolver functions can smash function
arguments
[15918] math: Unnecessary check for equality in hypotf()
[16061] localedata: Review / update transliteration data
[16068] math: [i386/x86_64] x86 and x86_64 fesetenv exclude state they
should include
[16141] time: strptime %z offset restriction
[16171] math: drem should be alias of remainder
[16296] math: fegetround is pure?
[16347] math: [ldbl-128ibm] ldbl-128/e_lgammal_r.c may not be suitable.
[16364] libc: sleep may leave SIGCHLD blocked on sync cancellation on
GNU/Linux
[16399] math: [mips] lrint / llrint / lround / llround missing exceptions
[16415] math: Clean up ldbl-128 / ldbl-128ibm expm1l for large positive
arguments
[16422] math: [powerpc] math-float, math-double failing llrint tests with
"Exception "Inexact" set" on ppc32
[16495] localedata: nl_NL: date_fmt: shuffle year/month around
[16517] math: Missing underflow exception from tanf/tan/tanl
[16519] math: Missing underflow exception from sinhf
[16520] math: Missing underflow exception from tanhf
[16521] math: Missing underflow exception from exp2
[16620] math: [ldbl-128ibm] exp10l spurious overflows / bad directed
rounding results
[16734] stdio: fopen calls mmap to allocate its buffer
[16961] math: nan function incorrect handling of bad sequences
[16962] math: nan function unbounded stack allocation (CVE-2014-9761)
[16973] localedata: Fix lang_lib/lang_term as per ISO 639-2
[16985] locale: localedef: confusing error message when opening output
fails
[17118] math: ctanh(INFINITY + 2 * I) returns incorrect value
[17197] locale: Redundant shift character in iconv conversion output at
block boundary
[17243] libc: trunk/posix/execl.c:53: va_args problem ?
[17244] libc: trunk/sysdeps/unix/sysv/linux/semctl.c:116: va_args muxup ?
[17250] dynamic-link: static linking breaks nss loading
(getaddrinfo/getpwnam/etc...)
[17404] libc: atomic_exchange_rel lacking a barrier on MIPS16, GCC before
4.7?
[17441] math: isnan() should use __builtin_isnan() in GCC
[17514] nptl: Assert failure unlocking ERRORCHECK mutex after timedlock
(related to lock elision)
[17787] manual: Exponent on page 324 of the PDF ends prematurely
[17886] time: strptime should be able to parse "Z" as a timezone with %z
[17887] time: strptime should be able to parse "+01:00" style timezones
[17905] libc: catopen() Multiple unbounded stack allocations
(CVE-2015-8779)
[18084] libc: backtrace (..., 0) dumps core on x86
[18086] libc: nice() sets errno to 0 on success
[18240] libc: hcreate, hcreate_r should fail with ENOMEM if element count
is too large (CVE-2015-8778)
[18251] dynamic-link: SONAME missing when audit modules provides path
[18265] libc: add attributes for wchar string and memory functions
[18370] math: csqrt missing underflows
[18421] libc: [hppa] read-only segment has dynamic relocations
[18472] libc: Obsolete syscall wrappers should be compat symbols
[18480] libc: hppa glibc miscompilation in sched_setaffinity()
[18491] localedata: Update tr_TR LC_CTYPE as part of Unicode updates
[18525] localedata: Remove locale timezone information
[18560] libc: [powerpc] spurious bits/ipc.h definitions
[18568] localedata: Update locale data to Unicode 8.0
[18589] locale: sort-test.sh fails at random
[18595] math: ctan, ctanh missing underflows
[18604] libc: assert macro-expands its argument
[18610] math: S390: fetestexcept() reports any exception if DXC-code
contains a vector instruction exception.
[18611] math: j1, jn missing errno setting on underflow
[18618] localedata: sync Chechen locale definitions with other *_RU
locales
[18647] math: powf(-0x1.000002p0, 0x1p30) returns 0 instead of +inf
[18661] libc: Some x86-64 assembly codes don't align stack to 16 bytes
[18665] network: In send_dg, the recvfrom function is NOT always using the
buffer size of a newly created buffer (CVE-2015-7547)
[18674] libc: [i386] trunk/sysdeps/i386/tst-auditmod3b.c:84: possible
missing break ?
[18675] libc: fpathconf(_PC_NAME_MAX) fails against large filesystems for
32bit processes
[18681] libc: regexp.h is obsolete and buggy, and should be desupported
[18699] math: tilegx cproj() for various complex infinities does not yield
infinity
[18724] libc: Harden put*ent functions against data injection
[18743] nptl: PowerPC: findutils testcase fails with --enable-lock-elision
[18755] build: build errors with -DNDEBUG
[18757] stdio: fmemopen fails to set errno on failure
[18778] dynamic-link: ld.so crashes if failed dlopen causes libpthread to
be forced unloaded
[18781] libc: openat64 lacks O_LARGEFILE
[18787] libc: [hppa] sysdeps/unix/sysv/linux/hppa/bits/atomic.h:71:6:
error: can’t find a register in class ‘R1_REGS’ while reloading ‘asm’
[18789] math: [ldbl-128ibm] sinhl inaccurate near 0
[18790] math: [ldbl-128ibm] tanhl inaccurate
[18795] libc: stpncpy fortification misses buffer lengths that are
statically too large
[18796] build: build fails for --disable-mathvec
[18803] math: hypot missing underflows
[18820] stdio: fmemopen may leak memory on failure
[18823] math: csqrt spurious underflows
[18824] math: fma spurious underflows
[18825] math: pow missing underflows
[18857] math: [ldbl-128ibm] nearbyintl wrongly uses signaling comparisons
[18868] nptl: pthread_barrier_init typo has in-theory-undefined behavior
[18870] build: sem_open.c fails to compile with missing symbol
FUTEX_SHARED
[18872] stdio: Fix memory leak in printf_positional
[18873] libc: posix_fallocate overflow check ineffective
[18875] math: Excess precision leads incorrect libm
[18877] libc: arm: mmap offset regression
[18887] libc: memory corruption when using getmntent on blank lines
[18918] localedata: hu_HU: change time to HH:MM:SS format
[18921] libc: Regression: extraneous stat() and fstat() performed by
opendir()
[18928] dynamic-link: LD_POINTER_GUARD is not ignored for privileged
binaries (CVE-2015-8777)
[18951] math: tgamma missing underflows
[18952] math: [ldbl-128/ldbl-128ibm] lgammal spurious "invalid", incorrect
signgam
[18953] localedata: lt_LT: change currency symbol to the euro
[18956] math: powf inaccuracy
[18961] math: [i386] exp missing underflows
[18966] math: [i386] exp10 missing underflows
[18967] math: math.h XSI POSIX namespace (gamma, isnan, scalb)
[18969] build: multiple string test failures due to missing locale
dependencies
[18970] libc: Reference of pthread_setcancelstate in libc.a
[18977] math: float / long double Bessel functions not in XSI POSIX
[18980] math: i386 libm functions return with excess range and precision
[18981] math: i386 scalb*, ldexp return with excess range and precision
[18982] stdio: va_list and vprintf
[18985] time: Passing out of range data to strftime() causes a segfault
(CVE-2015-8776)
[19003] math: [x86_64] fma4 version of pow inappropriate contraction
[19007] libc: FAIL: elf/check-localplt with -z now and binutils 2.26
[19012] locale: iconv_open leaks memory on error path
[19016] math: clog, clog10 inaccuracy
[19018] nptl: Mangle function pointers in tls_dtor_list
[19032] math: [i386] acosh (-qNaN) spurious "invalid" exception
[19046] math: ldbl-128 / ldbl-128ibm lgamma bad overflow handling
[19048] malloc: malloc: arena free list can become cyclic, increasing
contention
[19049] math: [powerpc] erfc incorrect zero sign
[19050] math: [powerpc] log* incorrect zero sign
[19058] math: [x86_64] Link fail with -fopenmp and -flto
[19059] math: nexttoward overflow incorrect in non-default rounding modes
[19071] math: ldbl-96 lroundl incorrect just below powers of 2
[19074] network: Data race in _res_hconf_reorder_addrs
[19076] math: [ldbl-128ibm] log1pl (-1) wrong sign of infinity
[19077] math: [ldbl-128ibm] logl (1) incorrect sign of zero result
[19078] math: [ldbl-128ibm] expl overflow incorrect in non-default
rounding modes
[19079] math: dbl-64/wordsize-64 lround based on llround incorrect for
ILP32
[19085] math: ldbl-128 lrintl, lroundl missing exceptions for 32-bit long
[19086] manual: posix_fallocate64 documented argument order is wrong.
[19088] math: lround, llround missing exceptions close to overflow
threshold
[19094] math: lrint, llrint missing exceptions close to overflow threshold
[19095] math: dbl-64 lrint incorrect for 64-bit long
[19122] dynamic-link: Unnecessary PLT relocations in librtld.os
[19124] dynamic-link: ld.so failed to build with older assmebler
[19125] math: [powerpc32] llroundf, llround incorrect exceptions
[19129] dynamic-link: [arm] Concurrent lazy TLSDESC resolution can crash
[19134] math: [powerpc32] lround, lroundf spurious exceptions
[19137] libc: i386/epoll_pwait.S doesn't support cancellation
[19143] nptl: Remove CPU set size checking from sched_setaffinity,
pthread_setaffinity_np
[19156] math: [ldbl-128] j0l spurious underflows
[19164] nptl: tst-getcpu fails with many possible CPUs
[19168] math: math/test-ildoubl and math/test-ldouble failure
[19174] nptl: PowerPC: TLE enabled pthread mutex performs poorly.
[19178] dynamic-link: ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATA confuses
prelink
[19181] math: [i386/x86_64] fesetenv (FE_DFL_ENV), fesetenv
(FE_NOMASK_ENV) do not clear SSE exceptions
[19182] malloc: malloc deadlock between ptmalloc_lock_all and
_int_new_arena/reused_arena
[19189] math: [ldbl-128] log1pl (-qNaN) spurious "invalid" exception
[19201] math: dbl-64 remainder incorrect sign of zero result
[19205] math: bits/math-finite.h conditions do not match math.h and
bits/mathcalls.h
[19209] math: bits/math-finite.h wrongly maps ldexp to scalbn
[19211] math: lgamma functions do not set signgam for -ffinite-math-only
for C99-based standards
[19212] libc: features.h not -Wundef clean
[19213] math: [i386/x86_64] log* (1) incorrect zero sign for -ffinite-
math-only
[19214] libc: Family and model identification for AMD CPU's are incorrect.
[19219] libc: GLIBC build fails for ia64 with missing __nearbyintl
[19228] math: [powerpc] nearbyint wrongly clears "inexact", leaves traps
disabled
[19235] math: [powerpc64] lround, lroundf, llround, llroundf spurious
"inexact" exceptions
[19238] math: [powerpc] round, roundf spurious "inexact" for integer
arguments
[19242] libc: strtol incorrect in Turkish locales
[19243] malloc: reused_arena can pick an arena on the free list, leading
to an assertion failure and reference count corruption
[19253] time: tzset() ineffective when temporary TZ did not include DST
rules
[19266] math: strtod ("NAN(I)") incorrect in Turkish locales
[19270] math: [hppa] Shared libm missing __isnanl
[19285] libc: [hppa] sysdeps/unix/sysv/linux/hppa/bits/mman.h: missing
MAP_HUGETLB and MAP_STACK defines
[19313] nptl: Wrong __cpu_mask for x32
[19347] libc: grantpt: try to force a specific gid even without pt_chown
[19349] math: [ldbl-128ibm] tanhl inaccurate for small arguments
[19350] math: [ldbl-128ibm] sinhl spurious overflows
[19351] math: [ldbl-128ibm] logl inaccurate near 1
[19363] time: x32: times() return value wrongly truncates/sign extends
from 32bit
[19367] dynamic-link: Improve branch prediction on Silvermont
[19369] network: Default domain name not reset by res_ninit when "search"
/ "domain" entry is removed from resolv.conf
[19375] math: powerpc: incorrect results for POWER7 logb with negative
subnormals
[19385] localedata: bg_BG: time separator should be colon, not comma
[19408] libc: linux personality syscall wrapper may erroneously return an
error on 32-bit architectures
[19415] libc: dladdr returns wrong names on hppa
[19432] libc: iconv rejects redundant escape sequences in IBM900, IBM903,
IBM905, IBM907, and IBM909
[19439] math: Unix98 isinf and isnan functions conflict with C++11
[19443] build: build failures with -DDEBUG
[19451] build: Make check fails on test-double-vlen2
[19462] libc: Glibc failed to build with -Os
[19465] math: Wrong code with -Os
[19466] time: time/tst-mktime2.c is compiled into an infinite loop with
-Os
[19467] string: Fast_Unaligned_Load needs to be enabled for Excavator core
CPU's.
[19475] libc: Glibc 2.22 doesn't build on sparc [PATCH]
[19486] math: S390: Math tests fail with "Exception Inexact set".
[19529] libc: [ARM]: FAIL: stdlib/tst-makecontext
[19550] libc: [mips] mmap negative offset handling inconsistent with other
architectures
[19590] math: Fail to build shared objects that use libmvec.so functions.
Contributors
============
This release was made possible by the contributions of many people.
The maintainers are grateful to everyone who has contributed
changes or bug reports. These include:
Adhemerval Zanella
Alan Modra
Amit Pawar
Andreas Schwab
Andrew Bennett
Andrew Senkevich
Andrew Stubbs
Anton Blanchard
Arjun Shankar
Arslanbek Astemirov
Aurelien Jarno
Brett Neumeier
Carlos Eduardo Seo
Carlos O'Donell
Chris Metcalf
Chung-Lin Tang
Damyan Ivanov
Daniel Marjamäki
David Kastrup
David Lamparter
David S. Miller
Dmitry V. Levin
Egmont Koblinger
Evert
Flavio Cruz
Florian Weimer
Gabriel F. T. Gomes
Geoffrey Thomas
Gleb Fotengauer-Malinovskiy
Gunnar Hjalmarsson
H.J. Lu
Helge Deller
James Perkins
John David Anglin
Joseph Myers
Justus Winter
Khem Raj
Ludovic Courtès
Maciej W. Rozycki
Manolis Ragkousis
Marcin Kościelnicki
Mark Wielaard
Marko Myllynen
Martin Sebor
Maxim Ostapenko
Mike FABIAN
Mike Frysinger
Namhyung Kim
Ondrej Bilka
Ondřej Bílka
Paul E. Murphy
Paul Eggert
Paul Murphy
Paul Pluzhnikov
Petar Jovanovic
Phil Blundell
Rajalakshmi Srinivasaraghavan
Rasmus Villemoes
Richard Henderson
Rob Wu
Roland McGrath
Samuel Thibault
Siddhesh Poyarekar
Stan Shebs
Stefan Liebler
Steve Ellcey
Szabolcs Nagy
Thomas Schwinge
Torvald Riegel
Tulio Magno Quites Machado Filho
Vincent Bernat
Wilco Dijkstra
Zack Weinberg
Diffstat (limited to 'sysdeps/s390')
260 files changed, 10637 insertions, 1276 deletions
diff --git a/sysdeps/s390/abort-instr.h b/sysdeps/s390/abort-instr.h index 6544b2d618..825601ad50 100644 --- a/sysdeps/s390/abort-instr.h +++ b/sysdeps/s390/abort-instr.h @@ -1,2 +1,2 @@ /* An op-code of 0 should crash any program. */ -#define ABORT_INSTRUCTION asm (".word 0") +#define ABORT_INSTRUCTION __asm__ (".word 0") diff --git a/sysdeps/s390/asm-syntax.h b/sysdeps/s390/asm-syntax.h index e7cf44f45b..64a03b7419 100644 --- a/sysdeps/s390/asm-syntax.h +++ b/sysdeps/s390/asm-syntax.h @@ -1,5 +1,5 @@ /* Definitions for S/390 syntax variations. - Copyright (C) 1992-2015 Free Software Foundation, Inc. + Copyright (C) 1992-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Its master source is NOT part of the C library, however. The master source lives in the GNU MP Library. diff --git a/sysdeps/s390/bits/atomic.h b/sysdeps/s390/atomic-machine.h index 16c8c54b94..4ba41077e4 100644 --- a/sysdeps/s390/bits/atomic.h +++ b/sysdeps/s390/atomic-machine.h @@ -1,4 +1,4 @@ -/* Copyright (C) 2003-2015 Free Software Foundation, Inc. +/* Copyright (C) 2003-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Martin Schwidefsky <schwidefsky@de.ibm.com>, 2003. @@ -55,9 +55,9 @@ typedef uintmax_t uatomic_max_t; #define __arch_compare_and_exchange_val_32_acq(mem, newval, oldval) \ ({ __typeof (mem) __archmem = (mem); \ __typeof (*mem) __archold = (oldval); \ - __asm __volatile ("cs %0,%2,%1" \ - : "+d" (__archold), "=Q" (*__archmem) \ - : "d" (newval), "m" (*__archmem) : "cc", "memory" ); \ + __asm__ __volatile__ ("cs %0,%2,%1" \ + : "+d" (__archold), "=Q" (*__archmem) \ + : "d" (newval), "m" (*__archmem) : "cc", "memory" ); \ __archold; }) #ifdef __s390x__ @@ -65,9 +65,9 @@ typedef uintmax_t uatomic_max_t; # define __arch_compare_and_exchange_val_64_acq(mem, newval, oldval) \ ({ __typeof (mem) __archmem = (mem); \ __typeof (*mem) __archold = (oldval); \ - __asm __volatile ("csg %0,%2,%1" \ - : "+d" (__archold), "=Q" (*__archmem) \ - : "d" ((long) (newval)), "m" (*__archmem) : "cc", "memory" ); \ + __asm__ __volatile__ ("csg %0,%2,%1" \ + : "+d" (__archold), "=Q" (*__archmem) \ + : "d" ((long) (newval)), "m" (*__archmem) : "cc", "memory" ); \ __archold; }) #else # define __HAVE_64B_ATOMICS 0 @@ -89,17 +89,17 @@ typedef uintmax_t uatomic_max_t; __typeof (*(mem)) __atg5_oldval = *__atg5_memp; \ __typeof (*(mem)) __atg5_value = (newvalue); \ if (sizeof (*mem) == 4) \ - __asm __volatile ("0: cs %0,%2,%1\n" \ - " jl 0b" \ - : "+d" (__atg5_oldval), "=Q" (*__atg5_memp) \ - : "d" (__atg5_value), "m" (*__atg5_memp) \ - : "cc", "memory" ); \ + __asm__ __volatile__ ("0: cs %0,%2,%1\n" \ + " jl 0b" \ + : "+d" (__atg5_oldval), "=Q" (*__atg5_memp) \ + : "d" (__atg5_value), "m" (*__atg5_memp) \ + : "cc", "memory" ); \ else if (sizeof (*mem) == 8) \ - __asm __volatile ("0: csg %0,%2,%1\n" \ - " jl 0b" \ - : "+d" ( __atg5_oldval), "=Q" (*__atg5_memp) \ - : "d" ((long) __atg5_value), "m" (*__atg5_memp) \ - : "cc", "memory" ); \ + __asm__ __volatile__ ("0: csg %0,%2,%1\n" \ + " jl 0b" \ + : "+d" ( __atg5_oldval), "=Q" (*__atg5_memp) \ + : "d" ((long) __atg5_value), "m" (*__atg5_memp) \ + : "cc", "memory" ); \ else \ abort (); \ __atg5_oldval; }) @@ -109,11 +109,11 @@ typedef uintmax_t uatomic_max_t; __typeof (*(mem)) __atg5_oldval = *__atg5_memp; \ __typeof (*(mem)) __atg5_value = (newvalue); \ if (sizeof (*mem) == 4) \ - __asm __volatile ("0: cs %0,%2,%1\n" \ - " jl 0b" \ - : "+d" (__atg5_oldval), "=Q" (*__atg5_memp) \ - : "d" (__atg5_value), "m" (*__atg5_memp) \ - : "cc", "memory" ); \ + __asm__ __volatile__ ("0: cs %0,%2,%1\n" \ + " jl 0b" \ + : "+d" (__atg5_oldval), "=Q" (*__atg5_memp) \ + : "d" (__atg5_value), "m" (*__atg5_memp) \ + : "cc", "memory" ); \ else \ abort (); \ __atg5_oldval; }) diff --git a/sysdeps/s390/bits/byteswap-16.h b/sysdeps/s390/bits/byteswap-16.h index d5f7f2f56f..87514d1b92 100644 --- a/sysdeps/s390/bits/byteswap-16.h +++ b/sysdeps/s390/bits/byteswap-16.h @@ -1,5 +1,5 @@ /* Macros to swap the order of bytes in 16-bit integer values. s390 version - Copyright (C) 2012-2015 Free Software Foundation, Inc. + Copyright (C) 2012-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU C Library. diff --git a/sysdeps/s390/bits/byteswap.h b/sysdeps/s390/bits/byteswap.h index af6eb45dc0..6a8cb9d82b 100644 --- a/sysdeps/s390/bits/byteswap.h +++ b/sysdeps/s390/bits/byteswap.h @@ -1,5 +1,5 @@ /* Macros to swap the order of bytes in integer values. s390 version. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU C Library. diff --git a/sysdeps/s390/bits/link.h b/sysdeps/s390/bits/link.h index 370a767d65..2ef7f44225 100644 --- a/sysdeps/s390/bits/link.h +++ b/sysdeps/s390/bits/link.h @@ -1,4 +1,4 @@ -/* Copyright (C) 2005-2015 Free Software Foundation, Inc. +/* Copyright (C) 2005-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/bits/mathdef.h b/sysdeps/s390/bits/mathdef.h index 2e1583902d..8c47ade208 100644 --- a/sysdeps/s390/bits/mathdef.h +++ b/sysdeps/s390/bits/mathdef.h @@ -1,4 +1,4 @@ -/* Copyright (C) 1997-2015 Free Software Foundation, Inc. +/* Copyright (C) 1997-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/bits/setjmp.h b/sysdeps/s390/bits/setjmp.h index c1225e4229..8d29e8dbd8 100644 --- a/sysdeps/s390/bits/setjmp.h +++ b/sysdeps/s390/bits/setjmp.h @@ -1,4 +1,4 @@ -/* Copyright (C) 2000-2015 Free Software Foundation, Inc. +/* Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/bits/string.h b/sysdeps/s390/bits/string.h index e69dc3a639..39e0b7fe7c 100644 --- a/sysdeps/s390/bits/string.h +++ b/sysdeps/s390/bits/string.h @@ -1,5 +1,5 @@ /* Optimized, inlined string functions. S/390 version. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU C Library. @@ -21,8 +21,8 @@ # error "Never use <bits/string.h> directly; include <string.h> instead." #endif -/* The s390 processors can access unaligned multi-byte variables. */ -#define _STRING_ARCH_unaligned 1 +/* Use the unaligned string inline ABI. */ +#define _STRING_INLINE_unaligned 1 /* We only provide optimizations if the user selects them and if GNU CC is used. */ @@ -64,7 +64,7 @@ __strlen_g (const char *__str) #ifndef _FORCE_INLINES #define strcpy(dest, src) __strcpy_g ((dest), (src)) -__STRING_INLINE char *__strcpy_g (char *, const char *) __asm ("strcpy"); +__STRING_INLINE char *__strcpy_g (char *, const char *) __asm__ ("strcpy"); __STRING_INLINE char * __strcpy_g (char *__dest, const char *__src) @@ -226,8 +226,8 @@ memchr (const void *__str, int __c, size_t __n) } #endif -/* Search N bytes of S for C. */ -#define _HAVE_STRING_ARCH_memchr 1 +/* Compare S1 and S2. */ +#define _HAVE_STRING_ARCH_strcmp 1 #ifndef _FORCE_INLINES __STRING_INLINE int strcmp (const char *__s1, const char *__s2) diff --git a/sysdeps/s390/bits/xtitypes.h b/sysdeps/s390/bits/xtitypes.h index a155e77c6e..3c9606a636 100644 --- a/sysdeps/s390/bits/xtitypes.h +++ b/sysdeps/s390/bits/xtitypes.h @@ -1,5 +1,5 @@ /* bits/xtitypes.h -- Define some types used by <bits/stropts.h>. S390/S390x - Copyright (C) 2002-2015 Free Software Foundation, Inc. + Copyright (C) 2002-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/configure b/sysdeps/s390/configure index be6397e271..0fa54c3061 100644 --- a/sysdeps/s390/configure +++ b/sysdeps/s390/configure @@ -104,5 +104,46 @@ if test "$enable_lock_elision" = yes && test "$libc_cv_gcc_builtin_tbegin" = no critic_missing="$critic_missing The used GCC has no support for __builtin_tbegin, which is needed for lock-elision on target S390." fi + +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for S390 vector instruction support" >&5 +$as_echo_n "checking for S390 vector instruction support... " >&6; } +if ${libc_cv_asm_s390_vx+:} false; then : + $as_echo_n "(cached) " >&6 +else + cat > conftest.c <<\EOF +void testvecinsn () +{ + __asm__ (".machine \"z13\" \n\t" + ".machinemode \"zarch_nohighgprs\" \n\t" + "vistrbs %%v16,%%v17 \n\t" + "locghie %%r1,0" : :); +} +EOF +if { ac_try='${CC-cc} --shared conftest.c -o conftest.o &> /dev/null' + { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_try\""; } >&5 + (eval $ac_try) 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; } ; +then + libc_cv_asm_s390_vx=yes +else + libc_cv_asm_s390_vx=no +fi +rm -f conftest* +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_asm_s390_vx" >&5 +$as_echo "$libc_cv_asm_s390_vx" >&6; } + +if test "$libc_cv_asm_s390_vx" = yes ; +then + $as_echo "#define HAVE_S390_VX_ASM_SUPPORT 1" >>confdefs.h + +else + { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Use binutils with vector-support in order to use optimized implementations." >&5 +$as_echo "$as_me: WARNING: Use binutils with vector-support in order to use optimized implementations." >&2;} +fi + + test -n "$critic_missing" && as_fn_error $? " *** $critic_missing" "$LINENO" 5 diff --git a/sysdeps/s390/configure.ac b/sysdeps/s390/configure.ac index 493e9a469c..4da134e9a0 100644 --- a/sysdeps/s390/configure.ac +++ b/sysdeps/s390/configure.ac @@ -36,5 +36,34 @@ if test "$enable_lock_elision" = yes && test "$libc_cv_gcc_builtin_tbegin" = no critic_missing="$critic_missing The used GCC has no support for __builtin_tbegin, which is needed for lock-elision on target S390." fi + +AC_CACHE_CHECK(for S390 vector instruction support, libc_cv_asm_s390_vx, [dnl +cat > conftest.c <<\EOF +void testvecinsn () +{ + __asm__ (".machine \"z13\" \n\t" + ".machinemode \"zarch_nohighgprs\" \n\t" + "vistrbs %%v16,%%v17 \n\t" + "locghie %%r1,0" : :); +} +EOF +dnl +dnl test, if assembler supports S390 vector instructions +if AC_TRY_COMMAND([${CC-cc} --shared conftest.c -o conftest.o &> /dev/null]) ; +then + libc_cv_asm_s390_vx=yes +else + libc_cv_asm_s390_vx=no +fi +rm -f conftest* ]) + +if test "$libc_cv_asm_s390_vx" = yes ; +then + AC_DEFINE(HAVE_S390_VX_ASM_SUPPORT) +else + AC_MSG_WARN([Use binutils with vector-support in order to use optimized implementations.]) +fi + + test -n "$critic_missing" && AC_MSG_ERROR([ *** $critic_missing]) diff --git a/sysdeps/s390/dl-irel.h b/sysdeps/s390/dl-irel.h index 0072c9fdb1..38c5761a6e 100644 --- a/sysdeps/s390/dl-irel.h +++ b/sysdeps/s390/dl-irel.h @@ -1,6 +1,6 @@ /* Machine-dependent ELF indirect relocation inline functions. Version for S/390 32 and 64 bit. - Copyright (C) 2012-2015 Free Software Foundation, Inc. + Copyright (C) 2012-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/dl-procinfo.c b/sysdeps/s390/dl-procinfo.c index 96106f1d66..22c4cf7a01 100644 --- a/sysdeps/s390/dl-procinfo.c +++ b/sysdeps/s390/dl-procinfo.c @@ -1,5 +1,5 @@ /* Data for s390 version of processor capability information. - Copyright (C) 2006-2015 Free Software Foundation, Inc. + Copyright (C) 2006-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Martin Schwidefsky <schwidefsky@de.ibm.com>, 2006. @@ -46,11 +46,11 @@ #if !defined PROCINFO_DECL && defined SHARED ._dl_s390_cap_flags #else -PROCINFO_CLASS const char _dl_s390_cap_flags[11][9] +PROCINFO_CLASS const char _dl_s390_cap_flags[12][9] #endif #ifndef PROCINFO_DECL = { - "esan3", "zarch", "stfle", "msa", "ldisp", "eimm", "dfp", "edat", "etf3eh", "highgprs", "te" + "esan3", "zarch", "stfle", "msa", "ldisp", "eimm", "dfp", "edat", "etf3eh", "highgprs", "te", "vx" } #endif #if !defined SHARED || defined PROCINFO_DECL @@ -62,11 +62,11 @@ PROCINFO_CLASS const char _dl_s390_cap_flags[11][9] #if !defined PROCINFO_DECL && defined SHARED ._dl_s390_platforms #else -PROCINFO_CLASS const char _dl_s390_platforms[7][7] +PROCINFO_CLASS const char _dl_s390_platforms[8][7] #endif #ifndef PROCINFO_DECL = { - "g5", "z900", "z990", "z9-109", "z10", "z196", "zEC12" + "g5", "z900", "z990", "z9-109", "z10", "z196", "zEC12", "z13" } #endif #if !defined SHARED || defined PROCINFO_DECL diff --git a/sysdeps/s390/dl-procinfo.h b/sysdeps/s390/dl-procinfo.h index 0f728ab6e1..4ae276e4ed 100644 --- a/sysdeps/s390/dl-procinfo.h +++ b/sysdeps/s390/dl-procinfo.h @@ -1,5 +1,5 @@ /* s390 version of processor capability information handling macros. - Copyright (C) 2006-2015 Free Software Foundation, Inc. + Copyright (C) 2006-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Martin Schwidefsky <schwidefsky@de.ibm.com>, 2006. @@ -21,9 +21,9 @@ #define _DL_PROCINFO_H 1 #include <ldsodefs.h> -#define _DL_HWCAP_COUNT 10 +#define _DL_HWCAP_COUNT 12 -#define _DL_PLATFORMS_COUNT 5 +#define _DL_PLATFORMS_COUNT 8 /* The kernel provides up to 32 capability bits with elf_hwcap. */ #define _DL_FIRST_PLATFORM 32 @@ -50,6 +50,7 @@ enum HWCAP_S390_ETF3EH = 1 << 8, HWCAP_S390_HIGH_GPRS = 1 << 9, HWCAP_S390_TE = 1 << 10, + HWCAP_S390_VX = 1 << 11, }; #define HWCAP_IMPORTANT (HWCAP_S390_ZARCH | HWCAP_S390_LDISP \ diff --git a/sysdeps/s390/dl-tls.h b/sysdeps/s390/dl-tls.h index 8132b10ab5..503048a622 100644 --- a/sysdeps/s390/dl-tls.h +++ b/sysdeps/s390/dl-tls.h @@ -1,5 +1,5 @@ /* Thread-local storage handling in the ELF dynamic linker. s390 version. - Copyright (C) 2003-2015 Free Software Foundation, Inc. + Copyright (C) 2003-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -62,7 +62,7 @@ versioned_symbol (ld, __tls_get_addr_internal_tmp, the thread descriptor instead of a pointer to the variable. */ # ifdef __s390x__ -asm("\n\ +__asm__("\n\ .text\n\ .globl __tls_get_offset\n\ .type __tls_get_offset, @function\n\ @@ -72,7 +72,7 @@ __tls_get_offset:\n\ jg __tls_get_addr\n\ "); # elif defined __s390__ -asm("\n\ +__asm__("\n\ .text\n\ .globl __tls_get_offset\n\ .type __tls_get_offset, @function\n\ diff --git a/sysdeps/s390/ffs.c b/sysdeps/s390/ffs.c index 645e40ddd0..7808185569 100644 --- a/sysdeps/s390/ffs.c +++ b/sysdeps/s390/ffs.c @@ -1,6 +1,6 @@ /* ffs -- find first set bit in a word, counted from least significant end. S/390 version. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU C Library. @@ -29,8 +29,7 @@ differs in spirit from the above ffz (man ffs). */ int -__ffs (x) - int x; +__ffs (int x) { int r; diff --git a/sysdeps/s390/fix-fp-int-convert-overflow.h b/sysdeps/s390/fix-fp-int-convert-overflow.h new file mode 100644 index 0000000000..61279edc19 --- /dev/null +++ b/sysdeps/s390/fix-fp-int-convert-overflow.h @@ -0,0 +1,33 @@ +/* Fix for conversion of floating point to integer overflow. S390 version. + Copyright (C) 2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#ifndef FIX_FP_INT_CONVERT_OVERFLOW_H +#define FIX_FP_INT_CONVERT_OVERFLOW_H 1 + +/* GCC emits "convert to fixed" instructions for casting floating point values + to integer values. These instructions raise invalid and inexact exceptions + if the floating point value exceeds the integer type ranges. */ +#define FIX_FLT_LLONG_CONVERT_OVERFLOW 1 +#define FIX_DBL_LLONG_CONVERT_OVERFLOW 1 +#define FIX_LDBL_LLONG_CONVERT_OVERFLOW 1 + +#define FIX_FLT_LONG_CONVERT_OVERFLOW 1 +#define FIX_DBL_LONG_CONVERT_OVERFLOW 1 +#define FIX_LDBL_LONG_CONVERT_OVERFLOW 1 + +#endif /* fix-fp-int-convert-overflow.h */ diff --git a/sysdeps/s390/fpu/bits/fenv.h b/sysdeps/s390/fpu/bits/fenv.h index 052b2b6aec..6de74b9939 100644 --- a/sysdeps/s390/fpu/bits/fenv.h +++ b/sysdeps/s390/fpu/bits/fenv.h @@ -1,4 +1,4 @@ -/* Copyright (C) 2000-2015 Free Software Foundation, Inc. +/* Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Denis Joseph Barrow <djbarrow@de.ibm.com>. @@ -77,8 +77,10 @@ typedef unsigned int fexcept_t; /* size of fpc */ typedef struct { fexcept_t __fpc; - void *__ieee_instruction_pointer; - /* failing instruction for ieee exceptions */ + void *__unused; + /* The field __unused (formerly __ieee_instruction_pointer) is a relict from + commit "Remove PTRACE_PEEKUSER" (87b9b50f0d4b92248905e95a06a13c513dc45e59) + and isn´t used anymore. */ } fenv_t; /* If the default argument is used we use this value. */ diff --git a/sysdeps/s390/fpu/bits/mathinline.h b/sysdeps/s390/fpu/bits/mathinline.h index 1d35746e8b..7c09a5c7da 100644 --- a/sysdeps/s390/fpu/bits/mathinline.h +++ b/sysdeps/s390/fpu/bits/mathinline.h @@ -1,5 +1,5 @@ /* Inline math functions for s390. - Copyright (C) 2004-2015 Free Software Foundation, Inc. + Copyright (C) 2004-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -71,7 +71,7 @@ __NTH (__ieee754_sqrt (double x)) { double res; - asm ( "sqdbr %0,%1" : "=f" (res) : "f" (x) ); + __asm__ ( "sqdbr %0,%1" : "=f" (res) : "f" (x) ); return res; } @@ -80,7 +80,7 @@ __NTH (__ieee754_sqrtf (float x)) { float res; - asm ( "sqebr %0,%1" : "=f" (res) : "f" (x) ); + __asm__ ( "sqebr %0,%1" : "=f" (res) : "f" (x) ); return res; } @@ -90,7 +90,7 @@ __NTH (sqrtl (long double __x)) { long double res; - asm ( "sqxbr %0,%1" : "=f" (res) : "f" (__x) ); + __asm__ ( "sqxbr %0,%1" : "=f" (res) : "f" (__x) ); return res; } # endif /* !__NO_LONG_DOUBLE_MATH */ diff --git a/sysdeps/s390/fpu/e_sqrt.c b/sysdeps/s390/fpu/e_sqrt.c index 35675623c6..efdb2865b9 100644 --- a/sysdeps/s390/fpu/e_sqrt.c +++ b/sysdeps/s390/fpu/e_sqrt.c @@ -1,4 +1,4 @@ -/* Copyright (C) 2004-2015 Free Software Foundation, Inc. +/* Copyright (C) 2004-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky <schwidefsky@de.ibm.com>. This file is part of the GNU C Library. @@ -23,7 +23,7 @@ __ieee754_sqrt (double x) { double res; - asm ( "sqdbr %0,%1" : "=f" (res) : "f" (x) ); + __asm__ ( "sqdbr %0,%1" : "=f" (res) : "f" (x) ); return res; } strong_alias (__ieee754_sqrt, __sqrt_finite) diff --git a/sysdeps/s390/fpu/e_sqrtf.c b/sysdeps/s390/fpu/e_sqrtf.c index 3fdd74fa12..38160acc12 100644 --- a/sysdeps/s390/fpu/e_sqrtf.c +++ b/sysdeps/s390/fpu/e_sqrtf.c @@ -1,4 +1,4 @@ -/* Copyright (C) 2004-2015 Free Software Foundation, Inc. +/* Copyright (C) 2004-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky <schwidefsky@de.ibm.com>. This file is part of the GNU C Library. @@ -23,7 +23,7 @@ __ieee754_sqrtf (float x) { float res; - asm ( "sqebr %0,%1" : "=f" (res) : "f" (x) ); + __asm__ ( "sqebr %0,%1" : "=f" (res) : "f" (x) ); return res; } strong_alias (__ieee754_sqrtf, __sqrtf_finite) diff --git a/sysdeps/s390/fpu/e_sqrtl.c b/sysdeps/s390/fpu/e_sqrtl.c index b5215a92a6..add859a3a8 100644 --- a/sysdeps/s390/fpu/e_sqrtl.c +++ b/sysdeps/s390/fpu/e_sqrtl.c @@ -1,5 +1,5 @@ /* Square root. S/390 FPU version. - Copyright (C) 2004-2015 Free Software Foundation, Inc. + Copyright (C) 2004-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky <schwidefsky@de.ibm.com>. This file is part of the GNU C Library. @@ -24,7 +24,7 @@ __ieee754_sqrtl (long double x) { long double res; - asm ( "sqxbr %0,%1" : "=f" (res) : "f" (x) ); + __asm__ ( "sqxbr %0,%1" : "=f" (res) : "f" (x) ); return res; } strong_alias (__ieee754_sqrtl, __sqrtl_finite) diff --git a/sysdeps/s390/fpu/fclrexcpt.c b/sysdeps/s390/fpu/fclrexcpt.c index dd138086ab..b19899a2eb 100644 --- a/sysdeps/s390/fpu/fclrexcpt.c +++ b/sysdeps/s390/fpu/fclrexcpt.c @@ -1,5 +1,5 @@ /* Clear given exceptions in current floating-point environment. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -29,7 +29,12 @@ feclearexcept (int excepts) _FPU_GETCW (temp); /* Clear the relevant bits. */ - temp &= ~((excepts << FPC_DXC_SHIFT)|(excepts << FPC_FLAGS_SHIFT)); + temp &= ~(excepts << FPC_FLAGS_SHIFT); + if ((temp & FPC_NOT_FPU_EXCEPTION) == 0) + /* Bits 6, 7 of dxc-byte are zero, + thus bits 0-5 of dxc-byte correspond to the flag-bits. + Clear the relevant bits in flags and dxc-field. */ + temp &= ~(excepts << FPC_DXC_SHIFT); /* Put the new data in effect. */ _FPU_SETCW (temp); diff --git a/sysdeps/s390/fpu/fedisblxcpt.c b/sysdeps/s390/fpu/fedisblxcpt.c index 6d87d057c6..d3b49789b9 100644 --- a/sysdeps/s390/fpu/fedisblxcpt.c +++ b/sysdeps/s390/fpu/fedisblxcpt.c @@ -1,5 +1,5 @@ /* Disable floating-point exceptions. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Denis Joseph Barrow (djbarrow@de.ibm.com). diff --git a/sysdeps/s390/fpu/feenablxcpt.c b/sysdeps/s390/fpu/feenablxcpt.c index 3a84ce8876..b9aab5976a 100644 --- a/sysdeps/s390/fpu/feenablxcpt.c +++ b/sysdeps/s390/fpu/feenablxcpt.c @@ -1,5 +1,5 @@ /* Enable floating-point exceptions. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Denis Joseph Barrow (djbarrow@de.ibm.com). diff --git a/sysdeps/s390/fpu/fegetenv.c b/sysdeps/s390/fpu/fegetenv.c index 76fc5eaf42..3b912b66cf 100644 --- a/sysdeps/s390/fpu/fegetenv.c +++ b/sysdeps/s390/fpu/fegetenv.c @@ -1,5 +1,5 @@ /* Store current floating-point environment. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Denis Joseph Barrow (djbarrow@de.ibm.com). diff --git a/sysdeps/s390/fpu/fegetexcept.c b/sysdeps/s390/fpu/fegetexcept.c index 4dbc4e1013..dc5033c550 100644 --- a/sysdeps/s390/fpu/fegetexcept.c +++ b/sysdeps/s390/fpu/fegetexcept.c @@ -1,5 +1,5 @@ /* Get enabled floating-point exceptions. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Denis Joseph Barrow (djbarrow@de.ibm.com). diff --git a/sysdeps/s390/fpu/fegetround.c b/sysdeps/s390/fpu/fegetround.c index f60821457c..bca8517577 100644 --- a/sysdeps/s390/fpu/fegetround.c +++ b/sysdeps/s390/fpu/fegetround.c @@ -1,5 +1,5 @@ /* Return current rounding direction. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Denis Joseph Barrow (djbarrow@de.ibm.com). diff --git a/sysdeps/s390/fpu/feholdexcpt.c b/sysdeps/s390/fpu/feholdexcpt.c index 1cbe4b1f58..2700028016 100644 --- a/sysdeps/s390/fpu/feholdexcpt.c +++ b/sysdeps/s390/fpu/feholdexcpt.c @@ -1,5 +1,5 @@ /* Store current floating-point environment and clear exceptions. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Denis Joseph Barrow (djbarrow@de.ibm.com). diff --git a/sysdeps/s390/fpu/fenv_libc.h b/sysdeps/s390/fpu/fenv_libc.h index 5488fb0936..dff8fd92e5 100644 --- a/sysdeps/s390/fpu/fenv_libc.h +++ b/sysdeps/s390/fpu/fenv_libc.h @@ -1,4 +1,4 @@ -/* Copyright (C) 2000-2015 Free Software Foundation, Inc. +/* Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Denis Joseph Barrow (djbarrow@de.ibm.com). diff --git a/sysdeps/s390/fpu/fesetenv.c b/sysdeps/s390/fpu/fesetenv.c index 467716a2ae..694a538c1e 100644 --- a/sysdeps/s390/fpu/fesetenv.c +++ b/sysdeps/s390/fpu/fesetenv.c @@ -1,5 +1,5 @@ /* Install given floating-point environment. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Denis Joseph Barrow (djbarrow@de.ibm.com). @@ -32,12 +32,10 @@ __fesetenv (const fenv_t *envp) if (envp == FE_DFL_ENV) { env.__fpc = _FPU_DEFAULT; - env.__ieee_instruction_pointer = 0; } else if (envp == FE_NOMASK_ENV) { env.__fpc = FPC_EXCEPTION_MASK; - env.__ieee_instruction_pointer = 0; } else env = (*envp); diff --git a/sysdeps/s390/fpu/fesetround.c b/sysdeps/s390/fpu/fesetround.c index d6eedcee00..5b15a1973d 100644 --- a/sysdeps/s390/fpu/fesetround.c +++ b/sysdeps/s390/fpu/fesetround.c @@ -1,5 +1,5 @@ /* Set current rounding direction. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Denis Joseph Barrow (djbarrow@de.ibm.com). @@ -28,9 +28,9 @@ __fesetround (int round) /* ROUND is not a valid rounding mode. */ return 1; } - __asm__ volatile ("srnm 0(%0)" - : - : "a" (round)); + __asm__ __volatile__ ("srnm 0(%0)" + : + : "a" (round)); return 0; } diff --git a/sysdeps/s390/fpu/feupdateenv.c b/sysdeps/s390/fpu/feupdateenv.c index 7e2cf15079..1aad35ec13 100644 --- a/sysdeps/s390/fpu/feupdateenv.c +++ b/sysdeps/s390/fpu/feupdateenv.c @@ -1,5 +1,5 @@ /* Install given floating-point environment and raise exceptions. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Denis Joseph Barrow (djbarrow@de.ibm.com). diff --git a/sysdeps/s390/fpu/fgetexcptflg.c b/sysdeps/s390/fpu/fgetexcptflg.c index c14dc15c25..09b7cfd99c 100644 --- a/sysdeps/s390/fpu/fgetexcptflg.c +++ b/sysdeps/s390/fpu/fgetexcptflg.c @@ -1,5 +1,5 @@ /* Store current representation for exceptions. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Denis Joseph Barrow (djbarrow@de.ibm.com). @@ -27,7 +27,13 @@ fegetexceptflag (fexcept_t *flagp, int excepts) /* Get the current exceptions. */ _FPU_GETCW (temp); - newexcepts = (excepts << FPC_DXC_SHIFT) | (excepts << FPC_FLAGS_SHIFT); + newexcepts = excepts << FPC_FLAGS_SHIFT; + if ((temp & FPC_NOT_FPU_EXCEPTION) == 0) + /* Bits 6, 7 of dxc-byte are zero, + thus bits 0-5 of dxc-byte correspond to the flag-bits. + Evaluate flags and last dxc-exception-code. */ + newexcepts |= excepts << FPC_DXC_SHIFT; + *flagp = temp & newexcepts; /* Success. */ diff --git a/sysdeps/s390/fpu/fpu_control.h b/sysdeps/s390/fpu/fpu_control.h index 098f0d06f3..e0266a703e 100644 --- a/sysdeps/s390/fpu/fpu_control.h +++ b/sysdeps/s390/fpu/fpu_control.h @@ -1,5 +1,5 @@ /* FPU control word definitions. Stub version. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Denis Joseph Barrow (djbarrow@de.ibm.com) and Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU C Library. @@ -19,12 +19,12 @@ <http://www.gnu.org/licenses/>. */ #ifndef _FPU_CONTROL_H -# define _FPU_CONTROL_H +#define _FPU_CONTROL_H -# include <features.h> +#include <features.h> /* These bits are reserved are not changed. */ -# define _FPU_RESERVED 0x070700FC +#define _FPU_RESERVED 0x0707FFFC /* The fdlibm code requires no interrupts for exceptions. Don't change the rounding mode, it would break long double I/O! */ @@ -34,8 +34,8 @@ typedef unsigned int fpu_control_t; /* Macros for accessing the hardware control word. */ -#define _FPU_GETCW(cw) __asm__ volatile ("efpc %0,0" : "=d" (cw)) -#define _FPU_SETCW(cw) __asm__ volatile ("sfpc %0,0" : : "d" (cw)) +#define _FPU_GETCW(cw) __asm__ __volatile__ ("efpc %0,0" : "=d" (cw)) +#define _FPU_SETCW(cw) __asm__ __volatile__ ("sfpc %0,0" : : "d" (cw)) /* Default control word set at startup. */ extern fpu_control_t __fpu_control; diff --git a/sysdeps/s390/fpu/fraiseexcpt.c b/sysdeps/s390/fpu/fraiseexcpt.c index 9970f20dfa..92a1a7db68 100644 --- a/sysdeps/s390/fpu/fraiseexcpt.c +++ b/sysdeps/s390/fpu/fraiseexcpt.c @@ -1,5 +1,5 @@ /* Raise given exceptions. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Denis Joseph Barrow (djbarrow@de.ibm.com) and Martin Schwidefsky (schwidefsky@de.ibm.com). diff --git a/sysdeps/s390/fpu/fsetexcptflg.c b/sysdeps/s390/fpu/fsetexcptflg.c index cbe9f34155..25ade854bd 100644 --- a/sysdeps/s390/fpu/fsetexcptflg.c +++ b/sysdeps/s390/fpu/fsetexcptflg.c @@ -1,5 +1,5 @@ /* Set floating-point environment exception handling. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Denis Joseph Barrow (djbarrow@de.ibm.com). @@ -24,16 +24,25 @@ int fesetexceptflag (const fexcept_t *flagp, int excepts) { - fexcept_t temp,newexcepts; + fexcept_t temp, newexcepts; /* Get the current environment. We have to do this since we cannot separately set the status word. */ _FPU_GETCW (temp); /* Install the new exception bits in the Accrued Exception Byte. */ excepts = excepts & FE_ALL_EXCEPT; - newexcepts = (excepts << FPC_DXC_SHIFT) | (excepts << FPC_FLAGS_SHIFT); + newexcepts = excepts << FPC_FLAGS_SHIFT; temp &= ~newexcepts; - temp |= *flagp & newexcepts; + if ((temp & FPC_NOT_FPU_EXCEPTION) == 0) + /* Bits 6, 7 of dxc-byte are zero, + thus bits 0-5 of dxc-byte correspond to the flag-bits. + Clear given exceptions in dxc-field. */ + temp &= ~(excepts << FPC_DXC_SHIFT); + + /* Integrate dxc-byte of flagp into flags. The dxc-byte of flagp contains + either an ieee-exception or 0 (see fegetexceptflag). */ + temp |= (*flagp | ((*flagp >> FPC_DXC_SHIFT) << FPC_FLAGS_SHIFT)) + & newexcepts; /* Store the new status word (along with the rest of the environment. Possibly new exceptions are set but they won't get executed unless diff --git a/sysdeps/s390/fpu/ftestexcept.c b/sysdeps/s390/fpu/ftestexcept.c index 6889632a5b..45cfcb52d0 100644 --- a/sysdeps/s390/fpu/ftestexcept.c +++ b/sysdeps/s390/fpu/ftestexcept.c @@ -1,5 +1,5 @@ /* Test exception in current environment. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Denis Joseph Barrow (djbarrow@de.ibm.com). @@ -23,11 +23,17 @@ int fetestexcept (int excepts) { - fexcept_t temp; + fexcept_t temp, res; /* Get current exceptions. */ _FPU_GETCW (temp); - temp = (temp >> FPC_DXC_SHIFT) | (temp >> FPC_FLAGS_SHIFT); - return temp & excepts & FE_ALL_EXCEPT; + res = temp >> FPC_FLAGS_SHIFT; + if ((temp & FPC_NOT_FPU_EXCEPTION) == 0) + /* Bits 6, 7 of dxc-byte are zero, + thus bits 0-5 of dxc-byte correspond to the flag-bits. + Evaluate flags and last dxc-exception-code. */ + res |= temp >> FPC_DXC_SHIFT; + + return res & excepts & FE_ALL_EXCEPT; } libm_hidden_def (fetestexcept) diff --git a/sysdeps/s390/fpu/get-rounding-mode.h b/sysdeps/s390/fpu/get-rounding-mode.h index d4a13dcdfc..5150b0ab25 100644 --- a/sysdeps/s390/fpu/get-rounding-mode.h +++ b/sysdeps/s390/fpu/get-rounding-mode.h @@ -1,5 +1,5 @@ /* Determine floating-point rounding mode within libc. S/390 version. - Copyright (C) 2012-2015 Free Software Foundation, Inc. + Copyright (C) 2012-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/fpu/libm-test-ulps b/sysdeps/s390/fpu/libm-test-ulps index 232730b8c3..bc5795b361 100644 --- a/sysdeps/s390/fpu/libm-test-ulps +++ b/sysdeps/s390/fpu/libm-test-ulps @@ -4,9 +4,13 @@ Function: "acos": float: 1 ifloat: 1 +ildouble: 1 +ldouble: 1 Function: "acos_downward": +double: 1 float: 1 +idouble: 1 ifloat: 1 ildouble: 1 ldouble: 1 @@ -28,34 +32,34 @@ ildouble: 1 ldouble: 1 Function: "acosh": -double: 1 +double: 2 float: 2 -idouble: 1 +idouble: 2 ifloat: 2 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: "acosh_downward": double: 1 -float: 1 +float: 2 idouble: 1 -ifloat: 1 -ildouble: 2 -ldouble: 2 +ifloat: 2 +ildouble: 3 +ldouble: 3 Function: "acosh_towardzero": double: 2 -float: 1 +float: 2 idouble: 2 -ifloat: 1 -ildouble: 1 -ldouble: 1 +ifloat: 2 +ildouble: 2 +ldouble: 2 Function: "acosh_upward": double: 2 -float: 1 +float: 2 idouble: 2 -ifloat: 1 +ifloat: 2 ildouble: 2 ldouble: 2 @@ -70,11 +74,13 @@ double: 1 float: 1 idouble: 1 ifloat: 1 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: "asin_towardzero": +double: 1 float: 1 +idouble: 1 ifloat: 1 ildouble: 1 ldouble: 1 @@ -84,24 +90,24 @@ double: 1 float: 1 idouble: 1 ifloat: 1 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: "asinh": double: 1 float: 1 idouble: 1 ifloat: 1 -ildouble: 1 -ldouble: 1 +ildouble: 3 +ldouble: 3 Function: "asinh_downward": double: 3 float: 3 idouble: 3 ifloat: 3 -ildouble: 3 -ldouble: 3 +ildouble: 4 +ldouble: 4 Function: "asinh_towardzero": double: 2 @@ -116,8 +122,8 @@ double: 3 float: 3 idouble: 3 ifloat: 3 -ildouble: 3 -ldouble: 3 +ildouble: 4 +ldouble: 4 Function: "atan": double: 1 @@ -138,16 +144,16 @@ double: 1 float: 2 idouble: 1 ifloat: 2 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: "atan2_towardzero": double: 1 float: 2 idouble: 1 ifloat: 2 -ildouble: 2 -ldouble: 2 +ildouble: 3 +ldouble: 3 Function: "atan2_upward": double: 1 @@ -178,24 +184,24 @@ double: 1 float: 2 idouble: 1 ifloat: 2 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: "atanh": -double: 1 +double: 2 float: 2 -idouble: 1 +idouble: 2 ifloat: 2 -ildouble: 2 -ldouble: 2 +ildouble: 3 +ldouble: 3 Function: "atanh_downward": double: 3 -float: 2 +float: 3 idouble: 3 -ifloat: 2 -ildouble: 3 -ldouble: 3 +ifloat: 3 +ildouble: 4 +ldouble: 4 Function: "atanh_towardzero": double: 2 @@ -206,28 +212,36 @@ ildouble: 2 ldouble: 2 Function: "atanh_upward": -double: 2 +double: 3 float: 3 -idouble: 2 +idouble: 3 ifloat: 3 -ildouble: 3 -ldouble: 3 +ildouble: 4 +ldouble: 4 Function: "cabs": double: 1 idouble: 1 +ildouble: 1 +ldouble: 1 Function: "cabs_downward": double: 1 idouble: 1 +ildouble: 1 +ldouble: 1 Function: "cabs_towardzero": double: 1 idouble: 1 +ildouble: 1 +ldouble: 1 Function: "cabs_upward": double: 1 idouble: 1 +ildouble: 1 +ldouble: 1 Function: Real part of "cacos": double: 1 @@ -358,34 +372,36 @@ ildouble: 3 ldouble: 3 Function: "carg": +double: 1 float: 1 +idouble: 1 ifloat: 1 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: "carg_downward": double: 1 -float: 1 +float: 2 idouble: 1 -ifloat: 1 -ildouble: 1 -ldouble: 1 +ifloat: 2 +ildouble: 2 +ldouble: 2 Function: "carg_towardzero": double: 1 float: 2 idouble: 1 ifloat: 2 -ildouble: 2 -ldouble: 2 +ildouble: 3 +ldouble: 3 Function: "carg_upward": double: 1 float: 1 idouble: 1 ifloat: 1 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: Real part of "casin": double: 1 @@ -652,17 +668,17 @@ ildouble: 1 ldouble: 1 Function: "cbrt_towardzero": -double: 2 +double: 3 float: 1 -idouble: 2 +idouble: 3 ifloat: 1 ildouble: 1 ldouble: 1 Function: "cbrt_upward": -double: 4 +double: 5 float: 1 -idouble: 4 +idouble: 5 ifloat: 1 ildouble: 1 ldouble: 1 @@ -864,8 +880,8 @@ double: 3 float: 3 idouble: 3 ifloat: 3 -ildouble: 4 -ldouble: 4 +ildouble: 2 +ldouble: 2 Function: Imaginary part of "clog": double: 1 @@ -877,11 +893,11 @@ ldouble: 1 Function: Real part of "clog10": double: 3 -float: 3 +float: 4 idouble: 3 -ifloat: 3 -ildouble: 4 -ldouble: 4 +ifloat: 4 +ildouble: 2 +ldouble: 2 Function: Imaginary part of "clog10": double: 1 @@ -892,12 +908,12 @@ ildouble: 2 ldouble: 2 Function: Real part of "clog10_downward": -double: 6 -float: 6 -idouble: 6 -ifloat: 6 -ildouble: 5 -ldouble: 5 +double: 5 +float: 4 +idouble: 5 +ifloat: 4 +ildouble: 3 +ldouble: 3 Function: Imaginary part of "clog10_downward": double: 2 @@ -909,11 +925,11 @@ ldouble: 3 Function: Real part of "clog10_towardzero": double: 5 -float: 4 +float: 5 idouble: 5 -ifloat: 4 -ildouble: 6 -ldouble: 6 +ifloat: 5 +ildouble: 4 +ldouble: 4 Function: Imaginary part of "clog10_towardzero": double: 2 @@ -924,28 +940,28 @@ ildouble: 3 ldouble: 3 Function: Real part of "clog10_upward": -double: 8 +double: 6 float: 5 -idouble: 8 +idouble: 6 ifloat: 5 -ildouble: 5 -ldouble: 5 +ildouble: 4 +ldouble: 4 Function: Imaginary part of "clog10_upward": double: 2 -float: 3 +float: 4 idouble: 2 -ifloat: 3 +ifloat: 4 ildouble: 3 ldouble: 3 Function: Real part of "clog_downward": -double: 7 -float: 5 -idouble: 7 -ifloat: 5 -ildouble: 6 -ldouble: 6 +double: 4 +float: 3 +idouble: 4 +ifloat: 3 +ildouble: 3 +ldouble: 3 Function: Imaginary part of "clog_downward": double: 1 @@ -956,28 +972,28 @@ ildouble: 2 ldouble: 2 Function: Real part of "clog_towardzero": -double: 7 -float: 5 -idouble: 7 -ifloat: 5 -ildouble: 6 -ldouble: 6 +double: 4 +float: 4 +idouble: 4 +ifloat: 4 +ildouble: 3 +ldouble: 3 Function: Imaginary part of "clog_towardzero": double: 1 -float: 2 +float: 3 idouble: 1 -ifloat: 2 +ifloat: 3 ildouble: 2 ldouble: 2 Function: Real part of "clog_upward": -double: 8 -float: 5 -idouble: 8 -ifloat: 5 -ildouble: 6 -ldouble: 6 +double: 4 +float: 3 +idouble: 4 +ifloat: 3 +ildouble: 4 +ldouble: 4 Function: Imaginary part of "clog_upward": double: 1 @@ -1238,64 +1254,64 @@ double: 2 float: 2 idouble: 2 ifloat: 2 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: Imaginary part of "csqrt": double: 2 float: 2 idouble: 2 ifloat: 2 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: Real part of "csqrt_downward": -double: 4 +double: 5 float: 4 -idouble: 4 +idouble: 5 ifloat: 4 -ildouble: 3 -ldouble: 3 +ildouble: 4 +ldouble: 4 Function: Imaginary part of "csqrt_downward": double: 4 float: 3 idouble: 4 ifloat: 3 -ildouble: 2 -ldouble: 2 +ildouble: 3 +ldouble: 3 Function: Real part of "csqrt_towardzero": -double: 3 +double: 4 float: 3 -idouble: 3 +idouble: 4 ifloat: 3 -ildouble: 2 -ldouble: 2 +ildouble: 3 +ldouble: 3 Function: Imaginary part of "csqrt_towardzero": double: 4 float: 3 idouble: 4 ifloat: 3 -ildouble: 2 -ldouble: 2 +ildouble: 3 +ldouble: 3 Function: Real part of "csqrt_upward": double: 5 float: 4 idouble: 5 ifloat: 4 -ildouble: 3 -ldouble: 3 +ildouble: 4 +ldouble: 4 Function: Imaginary part of "csqrt_upward": double: 3 float: 3 idouble: 3 ifloat: 3 -ildouble: 2 -ldouble: 2 +ildouble: 3 +ldouble: 3 Function: Real part of "ctan": double: 1 @@ -1450,7 +1466,9 @@ ildouble: 1 ldouble: 1 Function: "erf_upward": +double: 1 float: 1 +idouble: 1 ifloat: 1 ildouble: 2 ldouble: 2 @@ -1468,30 +1486,36 @@ double: 3 float: 4 idouble: 3 ifloat: 4 -ildouble: 3 -ldouble: 3 +ildouble: 5 +ldouble: 5 Function: "erfc_towardzero": double: 3 float: 3 idouble: 3 ifloat: 3 -ildouble: 3 -ldouble: 3 +ildouble: 4 +ldouble: 4 Function: "erfc_upward": double: 3 float: 4 idouble: 3 ifloat: 4 -ildouble: 3 -ldouble: 3 +ildouble: 5 +ldouble: 5 + +Function: "exp": +float: 1 +ifloat: 1 +ildouble: 1 +ldouble: 1 Function: "exp10": double: 2 idouble: 2 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: "exp10_downward": double: 2 @@ -1514,12 +1538,14 @@ double: 2 float: 1 idouble: 2 ifloat: 1 -ildouble: 2 -ldouble: 2 +ildouble: 3 +ldouble: 3 Function: "exp2": double: 1 +float: 1 idouble: 1 +ifloat: 1 ildouble: 1 ldouble: 1 @@ -1544,8 +1570,8 @@ double: 1 float: 1 idouble: 1 ifloat: 1 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: "exp_downward": double: 1 @@ -1577,11 +1603,11 @@ ldouble: 2 Function: "expm1_towardzero": double: 1 -float: 1 +float: 2 idouble: 1 -ifloat: 1 -ildouble: 3 -ldouble: 3 +ifloat: 2 +ildouble: 4 +ldouble: 4 Function: "expm1_upward": double: 1 @@ -1592,36 +1618,36 @@ ildouble: 3 ldouble: 3 Function: "gamma": -double: 1 -float: 1 -idouble: 1 -ifloat: 1 -ildouble: 1 -ldouble: 1 - -Function: "gamma_downward": double: 3 -float: 3 +float: 4 idouble: 3 -ifloat: 3 -ildouble: 2 -ldouble: 2 +ifloat: 4 +ildouble: 5 +ldouble: 5 + +Function: "gamma_downward": +double: 4 +float: 4 +idouble: 4 +ifloat: 4 +ildouble: 8 +ldouble: 8 Function: "gamma_towardzero": -double: 3 +double: 4 float: 3 -idouble: 3 +idouble: 4 ifloat: 3 -ildouble: 2 -ldouble: 2 +ildouble: 5 +ldouble: 5 Function: "gamma_upward": -double: 3 -float: 3 -idouble: 3 -ifloat: 3 -ildouble: 3 -ldouble: 3 +double: 4 +float: 5 +idouble: 4 +ifloat: 5 +ildouble: 8 +ldouble: 8 Function: "hypot": double: 1 @@ -1744,36 +1770,36 @@ ildouble: 7 ldouble: 7 Function: "lgamma": -double: 1 -float: 1 -idouble: 1 -ifloat: 1 -ildouble: 1 -ldouble: 1 - -Function: "lgamma_downward": double: 3 -float: 3 +float: 4 idouble: 3 -ifloat: 3 -ildouble: 2 -ldouble: 2 +ifloat: 4 +ildouble: 5 +ldouble: 5 + +Function: "lgamma_downward": +double: 4 +float: 4 +idouble: 4 +ifloat: 4 +ildouble: 8 +ldouble: 8 Function: "lgamma_towardzero": -double: 3 +double: 4 float: 3 -idouble: 3 +idouble: 4 ifloat: 3 -ildouble: 2 -ldouble: 2 +ildouble: 5 +ldouble: 5 Function: "lgamma_upward": -double: 3 -float: 3 -idouble: 3 -ifloat: 3 -ildouble: 3 -ldouble: 3 +double: 4 +float: 5 +idouble: 4 +ifloat: 5 +ildouble: 8 +ldouble: 8 Function: "log": float: 1 @@ -1818,48 +1844,48 @@ double: 1 float: 1 idouble: 1 ifloat: 1 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: "log1p_downward": double: 1 float: 2 idouble: 1 ifloat: 2 -ildouble: 1 -ldouble: 1 +ildouble: 3 +ldouble: 3 Function: "log1p_towardzero": double: 2 float: 2 idouble: 2 ifloat: 2 -ildouble: 1 -ldouble: 1 +ildouble: 3 +ldouble: 3 Function: "log1p_upward": double: 2 float: 2 idouble: 2 ifloat: 2 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: "log2": double: 1 float: 1 idouble: 1 ifloat: 1 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: "log2_downward": double: 3 float: 3 idouble: 3 ifloat: 3 -ildouble: 1 -ldouble: 1 +ildouble: 3 +ldouble: 3 Function: "log2_towardzero": double: 2 @@ -1886,11 +1912,13 @@ ldouble: 1 Function: "log_towardzero": float: 1 ifloat: 1 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: "log_upward": +double: 1 float: 1 +idouble: 1 ifloat: 1 ildouble: 1 ldouble: 1 @@ -1898,14 +1926,14 @@ ldouble: 1 Function: "pow": float: 1 ifloat: 1 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: "pow10": double: 2 idouble: 2 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: "pow10_downward": double: 2 @@ -1928,24 +1956,24 @@ double: 2 float: 1 idouble: 2 ifloat: 1 -ildouble: 2 -ldouble: 2 +ildouble: 3 +ldouble: 3 Function: "pow_downward": double: 1 float: 1 idouble: 1 ifloat: 1 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: "pow_towardzero": double: 1 float: 1 idouble: 1 ifloat: 1 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: "pow_upward": double: 1 @@ -1982,8 +2010,8 @@ double: 1 float: 2 idouble: 1 ifloat: 2 -ildouble: 2 -ldouble: 2 +ildouble: 3 +ldouble: 3 Function: "sincos": float: 1 @@ -2009,47 +2037,49 @@ ldouble: 2 Function: "sincos_upward": double: 1 -float: 1 +float: 2 idouble: 1 -ifloat: 1 -ildouble: 2 -ldouble: 2 +ifloat: 2 +ildouble: 3 +ldouble: 3 Function: "sinh": double: 2 float: 2 idouble: 2 ifloat: 2 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: "sinh_downward": double: 3 float: 3 idouble: 3 ifloat: 3 -ildouble: 2 -ldouble: 2 +ildouble: 3 +ldouble: 3 Function: "sinh_towardzero": double: 2 float: 2 idouble: 2 ifloat: 2 -ildouble: 2 -ldouble: 2 +ildouble: 3 +ldouble: 3 Function: "sinh_upward": double: 3 float: 3 idouble: 3 ifloat: 3 -ildouble: 2 -ldouble: 2 +ildouble: 4 +ldouble: 4 Function: "tan": float: 1 ifloat: 1 +ildouble: 1 +ldouble: 1 Function: "tan_downward": double: 1 @@ -2080,24 +2110,24 @@ double: 2 float: 2 idouble: 2 ifloat: 2 -ildouble: 1 -ldouble: 1 +ildouble: 2 +ldouble: 2 Function: "tanh_downward": double: 3 float: 3 idouble: 3 ifloat: 3 -ildouble: 2 -ldouble: 2 +ildouble: 4 +ldouble: 4 Function: "tanh_towardzero": double: 2 float: 2 idouble: 2 ifloat: 2 -ildouble: 2 -ldouble: 2 +ildouble: 3 +ldouble: 3 Function: "tanh_upward": double: 3 @@ -2108,34 +2138,34 @@ ildouble: 3 ldouble: 3 Function: "tgamma": -double: 3 -float: 3 -idouble: 3 -ifloat: 3 +double: 5 +float: 4 +idouble: 5 +ifloat: 4 ildouble: 4 ldouble: 4 Function: "tgamma_downward": -double: 3 -float: 3 -idouble: 3 -ifloat: 3 +double: 5 +float: 5 +idouble: 5 +ifloat: 5 ildouble: 5 ldouble: 5 Function: "tgamma_towardzero": -double: 3 -float: 3 -idouble: 3 -ifloat: 3 +double: 5 +float: 4 +idouble: 5 +ifloat: 4 ildouble: 5 ldouble: 5 Function: "tgamma_upward": -double: 3 -float: 3 -idouble: 3 -ifloat: 3 +double: 4 +float: 4 +idouble: 4 +ifloat: 4 ildouble: 4 ldouble: 4 diff --git a/sysdeps/s390/fpu/s_fma.c b/sysdeps/s390/fpu/s_fma.c index fbfeea4977..7d7e563b7e 100644 --- a/sysdeps/s390/fpu/s_fma.c +++ b/sysdeps/s390/fpu/s_fma.c @@ -1,5 +1,5 @@ /* Compute x * y + z as ternary operation. S/390 version. - Copyright (C) 2010-2015 Free Software Foundation, Inc. + Copyright (C) 2010-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Jakub Jelinek <jakub@redhat.com>, 2010. @@ -23,7 +23,7 @@ double __fma (double x, double y, double z) { double r; - asm ("madbr %0,%1,%2" : "=f" (r) : "%f" (x), "fR" (y), "0" (z)); + __asm__ ("madbr %0,%1,%2" : "=f" (r) : "%f" (x), "fR" (y), "0" (z)); return r; } #ifndef __fma diff --git a/sysdeps/s390/fpu/s_fmaf.c b/sysdeps/s390/fpu/s_fmaf.c index f65c73e389..50af2bbc5b 100644 --- a/sysdeps/s390/fpu/s_fmaf.c +++ b/sysdeps/s390/fpu/s_fmaf.c @@ -1,5 +1,5 @@ /* Compute x * y + z as ternary operation. S/390 version. - Copyright (C) 2010-2015 Free Software Foundation, Inc. + Copyright (C) 2010-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Jakub Jelinek <jakub@redhat.com>, 2010. @@ -23,7 +23,7 @@ float __fmaf (float x, float y, float z) { float r; - asm ("maebr %0,%1,%2" : "=f" (r) : "%f" (x), "fR" (y), "0" (z)); + __asm__ ("maebr %0,%1,%2" : "=f" (r) : "%f" (x), "fR" (y), "0" (z)); return r; } #ifndef __fmaf diff --git a/sysdeps/s390/gccframe.h b/sysdeps/s390/gccframe.h index 16ea5bbc33..ac0b6b9c4b 100644 --- a/sysdeps/s390/gccframe.h +++ b/sysdeps/s390/gccframe.h @@ -1,5 +1,5 @@ /* Definition of object in frame unwind info. s390 version. - Copyright (C) 2001-2015 Free Software Foundation, Inc. + Copyright (C) 2001-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/gmp-mparam.h b/sysdeps/s390/gmp-mparam.h index 445c35d0bd..0f7ec7af0c 100644 --- a/sysdeps/s390/gmp-mparam.h +++ b/sysdeps/s390/gmp-mparam.h @@ -1,5 +1,5 @@ /* gmp-mparam.h -- Compiler/machine parameter header file. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU MP Library. diff --git a/sysdeps/s390/jmpbuf-offsets.h b/sysdeps/s390/jmpbuf-offsets.h index a0893598e5..bf23695cd2 100644 --- a/sysdeps/s390/jmpbuf-offsets.h +++ b/sysdeps/s390/jmpbuf-offsets.h @@ -1,5 +1,5 @@ /* Private macros for accessing __jmp_buf contents. S/390 version. - Copyright (C) 2006-2015 Free Software Foundation, Inc. + Copyright (C) 2006-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/jmpbuf-unwind.h b/sysdeps/s390/jmpbuf-unwind.h index b7b6b9daa1..1e1b4a8b6d 100644 --- a/sysdeps/s390/jmpbuf-unwind.h +++ b/sysdeps/s390/jmpbuf-unwind.h @@ -1,4 +1,4 @@ -/* Copyright (C) 2003-2015 Free Software Foundation, Inc. +/* Copyright (C) 2003-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Jakub Jelinek <jakub@redhat.com>, 2003. diff --git a/sysdeps/s390/ldsodefs.h b/sysdeps/s390/ldsodefs.h index 8692556e34..b22d364be4 100644 --- a/sysdeps/s390/ldsodefs.h +++ b/sysdeps/s390/ldsodefs.h @@ -1,5 +1,5 @@ /* Run-time dynamic linker data structures for loaded ELF shared objects. - Copyright (C) 1995-2015 Free Software Foundation, Inc. + Copyright (C) 1995-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/libc-tls.c b/sysdeps/s390/libc-tls.c index b0cf4df32a..1df435c1f4 100644 --- a/sysdeps/s390/libc-tls.c +++ b/sysdeps/s390/libc-tls.c @@ -1,5 +1,5 @@ /* Thread-local storage handling in the ELF dynamic linker. S390 version. - Copyright (C) 2003-2015 Free Software Foundation, Inc. + Copyright (C) 2003-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/bits/linkmap.h b/sysdeps/s390/linkmap.h index fc1fba363a..fc1fba363a 100644 --- a/sysdeps/s390/bits/linkmap.h +++ b/sysdeps/s390/linkmap.h diff --git a/sysdeps/s390/longjmp.c b/sysdeps/s390/longjmp.c index 9b6fa54d7b..25b0145933 100644 --- a/sysdeps/s390/longjmp.c +++ b/sysdeps/s390/longjmp.c @@ -1,4 +1,4 @@ -/* Copyright (C) 2014-2015 Free Software Foundation, Inc. +/* Copyright (C) 2014-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -20,11 +20,21 @@ #include <shlib-compat.h> +#if defined SHARED && SHLIB_COMPAT (libc, GLIBC_2_19, GLIBC_2_20) +/* We don't want the weak alias to longjmp, _longjmp, siglongjmp here, + because we create the default/versioned symbols later. */ +# define __libc_siglongjmp __libc_siglongjmp +#endif /* SHARED && SHLIB_COMPAT (libc, GLIBC_2_19, GLIBC_2_20) */ + #include <setjmp/longjmp.c> #if defined SHARED && SHLIB_COMPAT (libc, GLIBC_2_19, GLIBC_2_20) /* In glibc release 2.19 new versions of longjmp-functions were introduced, but were reverted before 2.20. Thus both versions are the same function. */ + +strong_alias (__libc_siglongjmp, __libc_longjmp) +libc_hidden_def (__libc_longjmp) + weak_alias (__libc_siglongjmp, __v1_longjmp) weak_alias (__libc_siglongjmp, __v2_longjmp) versioned_symbol (libc, __v1_longjmp, _longjmp, GLIBC_2_0); diff --git a/sysdeps/s390/machine-gmon.h b/sysdeps/s390/machine-gmon.h index 3ad43975d8..0c978754f6 100644 --- a/sysdeps/s390/machine-gmon.h +++ b/sysdeps/s390/machine-gmon.h @@ -1,5 +1,5 @@ /* s390-specific implementation of profiling support. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/memusage.h b/sysdeps/s390/memusage.h index 4721b59a9d..888d708b29 100644 --- a/sysdeps/s390/memusage.h +++ b/sysdeps/s390/memusage.h @@ -1,4 +1,4 @@ -/* Copyright (C) 2000-2015 Free Software Foundation, Inc. +/* Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -15,6 +15,6 @@ License along with the GNU C Library; if not, see <http://www.gnu.org/licenses/>. */ -#define GETSP() ({ register uintptr_t stack_ptr asm ("15"); stack_ptr; }) +#define GETSP() ({ register uintptr_t stack_ptr __asm__ ("15"); stack_ptr; }) #include <sysdeps/generic/memusage.h> diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile new file mode 100644 index 0000000000..0805b07984 --- /dev/null +++ b/sysdeps/s390/multiarch/Makefile @@ -0,0 +1,44 @@ +ifeq ($(subdir),string) +sysdep_routines += strlen strlen-vx strlen-c \ + strnlen strnlen-vx strnlen-c \ + strcpy strcpy-vx \ + stpcpy stpcpy-vx stpcpy-c \ + strncpy strncpy-vx \ + stpncpy stpncpy-vx stpncpy-c \ + strcat strcat-vx strcat-c \ + strncat strncat-vx strncat-c \ + strcmp strcmp-vx \ + strncmp strncmp-vx strncmp-c \ + strchr strchr-vx strchr-c \ + strchrnul strchrnul-vx strchrnul-c \ + strrchr strrchr-vx strrchr-c \ + strspn strspn-vx strspn-c \ + strpbrk strpbrk-vx strpbrk-c \ + strcspn strcspn-vx strcspn-c \ + memchr memchr-vx \ + rawmemchr rawmemchr-vx rawmemchr-c \ + memccpy memccpy-vx memccpy-c \ + memrchr memrchr-vx memrchr-c +endif + +ifeq ($(subdir),wcsmbs) +sysdep_routines += wcslen wcslen-vx wcslen-c \ + wcsnlen wcsnlen-vx wcsnlen-c \ + wcscpy wcscpy-vx wcscpy-c \ + wcpcpy wcpcpy-vx wcpcpy-c \ + wcsncpy wcsncpy-vx wcsncpy-c \ + wcpncpy wcpncpy-vx wcpncpy-c \ + wcscat wcscat-vx wcscat-c \ + wcsncat wcsncat-vx wcsncat-c \ + wcscmp wcscmp-vx wcscmp-c \ + wcsncmp wcsncmp-vx wcsncmp-c \ + wcschr wcschr-vx wcschr-c \ + wcschrnul wcschrnul-vx wcschrnul-c \ + wcsrchr wcsrchr-vx wcsrchr-c \ + wcsspn wcsspn-vx wcsspn-c \ + wcspbrk wcspbrk-vx wcspbrk-c \ + wcscspn wcscspn-vx wcscspn-c \ + wmemchr wmemchr-vx wmemchr-c \ + wmemset wmemset-vx wmemset-c \ + wmemcmp wmemcmp-vx wmemcmp-c +endif diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c new file mode 100644 index 0000000000..62a435983c --- /dev/null +++ b/sysdeps/s390/multiarch/ifunc-impl-list.c @@ -0,0 +1,145 @@ +/* Enumerate available IFUNC implementations of a function. s390/s390x version. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include <assert.h> +#include <string.h> +#include <wchar.h> +#include <ifunc-impl-list.h> +#include <ifunc-resolve.h> + +/* Maximum number of IFUNC implementations. */ +#define MAX_IFUNC 3 + +/* Fill ARRAY of MAX elements with IFUNC implementations for function + NAME supported on target machine and return the number of valid + entries. */ +size_t +__libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, + size_t max) +{ + assert (max >= MAX_IFUNC); + + size_t i = 0; + + /* Get hardware information. */ + unsigned long int dl_hwcap = GLRO (dl_hwcap); + unsigned long long stfle_bits = 0ULL; + if ((dl_hwcap & HWCAP_S390_STFLE) + && (dl_hwcap & HWCAP_S390_ZARCH) + && (dl_hwcap & HWCAP_S390_HIGH_GPRS)) + { + S390_STORE_STFLE (stfle_bits); + } + + IFUNC_IMPL (i, name, memset, + IFUNC_IMPL_ADD (array, i, memset, + S390_IS_Z196 (stfle_bits), __memset_z196) + IFUNC_IMPL_ADD (array, i, memset, + S390_IS_Z10 (stfle_bits), __memset_z10) + IFUNC_IMPL_ADD (array, i, memset, 1, __memset_default)) + + IFUNC_IMPL (i, name, memcmp, + IFUNC_IMPL_ADD (array, i, memcmp, + S390_IS_Z196 (stfle_bits), __memcmp_z196) + IFUNC_IMPL_ADD (array, i, memcmp, + S390_IS_Z10 (stfle_bits), __memcmp_z10) + IFUNC_IMPL_ADD (array, i, memcmp, 1, __memcmp_default)) + +#ifdef SHARED + + IFUNC_IMPL (i, name, memcpy, + IFUNC_IMPL_ADD (array, i, memcpy, + S390_IS_Z196 (stfle_bits), __memcpy_z196) + IFUNC_IMPL_ADD (array, i, memcpy, + S390_IS_Z10 (stfle_bits), __memcpy_z10) + IFUNC_IMPL_ADD (array, i, memcpy, 1, __memcpy_default)) + +#endif /* SHARED */ + +#ifdef HAVE_S390_VX_ASM_SUPPORT + +# define IFUNC_VX_IMPL(FUNC) \ + IFUNC_IMPL (i, name, FUNC, \ + IFUNC_IMPL_ADD (array, i, FUNC, dl_hwcap & HWCAP_S390_VX, \ + __##FUNC##_vx) \ + IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c)) + + IFUNC_VX_IMPL (strlen); + IFUNC_VX_IMPL (wcslen); + + IFUNC_VX_IMPL (strnlen); + IFUNC_VX_IMPL (wcsnlen); + + IFUNC_VX_IMPL (strcpy); + IFUNC_VX_IMPL (wcscpy); + + IFUNC_VX_IMPL (stpcpy); + IFUNC_VX_IMPL (wcpcpy); + + IFUNC_VX_IMPL (strncpy); + IFUNC_VX_IMPL (wcsncpy); + + IFUNC_VX_IMPL (stpncpy); + IFUNC_VX_IMPL (wcpncpy); + + IFUNC_VX_IMPL (strcat); + IFUNC_VX_IMPL (wcscat); + + IFUNC_VX_IMPL (strncat); + IFUNC_VX_IMPL (wcsncat); + + IFUNC_VX_IMPL (strcmp); + IFUNC_VX_IMPL (wcscmp); + + IFUNC_VX_IMPL (strncmp); + IFUNC_VX_IMPL (wcsncmp); + + IFUNC_VX_IMPL (strchr); + IFUNC_VX_IMPL (wcschr); + + IFUNC_VX_IMPL (strchrnul); + IFUNC_VX_IMPL (wcschrnul); + + IFUNC_VX_IMPL (strrchr); + IFUNC_VX_IMPL (wcsrchr); + + IFUNC_VX_IMPL (strspn); + IFUNC_VX_IMPL (wcsspn); + + IFUNC_VX_IMPL (strpbrk); + IFUNC_VX_IMPL (wcspbrk); + + IFUNC_VX_IMPL (strcspn); + IFUNC_VX_IMPL (wcscspn); + + IFUNC_VX_IMPL (memchr); + IFUNC_VX_IMPL (wmemchr); + IFUNC_VX_IMPL (rawmemchr); + + IFUNC_VX_IMPL (memccpy); + + IFUNC_VX_IMPL (wmemset); + + IFUNC_VX_IMPL (wmemcmp); + + IFUNC_VX_IMPL (memrchr); + +#endif /* HAVE_S390_VX_ASM_SUPPORT */ + + return i; +} diff --git a/sysdeps/s390/multiarch/ifunc-resolve.h b/sysdeps/s390/multiarch/ifunc-resolve.h new file mode 100644 index 0000000000..744a0d8d6d --- /dev/null +++ b/sysdeps/s390/multiarch/ifunc-resolve.h @@ -0,0 +1,94 @@ +/* IFUNC resolver function for CPU specific functions. + 32/64 bit S/390 version. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include <unistd.h> +#include <dl-procinfo.h> + +#define S390_STFLE_BITS_Z10 34 /* General instructions extension */ +#define S390_STFLE_BITS_Z196 45 /* Distinct operands, pop ... */ + +#define S390_IS_Z196(STFLE_BITS) \ + ((STFLE_BITS & (1ULL << (63 - S390_STFLE_BITS_Z196))) != 0) + +#define S390_IS_Z10(STFLE_BITS) \ + ((STFLE_BITS & (1ULL << (63 - S390_STFLE_BITS_Z10))) != 0) + +#define S390_STORE_STFLE(STFLE_BITS) \ + /* We want just 1 double word to be returned. */ \ + register unsigned long reg0 __asm__("0") = 0; \ + \ + __asm__ __volatile__(".machine push" "\n\t" \ + ".machine \"z9-109\"" "\n\t" \ + ".machinemode \"zarch_nohighgprs\"\n\t" \ + "stfle %0" "\n\t" \ + ".machine pop" "\n" \ + : "=QS" (STFLE_BITS), "+d" (reg0) \ + : : "cc"); + +#define s390_libc_ifunc(FUNC) \ + __asm__ (".globl " #FUNC "\n\t" \ + ".type " #FUNC ",@gnu_indirect_function\n\t" \ + ".set " #FUNC ",__resolve_" #FUNC "\n\t" \ + ".globl __GI_" #FUNC "\n\t" \ + ".set __GI_" #FUNC "," #FUNC "\n"); \ + \ + /* Make the declarations of the optimized functions hidden in order + to prevent GOT slots being generated for them. */ \ + extern void *__##FUNC##_z196 attribute_hidden; \ + extern void *__##FUNC##_z10 attribute_hidden; \ + extern void *__##FUNC##_default attribute_hidden; \ + \ + void *__resolve_##FUNC (unsigned long int dl_hwcap) \ + { \ + if ((dl_hwcap & HWCAP_S390_STFLE) \ + && (dl_hwcap & HWCAP_S390_ZARCH) \ + && (dl_hwcap & HWCAP_S390_HIGH_GPRS)) \ + { \ + unsigned long long stfle_bits; \ + S390_STORE_STFLE (stfle_bits); \ + \ + if (S390_IS_Z196 (stfle_bits)) \ + return &__##FUNC##_z196; \ + else if (S390_IS_Z10 (stfle_bits)) \ + return &__##FUNC##_z10; \ + else \ + return &__##FUNC##_default; \ + } \ + else \ + return &__##FUNC##_default; \ + } + +#define s390_vx_libc_ifunc(FUNC) \ + s390_vx_libc_ifunc2(FUNC, FUNC) + +#define s390_vx_libc_ifunc2(RESOLVERFUNC, FUNC) \ + /* Make the declarations of the optimized functions hidden in order + to prevent GOT slots being generated for them. */ \ + extern __typeof (FUNC) RESOLVERFUNC##_vx attribute_hidden; \ + extern __typeof (FUNC) RESOLVERFUNC##_c attribute_hidden; \ + extern void *__resolve_##RESOLVERFUNC (unsigned long int) __asm__ (#FUNC); \ + \ + void *__resolve_##RESOLVERFUNC (unsigned long int dl_hwcap) \ + { \ + if (dl_hwcap & HWCAP_S390_VX) \ + return &RESOLVERFUNC##_vx; \ + else \ + return &RESOLVERFUNC##_c; \ + } \ + __asm__ (".type " #FUNC ", %gnu_indirect_function"); diff --git a/sysdeps/s390/multiarch/memccpy-c.c b/sysdeps/s390/multiarch/memccpy-c.c new file mode 100644 index 0000000000..9309bd108b --- /dev/null +++ b/sysdeps/s390/multiarch/memccpy-c.c @@ -0,0 +1,25 @@ +/* Default memccpy implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define MEMCCPY __memccpy_c + +# include <string.h> +extern __typeof (__memccpy) __memccpy_c; +# include <string/memccpy.c> +#endif diff --git a/sysdeps/s390/multiarch/memccpy-vx.S b/sysdeps/s390/multiarch/memccpy-vx.S new file mode 100644 index 0000000000..2db9b2cef4 --- /dev/null +++ b/sysdeps/s390/multiarch/memccpy-vx.S @@ -0,0 +1,156 @@ +/* Vector optimized 32/64 bit S/390 version of memccpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* void *memccpy (void * dest, const void *src, int c, size_t n) + Copies no more than n bytes from src to dest, + stopping when the character c is found + and returns pointer next to c in dest or null if c not found. + + Register usage: + -r0=tmp + -r1=tmp + -r2=dest + -r3=src + -r4=c + -r5=n + -r6=current_len + -v16=part of s + -v17=index of found c + -v18=c replicated + -v19=part #2 of s + -v31=save area for r6 +*/ +ENTRY(__memccpy_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + +# if !defined __s390x__ + llgfr %r5,%r5 +# endif /* !defined __s390x__ */ + + vlvgp %v31,%r6,%r7 /* Save registers. */ + clgije %r5,0,.Lnf_end /* If len == 0 then exit. */ + + vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */ + lcbb %r0,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */ + llgfr %r0,%r0 /* Convert 32bit to 64bit. */ + + vlvgb %v18,%r4,0 /* Generate vector which elements are all c. + if c > 255, c will be truncated. */ + vrepb %v18,%v18,0 + lghi %r6,0 /* current_len = 0. */ + + clgrjle %r5,%r0,.Lremaining_v16 /* If maxlen <= loaded-bytes + -> Process remaining. */ + + vfeebs %v17,%v16,%v18 /* Find c. */ + vlgvb %r1,%v17,7 /* Load byte index of c. */ + clgrjl %r1,%r0,.Lfound_v16 /* Found c is within loaded bytes. */ + + /* Align s to 16 byte. */ + risbgn %r1,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r6,15 /* current_len = 15. */ + slr %r6,%r1 /* Compute highest index to 16byte boundary. */ + + vstl %v16,%r6,0(%r2) /* Store prcessed bytes */ + ahi %r6,1 + +.Lpreloop1: + /* Now we are 16byte aligned, so we can load + a full vreg without page fault. */ + vl %v16,0(%r6,%r3) /* Load s. */ + clgijl %r5,17,.Lremaining_v16 /* If n <= 16, + process remaining bytes. */ + lgr %r7,%r5 + slgfi %r7,16 /* border_len = n - 16. */ + j .Lloop1 + +.Lloop2: + vl %v16,16(%r6,%r3) + vst %v19,0(%r6,%r2) + aghi %r6,16 + +.Lloop1: + clgrjhe %r6,%r7,.Lremaining_v16 /* If current_len >= border + then process remaining bytes. */ + vfeebs %v17,%v16,%v18 /* Find c. */ + jl .Lfound_v16 /* Jump away if c was found. */ + vl %v19,16(%r6,%r3) /* Load next s part. */ + vst %v16,0(%r6,%r2) /* Store previous part without c. */ + aghi %r6,16 + + clgrjhe %r6,%r7,.Lremaining_v19 + vfeebs %v17,%v19,%v18 + jl .Lfound_v19 + vl %v16,16(%r6,%r3) + vst %v19,0(%r6,%r2) + aghi %r6,16 + + clgrjhe %r6,%r7,.Lremaining_v16 + vfeebs %v17,%v16,%v18 + jl .Lfound_v16 + vl %v19,16(%r6,%r3) + vst %v16,0(%r6,%r2) + aghi %r6,16 + + clgrjhe %r6,%r7,.Lremaining_v19 + vfeebs %v17,%v19,%v18 + jo .Lloop2 + +.Lfound_v19: + vlr %v16,%v19 +.Lfound_v16: + /* v16 contains c. Store remaining bytes to c. currlen hasn´t + reached border, thus checking for maxlen is not needed! */ + vlgvb %r1,%v17,7 /* Load byte index of c. */ + la %r2,0(%r6,%r2) /* vstl has no support for index-register. */ +.Lfound_v16_store: + vstl %v16,%r1,0(%r2) /* Copy bytes including c. */ + la %r2,1(%r1,%r2) /* Return pointer next to c in dest. */ + vlgvg %r6,%v31,0 + vlgvg %r7,%v31,1 + br %r14 + +.Lremaining_v19: + vlr %v16,%v19 +.Lremaining_v16: + /* v16 contains the remaining bytes [1...16]. + Check and store remaining bytes. */ + vfeebs %v17,%v16,%v18 + slgrk %r7,%r5,%r6 /* Remaining bytes = maxlen - current_len. */ + aghi %r7,-1 /* vstl needs highest index. */ + la %r2,0(%r6,%r2) /* vstl has no index register. */ + vlgvb %r1,%v17,7 /* Load index of c or 16 if not found. */ + /* c in remaining bytes? -> Jump away (c-index <= max-index) */ + clrjle %r1,%r7,.Lfound_v16_store + vstl %v16,%r7,0(%r2) /* Store remaining bytes. */ + +.Lnf_end: + vlgvg %r6,%v31,0 + vlgvg %r7,%v31,1 + lghi %r2,0 /* Return null. */ + br %r14 +END(__memccpy_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/memccpy.c b/sysdeps/s390/multiarch/memccpy.c new file mode 100644 index 0000000000..0a0936e340 --- /dev/null +++ b/sysdeps/s390/multiarch/memccpy.c @@ -0,0 +1,28 @@ +/* Multiple versions of memccpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc (__memccpy) +weak_alias (__memccpy, memccpy) + +#else +# include <string/memccpy.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/memchr-vx.S b/sysdeps/s390/multiarch/memchr-vx.S new file mode 100644 index 0000000000..875eee2b43 --- /dev/null +++ b/sysdeps/s390/multiarch/memchr-vx.S @@ -0,0 +1,159 @@ +/* Vector optimized 32/64 bit S/390 version of memchr. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* void *memchr (const void *s, int c, size_t n) + Scans memory for character c + and returns pointer to first c. + + Register usage: + -r0=tmp + -r1=tmp + -r2=s + -r3=c + -r4=n + -r5=current_len + -v16=part of s + -v17=index of found c + -v18=c replicated +*/ +ENTRY(__memchr_vx) + + .machine "z13" + .machinemode "zarch_nohighgprs" + +# if !defined __s390x__ + llgfr %r4,%r4 +# endif /* !defined __s390x__ */ + + clgije %r4,0,.Lnf_end /* If len == 0 then exit. */ + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r0,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + llgfr %r0,%r0 /* Convert 32bit to 64bit. */ + + vlvgb %v18,%r3,0 /* Generate vector which elements are all c. + if c > 255, c will be truncated. */ + vrepb %v18,%v18,0 + lghi %r5,16 /* current_len = 16. */ + + clgrjhe %r0,%r4,.Llastcmp /* If (bytes to boundary) >= n, + jump to lastcmp. */ + + vfeebs %v17,%v16,%v18 /* Find c. */ + vlgvb %r1,%v17,7 /* Load byte index of c. */ + clgrjl %r1,%r0,.Lfound2 /* Found c is within loaded bytes. */ + + /* Align s to 16 byte. */ + risbgn %r1,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + slr %r5,%r1 /* Compute bytes to 16bytes boundary. */ + + lgr %r0,%r5 /* If %r5 + 64 < n? -> loop64. */ + aghi %r0,64 + clgrjl %r0,%r4,.Lloop64 +.Llt64: + vl %v16,0(%r5,%r2) + aghi %r5,16 + clgrjhe %r5,%r4,.Llastcmp /* Do last compare if curr-len >= n. */ + vfeebs %v17,%v16,%v18 /* Find c. */ + jl .Lfound /* Jump away if c was found. */ + + vl %v16,0(%r5,%r2) + aghi %r5,16 + clgrjhe %r5,%r4,.Llastcmp + vfeebs %v17,%v16,%v18 + jl .Lfound + + vl %v16,0(%r5,%r2) + aghi %r5,16 + clgrjhe %r5,%r4,.Llastcmp + vfeebs %v17,%v16,%v18 + jl .Lfound + + vl %v16,0(%r5,%r2) + aghi %r5,16 + +.Llastcmp: + /* Use comparision result only if located within first n characters. + %r5: current_len; + %r4: n; + (current_len - n): [0...16[ + first ignored match index: vr-width - (current_len - n) ]0...16] + */ + vfeebs %v17,%v16,%v18 /* Find c. */ + slgrk %r4,%r5,%r4 /* %r5 = current_len - n. */ + lghi %r0,16 /* Register width = 16. */ + vlgvb %r1,%v17,7 /* Extract found index or 16 if all equal. */ + slr %r0,%r4 /* %r0 = first ignored match index. */ + clrjl %r1,%r0,.Lfound2 /* Go away if miscompare is below n bytes. */ + /* c not found within n-bytes. */ +.Lnf_end: + lghi %r2,0 /* Return null. */ + br %r14 + +.Lfound48: + aghi %r5,16 +.Lfound32: + aghi %r5,16 +.Lfound16: + aghi %r5,16 +.Lfound0: + aghi %r5,16 +.Lfound: + vlgvb %r1,%v17,7 /* Load byte index of c. */ +.Lfound2: + slgfi %r5,16 /* current_len -=16 */ + algr %r5,%r1 /* Zero byte index is added to current len. */ + la %r2,0(%r5,%r2) /* Return pointer to c. */ + br %r14 + + +.Lloop64: + vl %v16,0(%r5,%r2) + vfeebs %v17,%v16,%v18 /* Find c. */ + jl .Lfound0 /* Jump away if c was found. */ + vl %v16,16(%r5,%r2) + vfeebs %v17,%v16,%v18 + jl .Lfound16 + vl %v16,32(%r5,%r2) + vfeebs %v17,%v16,%v18 + jl .Lfound32 + vl %v16,48(%r5,%r2) + vfeebs %v17,%v16,%v18 + jl .Lfound48 + + aghi %r5,64 + lgr %r0,%r5 /* If %r5 + 64 < n? -> loop64. */ + aghi %r0,64 + clgrjl %r0,%r4,.Lloop64 + + j .Llt64 +END(__memchr_vx) + +# define memchr __memchr_c +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) strong_alias(__memchr_c, __GI_memchr) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ + +#include <memchr.S> diff --git a/sysdeps/s390/multiarch/memchr.c b/sysdeps/s390/multiarch/memchr.c new file mode 100644 index 0000000000..f80de1cc1f --- /dev/null +++ b/sysdeps/s390/multiarch/memchr.c @@ -0,0 +1,24 @@ +/* Multiple versions of memchr. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__memchr, memchr) +#endif diff --git a/sysdeps/s390/multiarch/memrchr-c.c b/sysdeps/s390/multiarch/memrchr-c.c new file mode 100644 index 0000000000..af54097376 --- /dev/null +++ b/sysdeps/s390/multiarch/memrchr-c.c @@ -0,0 +1,25 @@ +/* Default memrchr implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define MEMRCHR __memrchr_c + +# include <string.h> +extern __typeof (__memrchr) __memrchr_c; +# include <string/memrchr.c> +#endif diff --git a/sysdeps/s390/multiarch/memrchr-vx.S b/sysdeps/s390/multiarch/memrchr-vx.S new file mode 100644 index 0000000000..fdb8c30ebe --- /dev/null +++ b/sysdeps/s390/multiarch/memrchr-vx.S @@ -0,0 +1,160 @@ +/* Vector optimized 32/64 bit S/390 version of memrchr. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* void *memrchr (const void *s, int c, size_t n) + Scans memory for character c backwards + and returns pointer to first c. + + Register usage: + -r0=tmp + -r1=tmp + -r2=s + -r3=c + -r4=n + -r5=s in loop + + -v16=part of s + -v17=index of found c + -v18=c replicated + -v20=permute pattern +*/ +ENTRY(__memrchr_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + +# if !defined __s390x__ + llgfr %r4,%r4 +# endif /* !defined __s390x__ */ + clgije %r4,0,.Lnot_found + + vlvgb %v18,%r3,0 /* Generate vector which elements are all c. + If c > 255, c will be truncated. */ + vrepb %v18,%v18,0 + + llcr %r3,%r3 /* char c_char = (char) c. */ + + /* check byte n - 1. */ + llc %r0,-1(%r4,%r2) + slgfi %r4,1 + clrje %r0,%r3,.Lfound_end + jh .Lnot_found /* Return NULL if n is now 0. */ + + larl %r1,.Lpermute_mask /* Load permute mask. */ + vl %v20,0(%r1) + + /* check byte n - 2. */ + llc %r0,-1(%r4,%r2) + slgfi %r4,1 + clrje %r0,%r3,.Lfound_end + jh .Lnot_found /* Return NULL if n is now 0. */ + + clgijhe %r4,64,.Lloop64 /* If n >= 64 -> loop64. */ + +.Llt64: + /* Process n < 64 bytes. */ + clgijl %r4,16,.Llt16 /* Jump away if n < 16. */ + aghi %r4,-16 + vl %v16,0(%r4,%r2) + vfeebs %v17,%v16,%v18 + jno .Lfound0 + clgijl %r4,16,.Llt16 + aghi %r4,-16 + vl %v16,0(%r4,%r2) + vfeebs %v17,%v16,%v18 + jno .Lfound0 + clgijl %r4,16,.Llt16 + aghi %r4,-16 + vl %v16,0(%r4,%r2) + vfeebs %v17,%v16,%v18 + jno .Lfound0 +.Llt16: + clgfi %r4,0 /* if remaining bytes == 0, return NULL. */ + locghie %r2,0 + ber %r14 + + aghi %r4,-1 /* vll needs highest index. */ + vll %v16,%r4,0(%r2) /* Load remaining bytes. */ + + /* Right-shift of v16 to mask bytes after highest index. */ + lhi %r0,15 + slr %r0,%r4 /* Compute byte count for vector shift right. */ + sll %r0,3 /* Convert to bit count. */ + vlvgb %v17,%r0,7 + vsrlb %v16,%v16,%v17 /* Vector shift right by byte by number of bytes + specified in bits 1-4 of byte 7 in v17. */ + j .Lfound_permute + +.Lfound48: + aghi %r4,16 +.Lfound32: + aghi %r4,16 +.Lfound16: + aghi %r4,16 +.Lfound0: + la %r2,0(%r4,%r2) /* Set pointer to start of v16. */ + lghi %r4,15 /* Set highest index in v16 to last index. */ +.Lfound_permute: + /* Search for a c in v16 in reversed byte order. v16 contains %r4 + 1 + bytes. If v16 was not fully loaded, the bytes are already + right shifted, so that the bytes in v16 can simply be reversed. */ + vperm %v16,%v16,%v16,%v20 /* Permute v16 to reversed order. */ + vfeeb %v16,%v16,%v18 /* Find c in reversed v16. */ + vlgvb %r1,%v16,7 /* Index of c or 16 if not found. */ + + /* Return NULL if there is no c in loaded bytes. */ + clrjh %r1,%r4,.Lnot_found + + slgr %r4,%r1 +.Lfound_end: + la %r2,0(%r4,%r2) /* Return pointer to c. */ + br %r14 + +.Lnot_found: + lghi %r2,0 + br %r14 + +.Lpermute_mask: + .byte 0x0F,0x0E,0x0D,0x0C,0x0B,0x0A,0x09,0x08 + .byte 0x07,0x06,0x05,0x04,0x03,0x02,0x01,0x00 + +.Lloop64: + aghi %r4,-64 + vl %v16,48(%r4,%r2) /* Load 16bytes of memory area. */ + vfeebs %v17,%v16,%v18 /* Find c. */ + jno .Lfound48 /* Jump away if c was found. */ + vl %v16,32(%r4,%r2) + vfeebs %v17,%v16,%v18 + jno .Lfound32 + vl %v16,16(%r4,%r2) + vfeebs %v17,%v16,%v18 + jno .Lfound16 + vl %v16,0(%r4,%r2) + vfeebs %v17,%v16,%v18 + jno .Lfound0 + + clgijhe %r4,64,.Lloop64 /* If n >= 64 -> loop64. */ + j .Llt64 +END(__memrchr_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/memrchr.c b/sysdeps/s390/multiarch/memrchr.c new file mode 100644 index 0000000000..7681890d01 --- /dev/null +++ b/sysdeps/s390/multiarch/memrchr.c @@ -0,0 +1,28 @@ +/* Multiple versions of memrchr. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc (__memrchr) +weak_alias (__memrchr, memrchr) + +#else +# include <string/memrchr.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/rawmemchr-c.c b/sysdeps/s390/multiarch/rawmemchr-c.c new file mode 100644 index 0000000000..20dcdb5a28 --- /dev/null +++ b/sysdeps/s390/multiarch/rawmemchr-c.c @@ -0,0 +1,34 @@ +/* Default rawmemchr implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> + +# define RAWMEMCHR __rawmemchr_c +# undef weak_alias +# define weak_alias(a, b) +# ifdef SHARED +# undef libc_hidden_def +# define libc_hidden_def(name) \ + __hidden_ver1 (__rawmemchr_c, __GI___rawmemchr, __rawmemchr_c); +# endif /* SHARED */ + +extern __typeof (rawmemchr) __rawmemchr_c attribute_hidden; + +# include <string/rawmemchr.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/rawmemchr-vx.S b/sysdeps/s390/multiarch/rawmemchr-vx.S new file mode 100644 index 0000000000..5af2419e98 --- /dev/null +++ b/sysdeps/s390/multiarch/rawmemchr-vx.S @@ -0,0 +1,92 @@ +/* Vector optimized 32/64 bit S/390 version of rawmemchr. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* void *rawmemchr (const void *s, int c) + Scans memory for character c + and returns pointer to first c. + + Register usage: + -r1=tmp + -r2=s + -r3=c + -r4=tmp + -r5=current_len + -v16=part of s + -v17=index of unequal + -v18=c replicated +*/ +ENTRY(__rawmemchr_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + vlvgb %v18,%r3,0 /* Generate vector which elements are all c. + If c > 255, c will be truncated. */ + vrepb %v18,%v18,0 + + vfeeb %v17,%v16,%v18 /* Vector find element equal. */ + vlgvb %r5,%v17,7 /* Load byte index of character or zero. */ + clrjl %r5,%r1,.Lend_found /* If found c is in loaded bytes, end. */ + + /* Align s to 16 byte. */ + risbgn %r1,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,16 + slr %r5,%r1 /* Compute bytes to 16bytes boundary. */ + + /* Find c in a 16byte aligned loop. */ +.Lloop: + vl %v16,0(%r5,%r2) /* Load s. */ + vfeebs %v17,%v16,%v18 /* Vector find element equal. */ + jno .Lcharacter /* Jump away if element found. */ + vl %v16,16(%r5,%r2) + vfeebs %v17,%v16,%v18 + jno .Lcharacter16 + vl %v16,32(%r5,%r2) + vfeebs %v17,%v16,%v18 + jno .Lcharacter32 + vl %v16,48(%r5,%r2) + vfeebs %v17,%v16,%v18 + jno .Lcharacter48 + + aghi %r5,64 + j .Lloop /* No character found -> loop. */ + + /* Found character. */ +.Lcharacter48: + aghi %r5,16 +.Lcharacter32: + aghi %r5,16 +.Lcharacter16: + aghi %r5,16 +.Lcharacter: + vlgvb %r1,%v17,7 /* Load byte index of character. */ + algr %r5,%r1 +.Lend_found: + la %r2,0(%r5,%r2) /* Return pointer to character. */ + br %r14 +END(__rawmemchr_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/rawmemchr.c b/sysdeps/s390/multiarch/rawmemchr.c new file mode 100644 index 0000000000..7186ccd9d4 --- /dev/null +++ b/sysdeps/s390/multiarch/rawmemchr.c @@ -0,0 +1,28 @@ +/* Multiple versions of rawmemchr. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc (__rawmemchr) +weak_alias (__rawmemchr, rawmemchr) + +#else +# include <string/rawmemchr.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/stpcpy-c.c b/sysdeps/s390/multiarch/stpcpy-c.c new file mode 100644 index 0000000000..85a8a93c7f --- /dev/null +++ b/sysdeps/s390/multiarch/stpcpy-c.c @@ -0,0 +1,35 @@ +/* Default stpcpy implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define STPCPY __stpcpy_c +# undef weak_alias +# define weak_alias(a, b) +# ifdef SHARED +# undef libc_hidden_def +# define libc_hidden_def(name) \ + __hidden_ver1 (__stpcpy_c, __GI___stpcpy, __stpcpy_c); +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) \ + strong_alias (__stpcpy_c, __stpcpy_c_1); \ + __hidden_ver1 (__stpcpy_c_1, __GI_stpcpy, __stpcpy_c_1); +# endif /* SHARED */ + + +# include <string/stpcpy.c> +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/stpcpy-vx.S b/sysdeps/s390/multiarch/stpcpy-vx.S new file mode 100644 index 0000000000..da9f2760de --- /dev/null +++ b/sysdeps/s390/multiarch/stpcpy-vx.S @@ -0,0 +1,104 @@ +/* Vector optimized 32/64 bit S/390 version of stpcpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* char * stpcpy (const char *dest, const char *src) + Copy string src to dest returning a pointer to its end. + + Register usage: + -r1=tmp + -r2=dest and return value + -r3=src + -r4=tmp + -r5=current_len + -v16=part of src + -v17=index of zero + -v18=part of src +*/ +ENTRY(__stpcpy_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */ + + vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r5,%v17,7 /* Load zero index or 16 if not found. */ + clrjl %r5,%r1,.Lfound_align /* If found zero within loaded bytes, + copy bytes before and return. */ + + /* Align s to 16 byte. */ + risbgn %r4,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,15 /* current_len = 15. */ + slr %r5,%r4 /* Compute highest index to 16byte boundary. */ + + vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */ + ahi %r5,1 /* Start loop at next character. */ + + /* Find zero in 16byte aligned loop. */ +.Lloop: + vl %v16,0(%r5,%r3) /* Load s. */ + vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound_v16_0 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3) /* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */ + vfenezbs %v17,%v18,%v18 + je .Lfound_v18_16 + vl %v16,32(%r5,%r3) + vst %v18,16(%r5,%r2) + vfenezbs %v17,%v16,%v16 + je .Lfound_v16_32 + vl %v18,48(%r5,%r3) + vst %v16,32(%r5,%r2) + vfenezbs %v17,%v18,%v18 + je .Lfound_v18_48 + vst %v18,48(%r5,%r2) + + aghi %r5,64 + j .Lloop /* No zero found -> loop. */ + +.Lfound_v16_32: + aghi %r5,32 +.Lfound_v16_0: + la %r3,0(%r5,%r2) + vlgvb %r1,%v17,7 /* Load byte index of zero. */ + vstl %v16,%r1,0(%r3) /* Copy characters including zero. */ + la %r2,0(%r1,%r3) /* Return pointer to zero. */ + br %r14 + +.Lfound_v18_48: + aghi %r5,32 +.Lfound_v18_16: + la %r3,16(%r5,%r2) + vlgvb %r1,%v17,7 /* Load byte index of zero. */ + vstl %v18,%r1,0(%r3) /* Copy characters including zero. */ + la %r2,0(%r1,%r3) /* Return pointer to zero. */ + br %r14 + +.Lfound_align: + vstl %v16,%r5,0(%r2) /* Copy characters including zero. */ + la %r2,0(%r5,%r2) /* Return pointer to zero. */ + br %r14 +END(__stpcpy_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/stpcpy.c b/sysdeps/s390/multiarch/stpcpy.c new file mode 100644 index 0000000000..dcde01278b --- /dev/null +++ b/sysdeps/s390/multiarch/stpcpy.c @@ -0,0 +1,30 @@ +/* Multiple versions of stpcpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define NO_MEMPCPY_STPCPY_REDIRECT +# include <string.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc (__stpcpy) +weak_alias (__stpcpy, stpcpy) +libc_hidden_builtin_def (stpcpy) + +#else +# include <string/stpcpy.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/stpncpy-c.c b/sysdeps/s390/multiarch/stpncpy-c.c new file mode 100644 index 0000000000..32b61a8e3e --- /dev/null +++ b/sysdeps/s390/multiarch/stpncpy-c.c @@ -0,0 +1,28 @@ +/* Default stpncpy implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define STPNCPY __stpncpy_c +# ifdef SHARED +# undef libc_hidden_def +# define libc_hidden_def(name) \ + __hidden_ver1 (__stpncpy_c, __GI___stpncpy, __stpncpy_c); +# endif /* SHARED */ + +# include <string/stpncpy.c> +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/stpncpy-vx.S b/sysdeps/s390/multiarch/stpncpy-vx.S new file mode 100644 index 0000000000..2e536d9e0f --- /dev/null +++ b/sysdeps/s390/multiarch/stpncpy-vx.S @@ -0,0 +1,200 @@ +/* Vector optimized 32/64 bit S/390 version of stpncpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* char * stpncpy (char *dest, const char *src, size_t n) + Copies at most n characters of string src to dest + returning a pointer to its end or dest+n + if src is smaller than n. + + Register usage: + -%r0 = return value + -%r1 = zero byte index + -%r2 = curr dst pointer + -%r3 = curr src pointer + -%r4 = n + -%r5 = current_len + -%r6 = loaded bytes + -%r7 = border, tmp +*/ +ENTRY(__stpncpy_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + +# if !defined __s390x__ + llgfr %r4,%r4 +# endif /* !defined __s390x__ */ + + clgfi %r4,0 + ber %r14 /* Nothing to do, if n == 0. */ + + la %r0,0(%r4,%r2) /* Save destination pointer + n for return. */ + vlvgp %v31,%r6,%r7 /* Save registers. */ + + vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */ + lcbb %r6,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */ + llgfr %r6,%r6 /* Convert 32bit to 64bit. */ + + lghi %r5,0 /* current_len = 0. */ + + clgrjle %r4,%r6,.Lremaining_v16 /* If n <= loaded-bytes + -> process remaining. */ + + /* n > loaded-byte-count */ + vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */ + clrjl %r1,%r6,.Lfound_v16_store /* Found zero within loaded bytes, + copy and return. */ + + /* Align s to 16 byte. */ + risbgn %r7,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,15 /* current_len = 15. */ + slr %r5,%r7 /* Compute highest index to 16byte boundary. */ + + /* Zero not found and n > loaded-byte-count. */ + vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */ + ahi %r5,1 /* Start loop at next character. */ + + /* Now we are 16byte aligned, so we can load a full vreg + without page fault. */ + lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */ + aghi %r1,64 + clgrjl %r1,%r4,.Lloop64 + + vl %v16,0(%r5,%r3) /* Load s. */ + clgijl %r4,17,.Lremaining_v16 /* If n <= 16, process remaining + bytes. */ +.Llt64: + lgr %r7,%r4 + slgfi %r7,16 /* border_len = n - 16. */ + + clgrjhe %r5,%r7,.Lremaining_v16 /* If current_len >= border + then process remaining bytes. */ + vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound_v16 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3) /* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Save previous part without zero to dst. */ + aghi %r5,16 + + clgrjhe %r5,%r7,.Lremaining_v18 + vfenezbs %v17,%v18,%v18 + je .Lfound_v18 + vl %v16,16(%r5,%r3) + vst %v18,0(%r5,%r2) + aghi %r5,16 + + clgrjhe %r5,%r7,.Lremaining_v16 + vfenezbs %v17,%v16,%v16 + je .Lfound_v16 + vl %v18,16(%r5,%r3) + vst %v16,0(%r5,%r2) + aghi %r5,16 + +.Lremaining_v18: + vlr %v16,%v18 +.Lremaining_v16: + /* v16 contains the remaining bytes [1...16]. + Store remaining bytes and append string-termination. */ + vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */ + slgrk %r7,%r4,%r5 /* Remaining bytes = maxlen - current_len */ + aghi %r7,-1 /* vstl needs highest index. */ + la %r2,0(%r5,%r2) /* vstl has no index register. */ + vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */ + /* Zero in remaining bytes? -> jump away (zero-index <= max-index). */ + clrjle %r1,%r7,.Lfound_v16_store + vstl %v16,%r7,0(%r2) /* Store remaining bytes without null + termination! */ +.Lend: + /* Restore saved registers. */ + vlgvg %r6,%v31,0 + vlgvg %r7,%v31,1 + lgr %r2,%r0 /* Load saved dest-ptr. */ + br %r14 + +.Lfound_v16_32: + aghi %r5,32 + j .Lfound_v16 +.Lfound_v18_48: + aghi %r5,32 +.Lfound_v18_16: + aghi %r5,16 +.Lfound_v18: + vlr %v16,%v18 +.Lfound_v16: + /* v16 contains a zero. Store remaining bytes to zero. current_len + has not reached border, thus checking for n is not needed! */ + vlgvb %r1,%v17,7 /* Load byte index of zero. */ + la %r2,0(%r5,%r2) /* vstl has no support for index-register. */ +.Lfound_v16_store: + vstl %v16,%r1,0(%r2) /* Copy characters including zero. */ + /* Fill remaining bytes with zero - remaining count always > 0. */ + algr %r5,%r1 /* Remaining bytes (=%r4) = ... */ + slgr %r4,%r5 /* = maxlen - (currlen + zero_index + 1) */ + la %r2,0(%r1,%r2) /* Pointer to zero. start filling beyond. */ + lgr %r0,%r2 /* Save return-pointer to found zero. */ + clgije %r4,1,.Lend /* Skip zero-filling, if found zero is last + possible character. + (1 is substracted from r4 below!). */ + aghi %r4,-2 /* mvc with exrl needs count - 1. + (additional -1, see remaining bytes above) */ + srlg %r6,%r4,8 /* Split into 256 byte blocks. */ + ltgr %r6,%r6 + je .Lzero_lt256 +.Lzero_loop256: + mvc 1(256,%r2),0(%r2) /* Fill 256 zeros at once. */ + la %r2,256(%r2) + brctg %r6,.Lzero_loop256 /* Loop until all blocks are processed. */ +.Lzero_lt256: + exrl %r4,.Lmvc_lt256 + j .Lend +.Lmvc_lt256: + mvc 1(1,%r2),0(%r2) + +.Lloop64: + vl %v16,0(%r5,%r3) + vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound_v16 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3) /* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Save previous part without zero to dst. */ + vfenezbs %v17,%v18,%v18 + je .Lfound_v18_16 + vl %v16,32(%r5,%r3) + vst %v18,16(%r5,%r2) + vfenezbs %v17,%v16,%v16 + je .Lfound_v16_32 + vl %v18,48(%r5,%r3) + vst %v16,32(%r5,%r2) + vfenezbs %v17,%v18,%v18 + je .Lfound_v18_48 + vst %v18,48(%r5,%r2) + + aghi %r5,64 + lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */ + aghi %r1,64 + clgrjl %r1,%r4,.Lloop64 + + vl %v16,0(%r5,%r3) /* Load s. */ + j .Llt64 +END(__stpncpy_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/stpncpy.c b/sysdeps/s390/multiarch/stpncpy.c new file mode 100644 index 0000000000..f5335b42ac --- /dev/null +++ b/sysdeps/s390/multiarch/stpncpy.c @@ -0,0 +1,28 @@ +/* Multiple versions of stpncpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc (__stpncpy) +weak_alias (__stpncpy, stpncpy) + +#else +# include <string/stpncpy.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/strcat-c.c b/sysdeps/s390/multiarch/strcat-c.c new file mode 100644 index 0000000000..ae7cc2149d --- /dev/null +++ b/sysdeps/s390/multiarch/strcat-c.c @@ -0,0 +1,28 @@ +/* Default strcat implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define STRCAT __strcat_c +# ifdef SHARED +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) \ + __hidden_ver1 (__strcat_c, __GI_strcat, __strcat_c); +# endif /* SHARED */ + +# include <string/strcat.c> +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strcat-vx.S b/sysdeps/s390/multiarch/strcat-vx.S new file mode 100644 index 0000000000..e77fc2aa2f --- /dev/null +++ b/sysdeps/s390/multiarch/strcat-vx.S @@ -0,0 +1,161 @@ +/* Vector optimized 32/64 bit S/390 version of strcat. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* char * strcat (const char *dest, const char *src) + Concatenate two strings. + + Register usage: + -r0=saved dest pointer for return + -r1=tmp + -r2=dest + -r3=src + -r4=tmp + -r5=current_len + -v16=part of src + -v17=index of zero + -v18=part of src +*/ +ENTRY(__strcat_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + lgr %r0,%r2 /* Save destination pointer for return. */ + + /* STRLEN + r1 = loaded bytes (tmp) + r4 = zero byte index (tmp) + r2 = dst + */ + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + vfenezb %v16,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r5,%v16,7 /* Load zero index or 16 if not found. */ + clrjl %r5,%r1,.Llen_end /* Found zero within loaded bytes, end. */ + + /* Align s to 16 byte. */ + risbgn %r1,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,16 /* current_len = 16. */ + slr %r5,%r1 /* Compute bytes to 16bytes boundary. */ + + /* Find zero in 16byte aligned loop. */ +.Llen_loop: + vl %v16,0(%r5,%r2) /* Load s. */ + vfenezbs %v16,%v16,%v16 /* Find element not equal with zero search. */ + je .Llen_found /* Jump away if zero was found. */ + vl %v16,16(%r5,%r2) + vfenezbs %v16,%v16,%v16 + je .Llen_found16 + vl %v16,32(%r5,%r2) + vfenezbs %v16,%v16,%v16 + je .Llen_found32 + vl %v16,48(%r5,%r2) + vfenezbs %v16,%v16,%v16 + je .Llen_found48 + + aghi %r5,64 + j .Llen_loop /* No zero -> loop. */ + +.Llen_found48: + aghi %r5,16 +.Llen_found32: + aghi %r5,16 +.Llen_found16: + aghi %r5,16 +.Llen_found: + vlgvb %r4,%v16,7 /* Load byte index of zero. */ + algr %r5,%r4 + +.Llen_end: + /* STRCPY + %r1 = loaded bytes (tmp) + %r4 = zero byte index (tmp) + %r3 = curr src pointer + %r2 = curr dst pointer + */ + la %r2,0(%r5,%r2) /* strcpy at end of dst-string. */ + + vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */ + + vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r5,%v17,7 /* Load zero index or 16 if not found. */ + clrjl %r5,%r1,.Lcpy_found_align /* If found zero within loaded bytes, + copy bytes before and return. */ + + /* Align s to 16 byte. */ + risbgn %r4,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,15 /* current_len = 15. */ + slr %r5,%r4 /* Compute highest index to 16byte boundary. */ + + vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */ + ahi %r5,1 /* Start loop at next character. */ + + /* Find zero in 16byte aligned loop. */ +.Lcpy_loop: + vl %v16,0(%r5,%r3) /* Load s. */ + vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lcpy_found_v16_0 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3)/* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */ + vfenezbs %v17,%v18,%v18 + je .Lcpy_found_v18_16 + vl %v16,32(%r5,%r3) + vst %v18,16(%r5,%r2) + vfenezbs %v17,%v16,%v16 + je .Lcpy_found_v16_32 + vl %v18,48(%r5,%r3) + vst %v16,32(%r5,%r2) + vfenezbs %v17,%v18,%v18 + je .Lcpy_found_v18_48 + vst %v18,48(%r5,%r2) + + aghi %r5,64 + j .Lcpy_loop /* No zero -> loop. */ + +.Lcpy_found_v16_32: + aghi %r5,32 +.Lcpy_found_v16_0: + la %r4,0(%r5,%r2) + vlgvb %r1,%v17,7 /* Load byte index of zero. */ + vstl %v16,%r1,0(%r4) /* Copy characters including zero. */ + lgr %r2,%r0 /* Load saved dest-ptr. */ + br %r14 + +.Lcpy_found_v18_48: + aghi %r5,32 +.Lcpy_found_v18_16: + la %r4,16(%r5,%r2) + vlgvb %r1,%v17,7 /* Load byte index of zero. */ + vstl %v18,%r1,0(%r4) /* Copy characters including zero. */ + lgr %r2,%r0 /* Load saved dest-ptr. */ + br %r14 + +.Lcpy_found_align: + vstl %v16,%r5,0(%r2) /* Copy characters including zero. */ + lgr %r2,%r0 /* Load saved dest-ptr. */ + br %r14 +END(__strcat_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strcat.c b/sysdeps/s390/multiarch/strcat.c new file mode 100644 index 0000000000..c3b5e1c9d6 --- /dev/null +++ b/sysdeps/s390/multiarch/strcat.c @@ -0,0 +1,27 @@ +/* Multiple versions of strcat. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__strcat, strcat) + +#else +# include <string/strcat.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/strchr-c.c b/sysdeps/s390/multiarch/strchr-c.c new file mode 100644 index 0000000000..2250dbbf5e --- /dev/null +++ b/sysdeps/s390/multiarch/strchr-c.c @@ -0,0 +1,29 @@ +/* Default strchr implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define STRCHR __strchr_c +# undef weak_alias +# ifdef SHARED +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) \ + __hidden_ver1 (__strchr_c, __GI_strchr, __strchr_c); +# endif /* SHARED */ + +# include <string/strchr.c> +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strchr-vx.S b/sysdeps/s390/multiarch/strchr-vx.S new file mode 100644 index 0000000000..4fe5dc0293 --- /dev/null +++ b/sysdeps/s390/multiarch/strchr-vx.S @@ -0,0 +1,100 @@ +/* Vector optimized 32/64 bit S/390 version of strchr. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* char *strchr (const char *s, int c) + Locate character in string. + + Register usage: + -r1=tmp + -r2=s + -r3=c + -r4=tmp + -r5=current_len + -v16=part of s + -v17=index of unequal + -v18=replicated c +*/ +ENTRY(__strchr_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + lghi %r5,0 /* current_len = 0. */ + + vlvgb %v18,%r3,0 /* Generate vector which elements are all c. + If c > 255, c will be truncated. */ + vrepb %v18,%v18,0 + + vfeezbs %v16,%v16,%v18 /* Find element equal with zero search. */ + vlgvb %r4,%v16,7 /* Load byte index of character or zero. */ + clrjl %r4,%r1,.Lfound /* Return if c/zero is in loaded bytes. */ + + /* Align s to 16 byte. */ + risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,16 /* current_len = 16. */ + slr %r5,%r4 /* Compute bytes to 16bytes boundary. */ + + /* Find c/zero in 16 byte aligned loop */ +.Lloop: + vl %v16,0(%r5,%r2) /* Load s. */ + vfeezbs %v16,%v16,%v18 /* Find element equal with zero search. */ + jno .Lfound /* Found c/zero (cc=0|1|2). */ + vl %v16,16(%r5,%r2) + vfeezbs %v16,%v16,%v18 + jno .Lfound16 + vl %v16,32(%r5,%r2) + vfeezbs %v16,%v16,%v18 + jno .Lfound32 + vl %v16,48(%r5,%r2) + vfeezbs %v16,%v16,%v18 + jno .Lfound48 + + aghi %r5,64 + j .Lloop /* No character and no zero -> loop. */ + +.Lfound48: + la %r5,16(%r5) /* Use la since aghi would clobber cc. */ +.Lfound32: + la %r5,16(%r5) +.Lfound16: + la %r5,16(%r5) +.Lfound: + je .Lzero /* Found zero, but no c before that zero. */ + +.Lcharacter: + vlgvb %r4,%v16,7 /* Load byte index of character. */ + algr %r5,%r4 + la %r2,0(%r5,%r2) /* Return pointer to character. */ + br %r14 + +.Lzero: + llgcr %r3,%r3 /* char c_char = (char) c. */ + clije %r3,0,.Lcharacter /* Found zero and c is zero. */ + lghi %r2,0 /* Return null if character not found. */ + br %r14 +END(__strchr_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strchr.c b/sysdeps/s390/multiarch/strchr.c new file mode 100644 index 0000000000..3c8c7e4600 --- /dev/null +++ b/sysdeps/s390/multiarch/strchr.c @@ -0,0 +1,28 @@ +/* Multiple versions of strchr. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__strchr, strchr) +weak_alias (strchr, index) + +#else +# include <string/strchr.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/strchrnul-c.c b/sysdeps/s390/multiarch/strchrnul-c.c new file mode 100644 index 0000000000..1f77c40cea --- /dev/null +++ b/sysdeps/s390/multiarch/strchrnul-c.c @@ -0,0 +1,26 @@ +/* Default strchrnul implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define STRCHRNUL __strchrnul_c +# define __strchrnul STRCHRNUL +# undef weak_alias +# define weak_alias(name, alias) + +# include <string/strchrnul.c> +#endif diff --git a/sysdeps/s390/multiarch/strchrnul-vx.S b/sysdeps/s390/multiarch/strchrnul-vx.S new file mode 100644 index 0000000000..43ca29ead0 --- /dev/null +++ b/sysdeps/s390/multiarch/strchrnul-vx.S @@ -0,0 +1,93 @@ +/* Vector optimized 32/64 bit S/390 version of strchrnul. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* char *strchrnul (const char *s, int c) + Returns pointer to first c or to \0 if c not found. + + Register usage: + -r1=tmp + -r2=s and return pointer + -r3=c + -r4=tmp + -r5=current_len + -v16=part of s + -v18=vector with c replicated in every byte +*/ +ENTRY(__strchrnul_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + lghi %r5,0 /* current_len = 0. */ + + vlvgb %v18,%r3,0 /* Generate vector which elements are all c. + If c > 255, c will be truncated. */ + vrepb %v18,%v18,0 + + vfeezbs %v16,%v16,%v18 /* Find element equal with zero search. */ + vlgvb %r4,%v16,7 /* Load byte index of character or zero. */ + clrjl %r4,%r1,.Lfound /* Return if c/zero is in loaded bytes. */ + + /* Align s to 16 byte. */ + risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,16 /* current_len = 16. */ + slr %r5,%r4 /* Compute bytes to 16bytes boundary. */ + + /* Find c/zero in 16byte aligned loop */ +.Lloop: + vl %v16,0(%r5,%r2) /* Load s */ + vfeezbs %v16,%v16,%v18 /* Find element equal with zero search. */ + jno .Lfound /* Found c/zero (cc=0|1|2). */ + vl %v16,16(%r5,%r2) + vfeezbs %v16,%v16,%v18 + jno .Lfound16 + vl %v16,32(%r5,%r2) + vfeezbs %v16,%v16,%v18 + jno .Lfound32 + vl %v16,48(%r5,%r2) + vfeezbs %v16,%v16,%v18 + jno .Lfound48 + + aghi %r5,64 + j .Lloop /* No character and no zero -> loop. */ + + /* Found character or zero */ +.Lfound48: + aghi %r5,16 +.Lfound32: + aghi %r5,16 +.Lfound16: + aghi %r5,16 +.Lfound: + vlgvb %r1,%v16,7 /* Load byte index of character. */ + algr %r5,%r1 + la %r2,0(%r5,%r2) /* Return pointer to character. */ + +.Lend: + br %r14 +END(__strchrnul_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strchrnul.c b/sysdeps/s390/multiarch/strchrnul.c new file mode 100644 index 0000000000..627c084521 --- /dev/null +++ b/sysdeps/s390/multiarch/strchrnul.c @@ -0,0 +1,28 @@ +/* Multiple versions of strchrnul. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc (__strchrnul) +weak_alias (__strchrnul, strchrnul) + +#else +# include <string/strchrnul.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/strcmp-vx.S b/sysdeps/s390/multiarch/strcmp-vx.S new file mode 100644 index 0000000000..edf557b5eb --- /dev/null +++ b/sysdeps/s390/multiarch/strcmp-vx.S @@ -0,0 +1,116 @@ +/* Vector optimized 32/64 bit S/390 version of strcmp. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* int strcmp (const char *s1, const char *s2) + Compare two strings + + Register usage: + -r1=loaded byte count s1 + -r2=s1 + -r3=s2 + -r4=loaded byte coutn s2, tmp + -r5=current_len + -v16=part of s1 + -v17=part of s2 + -v18=index of unequal +*/ +ENTRY(__strcmp_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + lghi %r5,0 /* current_len = 0. */ + +.Lloop: + vlbb %v16,0(%r5,%r2),6 /* Load s1 to block boundary. */ + vlbb %v17,0(%r5,%r3),6 /* Load s2 to block boundary. */ + lcbb %r1,0(%r5,%r2),6 /* Get loaded byte count of s1. */ + jo .Llt16_1 /* Jump away if vr is not fully loaded. */ + lcbb %r4,0(%r5,%r3),6 + jo .Llt16_2 /* Jump away if vr is not fully loaded. */ + /* Both vrs are fully loaded. */ + aghi %r5,16 + vfenezbs %v18,%v16,%v17 /* Compare not equal with zero search. */ + jno .Lfound + + vlbb %v16,0(%r5,%r2),6 + vlbb %v17,0(%r5,%r3),6 + lcbb %r1,0(%r5,%r2),6 + jo .Llt16_1 + lcbb %r4,0(%r5,%r3),6 + jo .Llt16_2 + aghi %r5,16 + vfenezbs %v18,%v16,%v17 + jno .Lfound + + vlbb %v16,0(%r5,%r2),6 + vlbb %v17,0(%r5,%r3),6 + lcbb %r1,0(%r5,%r2),6 + jo .Llt16_1 + lcbb %r4,0(%r5,%r3),6 + jo .Llt16_2 + aghi %r5,16 + vfenezbs %v18,%v16,%v17 + jno .Lfound + + vlbb %v16,0(%r5,%r2),6 + vlbb %v17,0(%r5,%r3),6 + lcbb %r1,0(%r5,%r2),6 + jo .Llt16_1 + lcbb %r4,0(%r5,%r3),6 + jo .Llt16_2 + aghi %r5,16 + vfenezbs %v18,%v16,%v17 + jno .Lfound + j .Lloop + +.Llt16_1: + lcbb %r4,0(%r5,%r3),6 /* Get loaded byte count of s2. */ +.Llt16_2: + clr %r1,%r4 + locrh %r1,%r4 /* Get minimum of bytes loaded in s1/2. */ + algfr %r5,%r1 /* Add smallest loaded bytes to current_len. */ + vfenezbs %v18,%v16,%v17 /* Compare not equal with zero search. */ + vlgvb %r4,%v18,7 /* Get not equal index or 16 if all equal. */ + clrjl %r4,%r1,.Lfound /* Jump away if miscompare is within loaded + bytes. */ + j .Lloop + +.Lfound: + je .Lend_equal + lghi %r2,1 + lghi %r1,-1 + locgrl %r2,%r1 + br %r14 +.Lend_equal: + lghi %r2,0 + br %r14 +END(__strcmp_vx) + +# define strcmp __strcmp_c +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) strong_alias(__strcmp_c, __GI_strcmp) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ + +#include <strcmp.S> diff --git a/sysdeps/s390/multiarch/strcmp.c b/sysdeps/s390/multiarch/strcmp.c new file mode 100644 index 0000000000..c4ccd34420 --- /dev/null +++ b/sysdeps/s390/multiarch/strcmp.c @@ -0,0 +1,26 @@ +/* Multiple versions of strcmp. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + + +# undef strcmp +s390_vx_libc_ifunc2 (__strcmp, strcmp) +#endif diff --git a/sysdeps/s390/multiarch/strcpy-vx.S b/sysdeps/s390/multiarch/strcpy-vx.S new file mode 100644 index 0000000000..d3472b821d --- /dev/null +++ b/sysdeps/s390/multiarch/strcpy-vx.S @@ -0,0 +1,109 @@ +/* Vector optimized 32/64 bit S/390 version of strcpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* char * strcpy (const char *dest, const char *src) + Copy string src to dest. + + Register usage: + -r1=tmp + -r2=dest and return_value + -r3=src + -r4=tmp + -r5=current_len + -v16=part of src + -v17=index of zero + -v18=part of src +*/ +ENTRY(__strcpy_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */ + + vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r5,%v17,7 /* Load zero index or 16 if not found. */ + clrjl %r5,%r1,.Lfound_align /* If found zero within loaded bytes, + copy bytes before and return. */ + + /* Align s to 16 byte. */ + risbgn %r4,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,15 /* current_len = 15. */ + slr %r5,%r4 /* Compute highest index to 16byte boundary. */ + + vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */ + ahi %r5,1 /* Start loop at next character. */ + + /* Find zero in 16byte aligned loop. */ +.Lloop: + vl %v16,0(%r5,%r3) /* Load s. */ + vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound_v16_0 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3)/* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */ + vfenezbs %v17,%v18,%v18 + je .Lfound_v18_16 + vl %v16,32(%r5,%r3) + vst %v18,16(%r5,%r2) + vfenezbs %v17,%v16,%v16 + je .Lfound_v16_32 + vl %v18,48(%r5,%r3) + vst %v16,32(%r5,%r2) + vfenezbs %v17,%v18,%v18 + je .Lfound_v18_48 + vst %v18,48(%r5,%r2) + + aghi %r5,64 + j .Lloop /* No zero found -> loop. */ + +.Lfound_v16_32: + aghi %r5,32 +.Lfound_v16_0: + la %r3,0(%r5,%r2) + vlgvb %r4,%v17,7 /* Load byte index of zero. */ + vstl %v16,%r4,0(%r3) /* Store characters including zero. */ + br %r14 + +.Lfound_v18_48: + aghi %r5,32 +.Lfound_v18_16: + la %r3,16(%r5,%r2) + vlgvb %r4,%v17,7 /* Load byte index of zero. */ + vstl %v18,%r4,0(%r3) /* Store characters including zero. */ + br %r14 + +.Lfound_align: + vstl %v16,%r5,0(%r2) /* Copy characters including zero. */ + br %r14 +END(__strcpy_vx) + +/* Use mvst-strcpy-implementation as default implementation. */ +# define strcpy __strcpy_c +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) strong_alias(__strcpy_c, __GI_strcpy) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ + +/* Include mvst-strcpy-implementation in s390-32/s390-64 subdirectory. */ +#include <strcpy.S> diff --git a/sysdeps/s390/multiarch/strcpy.c b/sysdeps/s390/multiarch/strcpy.c new file mode 100644 index 0000000000..f348199112 --- /dev/null +++ b/sysdeps/s390/multiarch/strcpy.c @@ -0,0 +1,24 @@ +/* Multiple versions of strcpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__strcpy, strcpy) +#endif diff --git a/sysdeps/s390/multiarch/strcspn-c.c b/sysdeps/s390/multiarch/strcspn-c.c new file mode 100644 index 0000000000..bc195b6625 --- /dev/null +++ b/sysdeps/s390/multiarch/strcspn-c.c @@ -0,0 +1,28 @@ +/* Default strcspn implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define STRCSPN __strcspn_c +# ifdef SHARED +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) \ + __hidden_ver1 (__strcspn_c, __GI_strcspn, __strcspn_c); +# endif /* SHARED */ + +# include <string/strcspn.c> +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strcspn-vx.S b/sysdeps/s390/multiarch/strcspn-vx.S new file mode 100644 index 0000000000..1c6250661e --- /dev/null +++ b/sysdeps/s390/multiarch/strcspn-vx.S @@ -0,0 +1,281 @@ +/* Vector optimized 32/64 bit S/390 version of strcspn. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* size_t strcspn (const char *s, const char * reject) + The strcspn() function calculates the length of the initial segment + of s which consists entirely of characters not in reject. + + This method checks the length of reject string. If it fits entirely + in one vector register, a fast algorithm is used, which does not need + to check multiple parts of accept-string. Otherwise a slower full + check of accept-string is used. + + register overview: + r3: pointer to start of reject-string + r2: pointer to start of search-string + r0: loaded byte count of vlbb search-string + r4: found byte index + r1: current return len + v16: search-string + v17: reject-string + v18: temp-vreg + + ONLY FOR SLOW: + v19: first reject-string + v20: zero for preparing acc-vector + v21: global mask; 1 indicates a match between + search-string-vreg and any reject-character + v22: current mask; 1 indicates a match between + search-string-vreg and any reject-character in current acc-vreg + v24: one for result-checking of former string-part + v30, v31: for re-/storing registers r6, r8, r9 + r5: current len of reject-string + r6: zero-index in search-string or 16 if no zero + or min(zero-index, loaded byte count) + r8: >0, if former reject-string-part contains a zero, + otherwise =0; + r9: loaded byte count of vlbb reject-string +*/ +ENTRY(__strcspn_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + /* + Check if reject-string fits in one vreg: + ---------------------------------------- + */ + vlbb %v17,0(%r3),6 /* Load reject. */ + lghi %r1,0 /* Zero out current len. */ + lcbb %r0,0(%r3),6 + jo .Lcheck_onbb /* Special case if reject + lays on block-boundary. */ +.Lcheck_notonbb: + vistrbs %v17,%v17 /* Fill with zeros after first zero. */ + je .Lfast /* Zero found -> reject fits in one vreg. */ + j .Lslow /* No zero -> reject exceeds one vreg. */ + + +.Lcheck_onbb: + /* Reject lays on block-boundary. */ + vfenezb %v18,%v17,%v17 /* Search zero in loaded reject bytes. */ + vlgvb %r4,%v18,7 /* Get index of zero or 16 if not found. */ + clrjl %r4,%r0,.Lcheck_notonbb /* Zero index < loaded bytes count -> + Reject fits in one vreg; + Fill with zeros and proceed + with FAST. */ + vl %v17,0(%r3) /* Load reject, which exceeds loaded bytes. */ + j .Lcheck_notonbb /* Check if reject fits in one vreg. */ + + + /* + Search s for reject in one vreg + ------------------------------- + */ +.Lfast: + /* Complete reject-string in v17 and remaining bytes are zero. */ + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r0,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + vfaezbs %v18,%v16,%v17,0 /* Find first element in v16 + unequal to any in v17 + or first zero element. */ + + vlgvb %r4,%v18,7 /* Load byte index of found element. */ + clrjl %r4,%r0,.Lfast_loop_found2 /* If found index is within loaded + bytes, return with found element + index (=equal count). */ + + /* Align s to 16 byte. */ + risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r1,16 /* current_len = 16. */ + slr %r1,%r4 /* Compute bytes to 16bytes boundary. */ + + /* Process s in 16byte aligned loop. */ +.Lfast_loop: + vl %v16,0(%r1,%r2) /* Load search-string. */ + vfaezbs %v18,%v16,%v17,0 /* Find first element in v16 equal to any + in v17 or first zero element. */ + jno .Lfast_loop_found + + vl %v16,16(%r1,%r2) + vfaezbs %v18,%v16,%v17,0 + jno .Lfast_loop_found16 + + vl %v16,32(%r1,%r2) + vfaezbs %v18,%v16,%v17,0 + jno .Lfast_loop_found32 + + vl %v16,48(%r1,%r2) + vfaezbs %v18,%v16,%v17,0 + jno .Lfast_loop_found48 + + aghi %r1,64 + j .Lfast_loop /* Loop if no element was unequal to reject + and not zero. */ + + /* Found equal or zero element. */ +.Lfast_loop_found48: + aghi %r1,16 +.Lfast_loop_found32: + aghi %r1,16 +.Lfast_loop_found16: + aghi %r1,16 +.Lfast_loop_found: + vlgvb %r4,%v18,7 /* Load byte index of found element or zero. */ +.Lfast_loop_found2: + algrk %r2,%r1,%r4 /* Add found index to current len. */ + br %r14 + + + + /* + Search s for reject in multiple vregs + ------------------------------------- + */ +.Lslow: + /* Save registers. */ + vlvgg %v30,%r6,0 + vlvgp %v31,%r8,%r9 + + /* Reject in v17 without zero. */ + vlr %v19,%v17 /* Save first acc-part for a fast reload. */ + vzero %v20 /* Zero for preparing acc-vector. */ + vone %v24 /* One for checking result of former + string-part. */ + + /* Align s to 16 byte. */ + risbg %r4,%r2,60,128+63,0 /* Test if s is aligned and + %r4 = bits 60-63 'and' 15. */ + je .Lslow_loop_str /* If s is aligned, loop aligned. */ + lghi %r0,15 + slr %r0,%r4 /* Compute highest index to load (15-x). */ + vll %v16,%r0,0(%r2) /* Load up to 16 byte boundary (vll needs + highest index, remaining bytes are 0). */ + ahi %r0,1 /* Work with loaded byte count. */ + vzero %v21 /* Zero out global mask. */ + lghi %r5,0 /* Set current len of reject-string to zero. */ + vfenezb %v18,%v16,%v16 /* Find zero in current string-part. */ + lghi %r8,0 /* There is no zero in first reject-part. */ + vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */ + clije %r6,0,.Lslow_end /* If first element is zero -> return 0. */ + clr %r0,%r6 /* cc==1 if loaded byte count < zero-index. */ + locrl %r6,%r0 /* Load on cc==1; zero-index = lbc. */ + j .Lslow_loop_acc + + + /* Process s in 16byte aligned loop. */ +.Lslow_next_str: + /* Check results of former processed str-part. */ + vfeeb %v18,%v21,%v24 /* Find first equal match in global mask + (ones in element). */ + vlgvb %r4,%v18,7 /* Get index of first one (=equal) or 16. */ + /* Equal-index < min(zero-index, loaded byte count) + -> Return pointer to equal element. */ + clrjl %r4,%r6,.Lslow_index_found + /* Zero-index < loaded byte count + -> Former str-part was last str-part + -> Return null */ + clrjl %r6,%r0,.Lslow_end_not_found + + /* All elements are zero (=no match) -> Proceed with next str-part. */ + vlr %v17,%v19 /* Load first part of reject (no zero). */ + algfr %r1,%r0 /* Add loaded byte count to current len. */ + +.Lslow_loop_str: + vl %v16,0(%r1,%r2) /* Load search-string. */ + lghi %r0,16 /* Loaded byte count is 16. */ + vzero %v21 /* Zero out global mask. */ + lghi %r5,0 /* Set current len of reject to zero. */ + vfenezb %v18,%v16,%v16 /* Find zero in current string-part. */ + lghi %r8,0 /* There is no zero in first reject-part. */ + vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */ + clije %r6,0,.Lslow_end /* If first element is zero (end of string) + -> Return current length. */ + +.Lslow_loop_acc: + vfaeb %v22,%v16,%v17,4 /* Create matching-mask (1 in mask -> + Character matches any rejected character in + this reject-string-part) IN=0, RT=1. */ + vlgvb %r4,%v22,0 /* Get result of first element. */ + /* First element is equal to any rejected characters? + (all other parts of reject cannot lead to a match before this one) + -> Return current len, which is pointing to this element. */ + clijh %r4,0,.Lslow_end + vo %v21,%v21,%v22 /* Global-mask = global-|matching-mask. */ + /* Proceed with next acc until end of acc is reached. */ + + +.Lslow_next_acc: + clijh %r8,0,.Lslow_next_str /* There was a zero in last reject-part + -> Add found index to current len + and end. */ + vlbb %v17,16(%r5,%r3),6 /* Load next reject part. */ + aghi %r5,16 /* Increment current len of reject-string. */ + lcbb %r9,0(%r5,%r3),6 /* Get loaded byte count of reject-string. */ + jo .Lslow_next_acc_onbb /* Jump away if reject-string is + on block-boundary. */ +.Lslow_next_acc_notonbb: + vistrbs %v17,%v17 /* Fill with zeros after first zero. */ + jo .Lslow_loop_acc /* No zero found -> no preparation needed. */ + +.Lslow_next_acc_prepare_zero: + /* Zero in reject-part: fill zeros with first-reject-character. */ + vlgvb %r8,%v17,0 /* Load first element of reject-part. */ + clije %r8,0,.Lslow_next_str /* Process next str-part if first + character in this part of reject + is a zero. */ + /* r8>0 -> zero found in this acc-part. */ + vrepb %v18,%v17,0 /* Replicate first char accross all chars. */ + vceqb %v22,%v20,%v17 /* Create a mask (v22) of null chars + by comparing with 0 (v20). */ + vsel %v17,%v18,%v17,%v22 /* Replace null chars with first char. */ + j .Lslow_loop_acc /* Reject-string part is prepared. */ + +.Lslow_next_acc_onbb: + vfenezb %v18,%v17,%v17 /* Find zero in loaded bytes of reject part. */ + vlgvb %r8,%v18,7 /* Load byte index of zero. */ + clrjl %r8,%r9,.Lslow_next_acc_notonbb /* Found a zero in loaded bytes + -> Prepare vreg. */ + vl %v17,0(%r5,%r3) /* Load over boundary ... */ + lghi %r8,0 /* r8=0 -> no zero in this part of acc, + check for zero is in jump-target. */ + j .Lslow_next_acc_notonbb /* ... and search for zero in + fully loaded vreg again. */ + +.Lslow_end_not_found: + algfr %r1,%r6 /* Add zero-index to current len. */ + j .Lslow_end +.Lslow_index_found: + algfr %r1,%r4 /* Add found index of char to current len. */ +.Lslow_end: + lgr %r2,%r1 + /* Restore registers. */ + vlgvg %r6,%v30,0 + vlgvg %r8,%v31,0 + vlgvg %r9,%v31,1 + br %r14 +END(__strcspn_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strcspn.c b/sysdeps/s390/multiarch/strcspn.c new file mode 100644 index 0000000000..c23452a791 --- /dev/null +++ b/sysdeps/s390/multiarch/strcspn.c @@ -0,0 +1,27 @@ +/* Multiple versions of strcspn. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__strcspn, strcspn) + +#else +# include <string/strcspn.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/strlen-c.c b/sysdeps/s390/multiarch/strlen-c.c new file mode 100644 index 0000000000..63c0d9e3e6 --- /dev/null +++ b/sysdeps/s390/multiarch/strlen-c.c @@ -0,0 +1,28 @@ +/* Default strlen implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define STRLEN __strlen_c +# ifdef SHARED +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) \ + __hidden_ver1 (__strlen_c, __GI_strlen, __strlen_c); +# endif /* SHARED */ + +# include <string/strlen.c> +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strlen-vx.S b/sysdeps/s390/multiarch/strlen-vx.S new file mode 100644 index 0000000000..3fe834a0c7 --- /dev/null +++ b/sysdeps/s390/multiarch/strlen-vx.S @@ -0,0 +1,84 @@ +/* Vector optimized 32/64 bit S/390 version of strlen. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* size_t strlen (const char *s) + Returns length of string s. + + Register usage: + -r1=bytes to 4k-byte boundary + -r2=s + -r3=tmp + -r4=tmp + -r5=current_len and return_value + -v16=part of s +*/ +ENTRY(__strlen_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + vfenezb %v16,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r4,%v16,7 /* Load zero index or 16 if not found. */ + clr %r4,%r1 /* If found zero within loaded bytes? */ + locgrl %r2,%r4 /* Then copy return value. */ + blr %r14 /* And return. */ + + /* Align s to 16 byte. */ + risbgn %r3,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,16 /* current_len = 16. */ + slr %r5,%r3 /* Compute bytes to 16bytes boundary. */ + + /* Find zero in 16 byte aligned loop. */ +.Lloop: + vl %v16,0(%r5,%r2) /* Load s. */ + vfenezbs %v16,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound /* Jump away if zero was found. */ + vl %v16,16(%r5,%r2) + vfenezbs %v16,%v16,%v16 + je .Lfound16 + vl %v16,32(%r5,%r2) + vfenezbs %v16,%v16,%v16 + je .Lfound32 + vl %v16,48(%r5,%r2) + vfenezbs %v16,%v16,%v16 + je .Lfound48 + + aghi %r5,64 + j .Lloop /* No zero found -> loop. */ + +.Lfound48: + aghi %r5,16 +.Lfound32: + aghi %r5,16 +.Lfound16: + aghi %r5,16 +.Lfound: + vlgvb %r2,%v16,7 /* Load byte index of zero. */ + algr %r2,%r5 + br %r14 +END(__strlen_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strlen.c b/sysdeps/s390/multiarch/strlen.c new file mode 100644 index 0000000000..098d4e1e58 --- /dev/null +++ b/sysdeps/s390/multiarch/strlen.c @@ -0,0 +1,27 @@ +/* Multiple versions of strlen. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__strlen, strlen) + +#else +# include <string/strlen.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/strncat-c.c b/sysdeps/s390/multiarch/strncat-c.c new file mode 100644 index 0000000000..538b1fa51e --- /dev/null +++ b/sysdeps/s390/multiarch/strncat-c.c @@ -0,0 +1,23 @@ +/* Default strncat implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define STRNCAT __strncat_c + +# include <string/strncat.c> +#endif diff --git a/sysdeps/s390/multiarch/strncat-vx.S b/sysdeps/s390/multiarch/strncat-vx.S new file mode 100644 index 0000000000..b9857c1233 --- /dev/null +++ b/sysdeps/s390/multiarch/strncat-vx.S @@ -0,0 +1,239 @@ +/* Vector optimized 32/64 bit S/390 version of strncat. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* char * strncat (const char *dest, const char *src, size_t n) + Concatenate two strings - at most n characters of src. + + Register usage: + -r0=saved dest pointer for return + -r1=tmp + -r2=dest + -r3=src + -r4=n + -r5=current_len + -r6=tmp + -r7=tmp + -v16=part of src + -v17=index of zero + -v18=part of src + -v31=register save area for r6, r7 +*/ +ENTRY(__strncat_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + +# if !defined __s390x__ + llgfr %r4,%r4 +# endif /* !defined __s390x__ */ + + clgfi %r4,0 + ber %r14 /* Nothing to do, if n == 0. */ + lgr %r0,%r2 /* Save destination pointer for return. */ + vlvgp %v31,%r6,%r7 /* Save registers. */ + + /* STRLEN + %r1 = loaded bytes (tmp) + %r6 = zero byte index (tmp) + %r2 = dst + */ + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + vfenezb %v16,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r5,%v16,7 /* Load zero index or 16 if not found. */ + clrjl %r5,%r1,.Llen_end /* Found zero within loaded bytes, end. */ + + /* Align s to 16 byte. */ + risbgn %r1,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,16 /* current_len = 16. */ + slr %r5,%r1 /* Compute bytes to 16bytes boundary. */ + + /* Find zero in 16byte aligned loop. */ +.Llen_loop: + vl %v16,0(%r5,%r2) /* Load s. */ + vfenezbs %v16,%v16,%v16 /* Find element not equal with zero search. */ + je .Llen_found /* Jump away if zero was found. */ + vl %v16,16(%r5,%r2) + vfenezbs %v16,%v16,%v16 + je .Llen_found16 + vl %v16,32(%r5,%r2) + vfenezbs %v16,%v16,%v16 + je .Llen_found32 + vl %v16,48(%r5,%r2) + vfenezbs %v16,%v16,%v16 + je .Llen_found48 + + aghi %r5,64 + j .Llen_loop /* No zero -> loop. */ + +.Llen_found48: + aghi %r5,16 +.Llen_found32: + aghi %r5,16 +.Llen_found16: + aghi %r5,16 +.Llen_found: + vlgvb %r1,%v16,7 /* Load byte index of zero. */ + algr %r5,%r1 + +.Llen_end: + /* STRCPY + %r1 = zero byte index (tmp) + %r6 = loaded bytes (tmp) + %r3 = curr src pointer + %r2 = curr dst pointer + %r7 = border, tmp + */ + la %r2,0(%r5,%r2) /* strcpy at end of dst-string. */ + + vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */ + lcbb %r6,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */ + llgfr %r6,%r6 /* Convert 32bit to 64bit. */ + + lghi %r5,0 /* current_len = 0. */ + + clgrjle %r4,%r6,.Lcpy_remaining_v16 /* If n <= loaded-bytes + -> process remaining. */ + + /* n > loaded-byte-count. */ + vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */ + clrjl %r1,%r6,.Lcpy_found_v16_store /* Found zero within loaded + bytes, copy and return. */ + + /* Align s to 16 byte. */ + risbgn %r7,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,15 /* current_len = 15. */ + slr %r5,%r7 /* Compute highest index to 16byte boundary. */ + + /* Zero not found and n > loaded-byte-count. */ + vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */ + ahi %r5,1 /* Start loop at next character. */ + + /* + Now we are 16byte aligned, so we can load a full vreg + without page fault. + */ + lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */ + aghi %r1,64 + clgrjl %r1,%r4,.Lcpy_loop64 + + vl %v16,0(%r5,%r3) /* Load s. */ + clgijl %r4,17,.Lcpy_remaining_v16 /* If n <=16, + process remaining bytes. */ +.Lcpy_lt64: + lgr %r7,%r4 + slgfi %r7,16 /* border_len = n - 16. */ + + /* If current_len >= border then process remaining bytes. */ + clgrjhe %r5,%r7,.Lcpy_remaining_v16 + vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lcpy_found_v16 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3) /* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */ + aghi %r5,16 + + clgrjhe %r5,%r7,.Lcpy_remaining_v18 + vfenezbs %v17,%v18,%v18 + je .Lcpy_found_v18 + vl %v16,16(%r5,%r3) + vst %v18,0(%r5,%r2) + aghi %r5,16 + + clgrjhe %r5,%r7,.Lcpy_remaining_v16 + vfenezbs %v17,%v16,%v16 + je .Lcpy_found_v16 + vl %v18,16(%r5,%r3) + vst %v16,0(%r5,%r2) + aghi %r5,16 + +.Lcpy_remaining_v18: + vlr %v16,%v18 +.Lcpy_remaining_v16: + /* v16 contains the remaining bytes [1...16]. + Store remaining bytes and append string-termination. */ + vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */ + slgrk %r7,%r4,%r5 /* Remaining bytes = maxlen - current_len. */ + aghi %r7,-1 /* vstl needs highest index. */ + vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */ + la %r2,0(%r5,%r2) /* vstl has no index register. */ + /* Zero-index within remaining-bytes, store up to zero and end. */ + clgrjle %r1,%r7,.Lcpy_found_v16_store + vstl %v16,%r7,0(%r2) /* Store remaining bytes. */ + lghi %r1,0 + stc %r1,1(%r7,%r2) /* Store string-null-termination beyond n. */ +.Lcpy_end: + /* Restore saved registers. */ + vlgvg %r6,%v31,0 + vlgvg %r7,%v31,1 + lgr %r2,%r0 /* Load saved dest-ptr. */ + br %r14 + +.Lcpy_found_v16_32: + aghi %r5,32 + j .Lcpy_found_v16 +.Lcpy_found_v18_48: + aghi %r5,32 +.Lcpy_found_v18_16: + aghi %r5,16 +.Lcpy_found_v18: + vlr %v16,%v18 +.Lcpy_found_v16: + /* v16 contains a zero. Store remaining bytes to zero. current_len + has not reached border, thus checking for n is not needed! */ + vlgvb %r1,%v17,7 /* Load byte index of zero. */ + la %r2,0(%r5,%r2) +.Lcpy_found_v16_store: + vstl %v16,%r1,0(%r2) /* Copy characters including zero. */ + j .Lcpy_end + + /* Find zero in 16byte aligned loop. */ +.Lcpy_loop64: + vl %v16,0(%r5,%r3) /* Load s. */ + vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lcpy_found_v16 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3) /* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */ + vfenezbs %v17,%v18,%v18 + je .Lcpy_found_v18_16 + vl %v16,32(%r5,%r3) + vst %v18,16(%r5,%r2) + vfenezbs %v17,%v16,%v16 + je .Lcpy_found_v16_32 + vl %v18,48(%r5,%r3) + vst %v16,32(%r5,%r2) + vfenezbs %v17,%v18,%v18 + je .Lcpy_found_v18_48 + vst %v18,48(%r5,%r2) + + aghi %r5,64 + lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */ + aghi %r1,64 + clgrjl %r1,%r4,.Lcpy_loop64 + + vl %v16,0(%r5,%r3) /* Load s. */ + j .Lcpy_lt64 +END(__strncat_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strncat.c b/sysdeps/s390/multiarch/strncat.c new file mode 100644 index 0000000000..eb1410d5ac --- /dev/null +++ b/sysdeps/s390/multiarch/strncat.c @@ -0,0 +1,27 @@ +/* Multiple versions of strncat. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__strncat, strncat) + +#else +# include <string/strncat.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/strncmp-c.c b/sysdeps/s390/multiarch/strncmp-c.c new file mode 100644 index 0000000000..e781aefbe3 --- /dev/null +++ b/sysdeps/s390/multiarch/strncmp-c.c @@ -0,0 +1,28 @@ +/* Default strncmp implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define STRNCMP __strncmp_c +# ifdef SHARED +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) \ + __hidden_ver1 (__strncmp_c, __GI_strncmp, __strncmp_c); +# endif /* SHARED */ + +# include <string/strncmp.c> +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strncmp-vx.S b/sysdeps/s390/multiarch/strncmp-vx.S new file mode 100644 index 0000000000..9c4b207f41 --- /dev/null +++ b/sysdeps/s390/multiarch/strncmp-vx.S @@ -0,0 +1,137 @@ +/* Vector optimized 32/64 bit S/390 version of strncmp. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* int strncmp (const char *s1, const char *s2, size_t n) + Compare at most n characters of two strings. + + Register usage: + -r0=tmp + -r1=tmp + -r2=s1 + -r3=s2 + -r4=n + -r5=current_len + -v16=part of s1 + -v17=part of s2 + -v18=index of unequal +*/ +ENTRY(__strncmp_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + +# if !defined __s390x__ + llgfr %r4,%r4 +# endif /* !defined __s390x__ */ + + clgije %r4,0,.Lend_equal /* Nothing to do if n == 0, */ + lghi %r5,0 /* current_len = 0. */ + +.Lloop: + vlbb %v16,0(%r5,%r2),6 /* Load s1 to block boundary. */ + vlbb %v17,0(%r5,%r3),6 /* Load s2 to block boundary. */ + lcbb %r0,0(%r5,%r2),6 /* Get loaded byte count of s1. */ + jo .Llt16_1 /* Jump away if vr is not fully loaded. */ + lcbb %r1,0(%r5,%r3),6 /* Get loaded byte count of s2. */ + jo .Llt16_2 /* Jump away if vr is not fully loaded. */ + aghi %r5,16 /* Both vrs are fully loaded. */ + clgrjhe %r5,%r4,.Llastcmp /* If current_len >= n ->last compare. */ + vfenezbs %v18,%v16,%v17 /* Compare not equal with zero search. */ + jno .Lfound + + vlbb %v16,0(%r5,%r2),6 + vlbb %v17,0(%r5,%r3),6 + lcbb %r0,0(%r5,%r2),6 + jo .Llt16_1 + lcbb %r1,0(%r5,%r3),6 + jo .Llt16_2 + aghi %r5,16 + clgrjhe %r5,%r4,.Llastcmp + vfenezbs %v18,%v16,%v17 + jno .Lfound + + vlbb %v16,0(%r5,%r2),6 + vlbb %v17,0(%r5,%r3),6 + lcbb %r0,0(%r5,%r2),6 + jo .Llt16_1 + lcbb %r1,0(%r5,%r3),6 + jo .Llt16_2 + aghi %r5,16 + clgrjhe %r5,%r4,.Llastcmp + vfenezbs %v18,%v16,%v17 + jno .Lfound + + vlbb %v16,0(%r5,%r2),6 + vlbb %v17,0(%r5,%r3),6 + lcbb %r0,0(%r5,%r2),6 + jo .Llt16_1 + lcbb %r1,0(%r5,%r3),6 + jo .Llt16_2 + aghi %r5,16 + clgrjhe %r5,%r4,.Llastcmp + vfenezbs %v18,%v16,%v17 + jno .Lfound + j .Lloop + +.Llt16_1: + lcbb %r1,0(%r5,%r3),6 /* Get loaded byte count ofs2. */ +.Llt16_2: + clr %r0,%r1 /* Compare logical. */ + locrh %r0,%r1 /* Compute minimum of bytes loaded. */ + algfr %r5,%r0 /* Add smallest loaded bytes to current_len. */ + clgrj %r5,%r4,10,.Llastcmp /* If current_len >= n ->last compare. */ + vfenezbs %v18,%v16,%v17 /* Compare not equal with zero search. */ + vlgvb %r1,%v18,7 /* Get not equal index or 16 if all equal. */ + clrjl %r1,%r0,.Lfound /* Jump away if miscompare is within + loaded bytes (index < loaded-bytes) */ + j .Lloop + +.Llastcmp: + /* Use comparision result only if located within first n characters. + %r0: loaded byte count in vreg; + %r5: current_len; + %r4: n; + (current_len - n): [0...16[ + First ignored match index: loaded bytes - (current_len-n): ]0...16] + */ + slgr %r5,%r4 /* %r5 = current_len - n. */ + slr %r0,%r5 /* %r0 = first ignored match index. */ + vfenezbs %v18,%v16,%v17 /* Compare not equal with zero search. */ + vlgvb %r1,%v18,7 /* Get not equal index or 16 if all equal. */ + clrjl %r1,%r0,.Lfound /* Jump away if miscompare is within + loaded bytes and below n bytes. */ + j .Lend_equal /* Miscompare after n-bytes -> end equal. */ + +.Lfound: + /* Difference or end of string. */ + je .Lend_equal + lghi %r2,1 + lghi %r1,-1 + locgrl %r2,%r1 + br %r14 +.Lend_equal: + lghi %r2,0 + br %r14 +END(__strncmp_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strncmp.c b/sysdeps/s390/multiarch/strncmp.c new file mode 100644 index 0000000000..9a72c79bfd --- /dev/null +++ b/sysdeps/s390/multiarch/strncmp.c @@ -0,0 +1,30 @@ +/* Multiple versions of strncmp. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + + +# undef strcmp +extern __typeof (strncmp) __strncmp; +s390_vx_libc_ifunc2 (__strncmp, strncmp) + +#else +# include <string/strncmp.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/strncpy-vx.S b/sysdeps/s390/multiarch/strncpy-vx.S new file mode 100644 index 0000000000..08a0b29e8b --- /dev/null +++ b/sysdeps/s390/multiarch/strncpy-vx.S @@ -0,0 +1,207 @@ +/* Vector optimized 32/64 bit S/390 version of strncpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* char * strncpy (const char *dest, const char *src, size_t n) + Copy at most n characters of string src to dest. + + Register usage: + -r0=dest pointer for return + -r1=tmp, zero byte index + -r2=dest + -r3=src + -r4=n + -r5=current_len + -r6=tmp, loaded bytes + -r7=tmp, border + -v16=part of src + -v17=index of zero + -v18=part of src + -v31=register save area for r6, r7 +*/ +ENTRY(__strncpy_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + +# if !defined __s390x__ + llgfr %r4,%r4 +# endif /* !defined __s390x__ */ + + clgfi %r4,0 + ber %r14 /* Nothing to do, if n == 0. */ + lgr %r0,%r2 /* Save destination pointer for return. */ + vlvgp %v31,%r6,%r7 /* Save registers. */ + + vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */ + lcbb %r6,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */ + llgfr %r6,%r6 /* Convert 32bit to 64bit. */ + + lghi %r5,0 /* current_len = 0. */ + + clgrjle %r4,%r6,.Lremaining_v16 /* If n <= loaded-bytes + -> process remaining. */ + + /* n > loaded-byte-count. */ + vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */ + clrjl %r1,%r6,.Lfound_v16_store /* Found zero within loaded bytes, + copy and return. */ + + /* Align s to 16 byte. */ + risbgn %r7,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,15 /* current_len = 15. */ + slr %r5,%r7 /* Compute highest index to 16byte boundary. */ + + /* Zero not found and n > loaded-byte-count. */ + vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */ + ahi %r5,1 /* Start loop at next character. */ + + /* Now we are 16byte aligned, so we can load + a full vreg without page fault. */ + lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */ + aghi %r1,64 + clgrjl %r1,%r4,.Lloop64 + + vl %v16,0(%r5,%r3) /* Load s. */ + clgijl %r4,17,.Lremaining_v16 /* If n <= 16, process remaining + bytes. */ +.Llt64: + lgr %r7,%r4 + slgfi %r7,16 /* border_len = n - 16. */ + + clgrjhe %r5,%r7,.Lremaining_v16 /* If current_len >= border + then process remaining bytes. */ + vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound_v16 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3) /* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */ + aghi %r5,16 + + clgrjhe %r5,%r7,.Lremaining_v18 + vfenezbs %v17,%v18,%v18 + je .Lfound_v18 + vl %v16,16(%r5,%r3) + vst %v18,0(%r5,%r2) + aghi %r5,16 + + clgrjhe %r5,%r7,.Lremaining_v16 + vfenezbs %v17,%v16,%v16 + je .Lfound_v16 + vl %v18,16(%r5,%r3) + vst %v16,0(%r5,%r2) + aghi %r5,16 + +.Lremaining_v18: + vlr %v16,%v18 +.Lremaining_v16: + /* v16 contains the remaining bytes [1...16]. + Store remaining bytes and append string-termination. */ + vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */ + slgrk %r7,%r4,%r5 /* Remaining bytes = maxlen - current_len. */ + aghi %r7,-1 /* vstl needs highest index. */ + la %r2,0(%r5,%r2) /* vstl has no index register. */ + vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */ + /* Zero in remaining bytes? -> jump away (zero-index < max-index) + Do not jump away if zero-index == max-index, + but simply copy zero with vstl below. */ + clrjl %r1,%r7,.Lfound_v16_store + vstl %v16,%r7,0(%r2) /* Store remaining bytes without null + termination!. */ +.Lend: + /* Restore saved registers. */ + vlgvg %r6,%v31,0 + vlgvg %r7,%v31,1 + lgr %r2,%r0 /* Load saved dest-ptr. */ + br %r14 + + +.Lfound_v16_32: + aghi %r5,32 + j .Lfound_v16 +.Lfound_v18_48: + aghi %r5,32 +.Lfound_v18_16: + aghi %r5,16 +.Lfound_v18: + vlr %v16,%v18 +.Lfound_v16: + /* v16 contains a zero. Store remaining bytes to zero. current_len + has not reached border, thus checking for n is not needed! */ + vlgvb %r1,%v17,7 /* Load byte index of zero. */ + la %r2,0(%r5,%r2) /* vstl has no support for index-register. */ +.Lfound_v16_store: + vstl %v16,%r1,0(%r2) /* Copy characters including zero. */ + /* Fill remaining bytes with zero - remaining count always > 0. */ + algr %r5,%r1 /* Remaining bytes (=%r4) = ... */ + slgr %r4,%r5 /* = n - (current_len + zero_index + 1). */ + la %r2,0(%r1,%r2) /* Pointer to zero. start filling beyond. */ + aghi %r4,-2 /* mvc with exrl needs count - 1. + (additional -1, see remaining bytes above) */ + srlg %r6,%r4,8 /* Split into 256 byte blocks. */ + ltgr %r6,%r6 + je .Lzero_lt256 +.Lzero_loop256: + mvc 1(256,%r2),0(%r2) /* Fill 256 zeros at once. */ + la %r2,256(%r2) + brctg %r6,.Lzero_loop256 /* Loop until all blocks are processed. */ +.Lzero_lt256: + exrl %r4,.Lmvc_lt256 + j .Lend +.Lmvc_lt256: + mvc 1(1,%r2),0(%r2) + +.Lloop64: + vl %v16,0(%r5,%r3) /* Load s. */ + vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound_v16 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3) /* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */ + vfenezbs %v17,%v18,%v18 + je .Lfound_v18_16 + vl %v16,32(%r5,%r3) + vst %v18,16(%r5,%r2) + vfenezbs %v17,%v16,%v16 + je .Lfound_v16_32 + vl %v18,48(%r5,%r3) + vst %v16,32(%r5,%r2) + vfenezbs %v17,%v18,%v18 + je .Lfound_v18_48 + vst %v18,48(%r5,%r2) + + aghi %r5,64 + lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */ + aghi %r1,64 + clgrjl %r1,%r4,.Lloop64 + + vl %v16,0(%r5,%r3) /* Load s. */ + j .Llt64 +END(__strncpy_vx) + +# define strncpy __strncpy_c +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) strong_alias(__strncpy_c, __GI_strncpy) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ + +/* Include strncpy-implementation in s390-32/s390-64 subdirectory. */ +#include <strncpy.S> diff --git a/sysdeps/s390/multiarch/strncpy.c b/sysdeps/s390/multiarch/strncpy.c new file mode 100644 index 0000000000..1464551875 --- /dev/null +++ b/sysdeps/s390/multiarch/strncpy.c @@ -0,0 +1,24 @@ +/* Multiple versions of strncpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__strncpy, strncpy) +#endif diff --git a/sysdeps/s390/multiarch/strnlen-c.c b/sysdeps/s390/multiarch/strnlen-c.c new file mode 100644 index 0000000000..99ad65a103 --- /dev/null +++ b/sysdeps/s390/multiarch/strnlen-c.c @@ -0,0 +1,30 @@ +/* Default strnlen implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define STRNLEN __strnlen_c +# ifdef SHARED +# undef libc_hidden_def +# define libc_hidden_def(name) \ + __hidden_ver1 (__strnlen_c, __GI_strnlen, __strnlen_c); \ + strong_alias (__strnlen_c, __strnlen_c_1); \ + __hidden_ver1 (__strnlen_c_1, __GI___strnlen, __strnlen_c_1); +# endif /* SHARED */ + +# include <string/strnlen.c> +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strnlen-vx.S b/sysdeps/s390/multiarch/strnlen-vx.S new file mode 100644 index 0000000000..3e3a31dd9c --- /dev/null +++ b/sysdeps/s390/multiarch/strnlen-vx.S @@ -0,0 +1,134 @@ +/* Vector optimized 32/64 bit S/390 version of strnlen. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* size_t strnlen (const char *s, size_t maxlen) + Returns the number of characters in s or at most maxlen. + + Register usage: + -r1=tmp + -r2=address of string + -r3=maxlen (number of characters to be read) + -r4=tmp + -r5=current_len and return_value + -v16=part of s +*/ +ENTRY(__strnlen_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + +# if !defined __s390x__ + llgfr %r3,%r3 +# endif /* !defined __s390x__ */ + + clgfi %r3,0 /* if maxlen == 0, return 0. */ + locgre %r2,%r3 + ber %r14 + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + llgfr %r1,%r1 /* Convert 32bit to 64bit. */ + + vfenezb %v16,%v16,%v16 /* Find element not equal with zero search. */ + clgr %r1,%r3 + locgrh %r1,%r3 /* loaded_byte_count + = min (loaded_byte_count, maxlen) */ + + vlgvb %r5,%v16,7 /* Load zero index or 16 if not found. */ + clr %r5,%r1 /* If found zero within loaded bytes? */ + locgrl %r2,%r5 /* Then copy return value. */ + blr %r14 /* And return. */ + + clgr %r1,%r3 /* If loaded_byte_count == maxlen? */ + locgre %r2,%r3 /* Then copy return value. */ + ber %r14 /* And return. */ + + /* Align s to 16 byte. */ + risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,16 /* current_len = 16. */ + slr %r5,%r4 /* Compute bytes to 16bytes boundary. */ + + lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */ + aghi %r1,64 + clgrjl %r1,%r3,.Lloop64 + + /* Find zero in max 64byte with aligned s. */ +.Llt64: + vl %v16,0(%r5,%r2) /* Load s. */ + vfenezbs %v16,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound /* Jump away if zero was found. */ + aghi %r5,16 + clgrjhe %r5,%r3,.Lfound /* current_len >= maxlen -> end. */ + vl %v16,0(%r5,%r2) + vfenezbs %v16,%v16,%v16 + je .Lfound + aghi %r5,16 + clgrjhe %r5,%r3,.Lfound + vl %v16,0(%r5,%r2) + vfenezbs %v16,%v16,%v16 + je .Lfound + aghi %r5,16 + clgrjhe %r5,%r3,.Lfound + vl %v16,0(%r5,%r2) + vfenezbs %v16,%v16,%v16 + j .Lfound + +.Lfound48: + aghi %r5,16 +.Lfound32: + aghi %r5,16 +.Lfound16: + aghi %r5,16 +.Lfound: + vlgvb %r4,%v16,7 /* Load byte index of zero or 16 if no zero. */ + algr %r5,%r4 + + clgr %r5,%r3 + locgrh %r5,%r3 /* Return min (current_len, maxlen). */ + lgr %r2,%r5 + br %r14 + + /* Find zero in 16 byte aligned loop. */ +.Lloop64: + vl %v16,0(%r5,%r2) /* Load s. */ + vfenezbs %v16,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound /* Jump away if zero was found. */ + vl %v16,16(%r5,%r2) + vfenezbs %v16,%v16,%v16 + je .Lfound16 + vl %v16,32(%r5,%r2) + vfenezbs %v16,%v16,%v16 + je .Lfound32 + vl %v16,48(%r5,%r2) + vfenezbs %v16,%v16,%v16 + je .Lfound48 + + aghi %r5,64 + lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */ + aghi %r1,64 + clgrjl %r1,%r3,.Lloop64 + + j .Llt64 +END(__strnlen_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strnlen.c b/sysdeps/s390/multiarch/strnlen.c new file mode 100644 index 0000000000..48c3bb73e6 --- /dev/null +++ b/sysdeps/s390/multiarch/strnlen.c @@ -0,0 +1,29 @@ +/* Multiple versions of strnlen. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc (__strnlen) +weak_alias (__strnlen, strnlen) +libc_hidden_def (strnlen) + +#else +# include <string/strnlen.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/strpbrk-c.c b/sysdeps/s390/multiarch/strpbrk-c.c new file mode 100644 index 0000000000..49c5e1258b --- /dev/null +++ b/sysdeps/s390/multiarch/strpbrk-c.c @@ -0,0 +1,28 @@ +/* Default strpbrk implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define STRPBRK __strpbrk_c +# ifdef SHARED +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) \ + __hidden_ver1 (__strpbrk_c, __GI_strpbrk, __strpbrk_c); +# endif /* SHARED */ + +# include <string/strpbrk.c> +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strpbrk-vx.S b/sysdeps/s390/multiarch/strpbrk-vx.S new file mode 100644 index 0000000000..6a0bbd9d19 --- /dev/null +++ b/sysdeps/s390/multiarch/strpbrk-vx.S @@ -0,0 +1,302 @@ +/* Vector optimized 32/64 bit S/390 version of strpbrk. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* char *strpbrk (const char *s, const char * accept) + The strpbrk() function locates the first occurrence in the string s + of any of the characters in the string accept and returns a pointer + to that character or NULL if not found. + + This method checks the length of accept string. If it fits entirely + in one vector register, a fast algorithm is used, which does not need + to check multiple parts of accept-string. Otherwise a slower full + check of accept-string is used. + + register overview: + r3: pointer to start of accept-string + r2: pointer to start of search-string + r0: loaded byte count of vlbb search-string (32bit unsigned) + r4: found byte index (32bit unsigned) + r1: current return len (64bit unsigned) + v16: search-string + v17: accept-string + v18: temp-vreg + + ONLY FOR SLOW: + v19: first accept-string + v20: zero for preparing acc-vector + v21: global mask; 1 indicates a match between + search-string-vreg and any accept-character + v22: current mask; 1 indicates a match between + search-string-vreg and any accept-character in current acc-vreg + v24: one for result-checking of former string-part + v30, v31: for re-/storing registers r6, r8, r9 + r5: current len of accept-string + r6: zero-index in search-string or 16 if no zero + or min(zero-index, loaded byte count) + r8: >0, if former accept-string-part contains a zero, + otherwise =0; + r9: loaded byte count of vlbb accept-string +*/ +ENTRY(__strpbrk_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + /* + Check if accept-string fits in one vreg: + ---------------------------------------- + */ + vlbb %v17,0(%r3),6 /* Load accept. */ + lghi %r1,0 /* Zero out current len. */ + vlgvb %r0,%v17,0 /* Get first element. */ + clije %r0,0,.Lfast_end_null /* Return null if accept is empty. */ + lcbb %r0,0(%r3),6 + jo .Lcheck_onbb /* Special case if accept lays + on block-boundary. */ +.Lcheck_notonbb: + vistrbs %v17,%v17 /* Fill with zeros after first zero. */ + je .Lfast /* Zero found -> accept fits in one vreg. */ + j .Lslow /* No zero -> accept exceeds one vreg */ + + +.Lcheck_onbb: + /* Accept lays on block-boundary. */ + vfenezb %v18,%v17,%v17 /* Search zero in loaded accept bytes. */ + vlgvb %r4,%v18,7 /* Get index of zero or 16 if not found. */ + clrjl %r4,%r0,.Lcheck_notonbb /* Zero index < loaded bytes count -> + Accept fits in one vreg; + Fill with zeros and proceed + with FAST. */ + vl %v17,0(%r3) /* Load accept, which exceeds loaded bytes. */ + j .Lcheck_notonbb /* Check if accept fits in one vreg. */ + + + /* + Search s for accept in one vreg + ------------------------------- + */ +.Lfast: + /* Complete accept-string in v17 and remaining bytes are zero. */ + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r0,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + vfaezbs %v18,%v16,%v17,0 /* Find first element in v16 unequal to any + in v17 or first zero element. */ + + vlgvb %r4,%v18,7 /* Load byte index of found element. */ + /* If found index is within loaded bytes, return with found + element index (=equal count). */ + clrjl %r4,%r0,.Lfast_loop_found2 + + /* Align s to 16 byte. */ + risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r1,16 /* current_len = 16. */ + slr %r1,%r4 /* Compute bytes to 16bytes boundary. */ + + /* Process s in 16byte aligned loop. */ +.Lfast_loop: + vl %v16,0(%r1,%r2) /* Load search-string. */ + vfaezbs %v18,%v16,%v17,0 /* Find first element in v16 equal to any + in v17 or first zero element. */ + jno .Lfast_loop_found + + vl %v16,16(%r1,%r2) + vfaezbs %v18,%v16,%v17,0 + jno .Lfast_loop_found16 + + vl %v16,32(%r1,%r2) + vfaezbs %v18,%v16,%v17,0 + jno .Lfast_loop_found32 + + vl %v16,48(%r1,%r2) + vfaezbs %v18,%v16,%v17,0 + jno .Lfast_loop_found48 + + aghi %r1,64 + j .Lfast_loop /* Loop if no element was unequal to accept + and not zero. */ + + /* Found equal or zero element. */ +.Lfast_loop_found48: + aghi %r1,16 +.Lfast_loop_found32: + aghi %r1,16 +.Lfast_loop_found16: + aghi %r1,16 +.Lfast_loop_found: + vlgvb %r4,%v18,7 /* Load byte index of found element. */ +.Lfast_loop_found2: + vlgvb %r0,%v16,0(%r4) /* Get found element. */ + clije %r0,0,.Lfast_end_null /* Return null if no accept-char found */ + algfr %r1,%r4 /* Add found index of char to current len. */ + la %r2,0(%r1,%r2) /* And return pointer to first equal char. */ + br %r14 + +.Lfast_end_null: + lghi %r2,0 /* Return null if no character is equal. */ + br %r14 + + + + + /* + Search s for accept in multiple vregs + ------------------------------------- + */ +.Lslow: + /* Save registers. */ + vlvgg %v30,%r6,0 + vlvgp %v31,%r8,%r9 + + /* accept in v17 without zero. */ + vlr %v19,%v17 /* Save first acc-part for a fast reload. */ + vzero %v20 /* Zero for preparing acc-vector. */ + vone %v24 /* One for checking result of former string. */ + + /* Align s to 16 byte. */ + risbg %r4,%r2,60,128+63,0 /* Test if s is aligned and + %r4 = bits 60-63 'and' 15. */ + je .Lslow_loop_str /* If s is aligned, loop aligned. */ + lghi %r0,15 + slr %r0,%r4 /* Compute highest index to load (15-x). */ + vll %v16,%r0,0(%r2) /* Load up to 16 byte boundary (vll needs + highest index, remaining bytes are 0). */ + ahi %r0,1 /* Work with loaded byte count. */ + vzero %v21 /* Zero out global mask. */ + lghi %r5,0 /* Set current len of accept-string to zero. */ + vfenezb %v18,%v16,%v16 /* Find zero in current string-part. */ + lghi %r8,0 /* There is no zero in first accept-part. */ + vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */ + clije %r6,0,.Lslow_end_null /* If first element is zero + (end of string) -> return null */ + clr %r0,%r6 /* cc==1 if loaded byte count < zero-index. */ + locrl %r6,%r0 /* Load on cc==1; zero-index = lbc. */ + j .Lslow_loop_acc + + + /* Process s in 16byte aligned loop. */ +.Lslow_next_str: + /* Check results of former processed str-part. */ + vfeeb %v18,%v21,%v24 /* Find first equal match in global mask + (ones in element). */ + vlgvb %r4,%v18,7 /* Get index of first one (=equal) + or 16 if no match. */ + /* Equal-index < min(zero-index, loaded byte count) + -> return pointer to equal element. */ + clrjl %r4,%r6,.Lslow_index_found + /* Zero-index < loaded byte count + -> former str-part was last str-part + -> return null */ + clrjl %r6,%r0,.Lslow_end_null + /* All elements are zero (=no match) -> proceed with next str-part. */ + + vlr %v17,%v19 /* Load first part of accept (no zero). */ + algfr %r1,%r0 /* Add loaded byte count to current len. */ + +.Lslow_loop_str: + vl %v16,0(%r1,%r2) /* Load search-string */ + lghi %r0,16 /* Loaded byte count is 16. */ + vzero %v21 /* Zero out global mask. */ + lghi %r5,0 /* Set current len of accept to zero. */ + vfenezb %v18,%v16,%v16 /* Find zero in current string-part. */ + lghi %r8,0 /* There is no zero in first accept-part. */ + vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */ + clije %r6,0,.Lslow_end_null /* If first element is zero + (end of string) -> return null. */ + +.Lslow_loop_acc: + vfaeb %v22,%v16,%v17,4 /* Create matching-mask (1 in mask -> + Character matches any accepted character in + this accept-string-part) IN=0, RT=1. */ + vlgvb %r4,%v22,0 /* Get result of first element. */ + /* First element is equal to any accepted characters + (all other parts of accept cannot lead to a match before this one) + -> current len is pointing to first element + -> return found */ + clijh %r4,0,.Lslow_end_found + vo %v21,%v21,%v22 /* Global-mask = global-|matching-mask. */ + /* Proceed with next acc until end of acc is reached. */ + + +.Lslow_next_acc: + clijh %r8,0,.Lslow_next_str /* There was a zero in the last acc-part + -> add index to current_len and + end. */ + vlbb %v17,16(%r5,%r3),6 /* Load next accept part. */ + aghi %r5,16 /* Increment current len of accept-string. */ + lcbb %r9,0(%r5,%r3),6 /* Get loaded byte count of accept-string. */ + jo .Lslow_next_acc_onbb /* Jump away ifaccept-string is + on block-boundary. */ +.Lslow_next_acc_notonbb: + vistrbs %v17,%v17 /* Fill with zeros after first zero. */ + jo .Lslow_loop_acc /* No zero found -> no preparation needed. */ + +.Lslow_next_acc_prepare_zero: + /* Zero in accept-part: fill zeros with first-accept-character. */ + vlgvb %r8,%v17,0 /* Load first element of acc-part. */ + clije %r8,0,.Lslow_next_str /* Proceed with next string-part, + if first char in this part of accept + is a zero. */ + /* r8>0 -> zero found in this acc-part. */ + vrepb %v18,%v17,0 /* Replicate first char accross all chars. */ + vceqb %v22,%v20,%v17 /* Create a mask (v22) of null chars + by comparing with 0 (v20). */ + vsel %v17,%v18,%v17,%v22 /* Replace null chars with first char. */ + j .Lslow_loop_acc /* Accept part is prepared -> process. */ + +.Lslow_next_acc_onbb: + vfenezb %v18,%v17,%v17 /* Find zero in loaded bytes of accept part. */ + vlgvb %r8,%v18,7 /* Load byte index of zero. */ + clrjl %r8,%r9,.Lslow_next_acc_notonbb /* Found a zero in loaded bytes + -> Prepare vreg. */ + vl %v17,0(%r5,%r3) /* Load over boundary ... */ + lghi %r8,0 /* r8=0 -> no zero in this part of acc, + check for zero is in jump-target. */ + j .Lslow_next_acc_notonbb /* ... and search for zero in + fully loaded vreg again. */ + +.Lslow_end_null: + lghi %r1,0 /* Return null if no character is equal. */ + j .Lslow_end + +.Lslow_loop_found: + vlgvb %r4,%v18,7 /* Load byte index of found element. */ + vlgvb %r0,%v16,0(%r4) /* Get found element. */ + clije %r0,0,.Lslow_end_null /* Return null if no acc-char found. */ + +.Lslow_index_found: + algfr %r1,%r4 /* Add found index of char to current len. */ +.Lslow_end_found: + la %r1,0(%r1,%r2) /* And return pointer to first equal char. */ + +.Lslow_end: + /* Restore registers. */ + vlgvg %r6,%v30,0 + vlgvg %r8,%v31,0 + vlgvg %r9,%v31,1 + lgr %r2,%r1 + br %r14 +END(__strpbrk_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strpbrk.c b/sysdeps/s390/multiarch/strpbrk.c new file mode 100644 index 0000000000..cdc139929f --- /dev/null +++ b/sysdeps/s390/multiarch/strpbrk.c @@ -0,0 +1,27 @@ +/* Multiple versions of strpbrk. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__strpbrk, strpbrk) + +#else +# include <string/strpbrk.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/strrchr-c.c b/sysdeps/s390/multiarch/strrchr-c.c new file mode 100644 index 0000000000..2513af956d --- /dev/null +++ b/sysdeps/s390/multiarch/strrchr-c.c @@ -0,0 +1,29 @@ +/* Default strrchr implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define STRRCHR __strrchr_c +# undef weak_alias +# ifdef SHARED +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) \ + __hidden_ver1 (__strrchr_c, __GI_strrchr, __strrchr_c); +# endif /* SHARED */ + +# include <string/strrchr.c> +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strrchr-vx.S b/sysdeps/s390/multiarch/strrchr-vx.S new file mode 100644 index 0000000000..175d2cba3c --- /dev/null +++ b/sysdeps/s390/multiarch/strrchr-vx.S @@ -0,0 +1,180 @@ +/* Vector optimized 32/64 bit S/390 version of strrchr. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* char *strrchr (const char *s, int c) + Locate the last character c in string. + + Register usage: + -r0=loaded bytes in first part of s. + -r1=pointer to last occurence of c or NULL if not found. + -r2=s + -r3=c + -r4=tmp + -r5=current_len + -v16=part of s + -v17=index of found element + -v18=replicated c + -v19=part of s with last occurence of c. + -v20=permute pattern +*/ +ENTRY(__strrchr_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r0,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + vlvgb %v18,%r3,0 /* Generate vector which elements are all c. + if c > 255, c will be truncated. */ + vrepb %v18,%v18,0 + + lghi %r1,-1 /* Currently no c found. */ + lghi %r5,0 /* current_len = 0. */ + + vfeezbs %v17,%v16,%v18 /* Find element equal or zero. */ + vlgvb %r4,%v17,7 /* Load byte index of c/zero or 16. */ + clrjl %r4,%r0,.Lfound_first_part /* Found c/zero in loaded bytes. */ +.Lalign: + /* Align s to 16 byte. */ + risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,16 /* current_len = 16. */ + slr %r5,%r4 /* Compute bytes to 16bytes boundary. */ + +.Lloop: + vl %v16,0(%r5,%r2) /* Load s. */ + vfeezbs %v17,%v16,%v18 /* Find element equal with zero search. */ + jno .Lfound /* Found c/zero (cc=0|1|2). */ + vl %v16,16(%r5,%r2) + vfeezbs %v17,%v16,%v18 + jno .Lfound16 + vl %v16,32(%r5,%r2) + vfeezbs %v17,%v16,%v18 + jno .Lfound32 + vl %v16,48(%r5,%r2) + vfeezbs %v17,%v16,%v18 + jno .Lfound48 + + aghi %r5,64 + j .Lloop /* No character and no zero -> loop. */ + +.Lfound48: + la %r5,16(%r5) /* Use la since aghi would clobber cc. */ +.Lfound32: + la %r5,16(%r5) +.Lfound16: + la %r5,16(%r5) +.Lfound: + je .Lzero /* Found zero, but no c before that zero. */ + /* Save this part of s to check for further matches after reaching + the end of the complete string. */ + vlr %v19,%v16 + lgr %r1,%r5 + + jh .Lzero /* Found a zero after the found c. */ + aghi %r5,16 /* Start search of next part of s. */ + j .Lloop + +.Lfound_first_part: + /* This code is only executed if the found c/zero is whithin loaded + bytes. If no c/zero was found (cc==3) the found index = 16, thus + this code is not called. + Resulting condition code of vector find element equal: + cc==0: no c, found zero + cc==1: c found, no zero + cc==2: c found, found zero after c + cc==3: no c, no zero (this case can be ignored). */ + je .Lzero /* Found zero, but no c before that zero. */ + + locgrne %r1,%r5 /* Mark c as found in first part of s. */ + vlr %v19,%v16 + + jl .Lalign /* No zero (e.g. if vr was fully loaded) + -> Align and loop afterwards. */ + + /* Found a zero in vr. If vr was not fully loaded due to block + boundary, the remaining bytes are filled with zero and we can't + rely on zero indication of condition code here! */ + + vfenezb %v17,%v16,%v16 /* Find zero. */ + vlgvb %r4,%v17,7 /* Load byte index of zero or 16. */ + clrjl %r4,%r0,.Lzero /* Zero within loaded bytes -> end. */ + j .Lalign /* Align and loop afterwards. */ + +.Lend_searched_zero: + vlgvb %r4,%v17,7 /* Load byte index of zero. */ + algr %r5,%r4 + la %r2,0(%r5,%r2) /* Return pointer to zero. */ + br %r14 + +.Lzero: + /* Reached end of string. Check if one c was found before. */ + clije %r3,0,.Lend_searched_zero /* Found zero and c is zero. */ + + cgfi %r1,-1 /* No c found -> return NULL. */ + locghie %r2,0 + ber %r14 + + larl %r3,.Lpermute_mask /* Load permute mask. */ + vl %v20,0(%r3) + + /* c was found and is part of v19. */ + vfenezb %v17,%v19,%v19 /* Find zero. */ + vlgvb %r4,%v17,7 /* Load byte index of zero or 16. */ + + clgfi %r5,0 /* Loaded byte count in v19 is 16, ... */ + lochine %r0,16 /* ... if v19 is not the first part of s. */ + ahi %r0,-1 /* Convert byte count to highest index. */ + + clr %r0,%r4 + locrl %r4,%r0 /* r4 = min (zero-index, highest-index). */ + + /* Right-shift of v19 to mask bytes after zero. */ + clije %r4,15,.Lzero_permute /* No shift is needed if highest index + in vr is 15. */ + lhi %r0,15 + slr %r0,%r4 /* Compute byte count for vector shift right. */ + sll %r0,3 /* Convert to bit count. */ + vlvgb %v17,%r0,7 + vsrlb %v19,%v19,%v17 /* Vector shift right by byte by number of bytes + specified in bits 1-4 of byte 7 in v17. */ + + /* Reverse bytes in v19. */ +.Lzero_permute: + vperm %v19,%v19,%v19,%v20 /* Permute v19 to reversed order. */ + + /* Find c in reversed v19. */ + vfeeb %v19,%v19,%v18 /* Find c. */ + la %r2,0(%r1,%r2) + vlgvb %r3,%v19,7 /* Load byte index of c. */ + + /* Compute index in real s and return. */ + slgr %r4,%r3 + la %r2,0(%r4,%r2) /* Return pointer to zero. */ + br %r14 +.Lpermute_mask: + .byte 0x0F,0x0E,0x0D,0x0C,0x0B,0x0A,0x09,0x08 + .byte 0x07,0x06,0x05,0x04,0x03,0x02,0x01,0x00 +END(__strrchr_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strrchr.c b/sysdeps/s390/multiarch/strrchr.c new file mode 100644 index 0000000000..e515d6b6e6 --- /dev/null +++ b/sysdeps/s390/multiarch/strrchr.c @@ -0,0 +1,28 @@ +/* Multiple versions of strrchr. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__strrchr, strrchr) +weak_alias (strrchr, rindex) + +#else +# include <string/strrchr.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/strspn-c.c b/sysdeps/s390/multiarch/strspn-c.c new file mode 100644 index 0000000000..8928d3cc24 --- /dev/null +++ b/sysdeps/s390/multiarch/strspn-c.c @@ -0,0 +1,28 @@ +/* Default strspn implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define STRSPN __strspn_c +# ifdef SHARED +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) \ + __hidden_ver1 (__strspn_c, __GI_strspn, __strspn_c); +# endif /* SHARED */ + +# include <string/strspn.c> +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strspn-vx.S b/sysdeps/s390/multiarch/strspn-vx.S new file mode 100644 index 0000000000..65d295937a --- /dev/null +++ b/sysdeps/s390/multiarch/strspn-vx.S @@ -0,0 +1,256 @@ +/* Vector optimized 32/64 bit S/390 version of strspn. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* size_t strspn (const char *s, const char * accept) + The strspn() function calculates the length of the initial segment + of s which consists entirely of characters in accept. + + This method checks the length of accept string. If it fits entirely + in one vector register, a fast algorithm is used, which does not need + to check multiple parts of accept-string. Otherwise a slower full + check of accept-string is used. + + register overview: + r3: pointer to start of accept-string + r2: pointer to start of search-string + r4: loaded byte count of vl search-string + r0: found byte index + r1: current return len of s + v16: search-string + v17: accept-string + v18: temp-vreg + + ONLY FOR SLOW: + v19: first accept-string + v20: zero for preparing acc-vector + v21: global mask; 1 indicates a match between + search-string-vreg and any accept-character + v22: current mask; 1 indicates a match between + search-string-vreg and any accept-character in current acc-vreg + v30, v31: for re-/storing registers r6, r8, r9 + r5: current len of accept-string + r6: zero-index in search-string or 16 if no zero + or min(zero-index, loaded byte count) + r8: >0, if former accept-string-part contains a zero, + otherwise =0; + r9: loaded byte count of vlbb accept-string +*/ +ENTRY(__strspn_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + /* + Check if accept-string fits in one vreg: + ---------------------------------------- + */ + vlbb %v17,0(%r3),6 /* Load accept. */ + lcbb %r4,0(%r3),6 + jo .Lcheck_onbb /* Special case if accept lays + on block-boundary. */ +.Lcheck_notonbb: + vistrbs %v17,%v17 /* Fill with zeros after first zero. */ + je .Lfast /* Zero found -> accept fits in one vreg. */ + j .Lslow /* No zero -> accept exceeds one vreg. */ + +.Lcheck_onbb: + /* Accept lays on block-boundary. */ + vfenezb %v18,%v17,%v17 /* Search zero in loaded accept bytes. */ + vlgvb %r0,%v18,7 /* Get index of zero or 16 if not found. */ + clrjl %r0,%r4,.Lcheck_notonbb /* Zero index < loaded bytes count -> + Accept fits in one vreg; + Fill with zeros and proceed + with FAST. */ + vl %v17,0(%r3) /* Load accept, which exceeds loaded bytes. */ + j .Lcheck_notonbb /* Check if accept fits in one vreg. */ + + + /* + Search s for accept in one vreg + ------------------------------- + */ +.Lfast: + /* Complete accept-string is in v17 and remaining bytes are zero. */ + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + vfaezbs %v16,%v16,%v17,8 /* Find first element in v16 + unequal to any in v17 + or first zero element. */ + vlgvb %r0,%v16,7 /* Load byte index of found element. */ + /* If found index is within loaded bytes (%r0 < %r1), + return with found element index (=equal count). */ + clr %r0,%r1 + locgrl %r2,%r0 + blr %r14 + + /* Align s to 16 byte. */ + risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r1,16 /* current_len = 16. */ + slr %r1,%r4 /* Compute bytes to 16bytes boundary. */ + +.Lfast_loop: + vl %v16,0(%r1,%r2) /* Load search-string. */ + vfaezbs %v16,%v16,%v17,8 /* Find first element in v16 + unequal to any in v17 + or first zero element. */ + jno .Lfast_loop_found + vl %v16,16(%r1,%r2) + vfaezbs %v16,%v16,%v17,8 + jno .Lfast_loop_found16 + vl %v16,32(%r1,%r2) + vfaezbs %v16,%v16,%v17,8 + jno .Lfast_loop_found32 + vl %v16,48(%r1,%r2) + vfaezbs %v16,%v16,%v17,8 + jno .Lfast_loop_found48 + + aghi %r1,64 + j .Lfast_loop /* Loop if no element was unequal to accept + and not zero. */ + + /* Found unequal or zero element. */ +.Lfast_loop_found48: + aghi %r1,16 +.Lfast_loop_found32: + aghi %r1,16 +.Lfast_loop_found16: + aghi %r1,16 +.Lfast_loop_found: + vlgvb %r0,%v16,7 /* Load byte index of found element. */ + algrk %r2,%r1,%r0 /* And add it to current len. */ + br %r14 + + + /* + Search s for accept in multiple vregs + ------------------------------------- + */ +.Lslow: + /* Save registers. */ + vlvgg %v30,%r6,0 + vlvgp %v31,%r8,%r9 + lghi %r1,0 /* current_len = 0. */ + + /* Accept in v17 without zero. */ + vlr %v19,%v17 /* Save first acc-part for a fast reload. */ + vzero %v20 /* Zero for preparing acc-vector. */ + + /* Align s to 16 byte. */ + risbg %r0,%r2,60,128+63,0 /* Test if s is aligned and + %r0 = bits 60-63 'and' 15 */ + je .Lslow_loop_str /* If s is aligned, loop aligned */ + lghi %r4,15 + slr %r4,%r0 /* Compute highest index to load (15-x). */ + vll %v16,%r4,0(%r2) /* Load up to 16byte boundary (vll needs + highest index, left bytes are 0). */ + ahi %r4,1 /* Work with loaded byte count. */ + vzero %v21 /* Zero out global mask. */ + lghi %r5,0 /* Set current len of accept-string to zero. */ + vfenezb %v18,%v16,%v16 /* Find zero in current string-part. */ + lghi %r8,0 /* There is no zero in first accept-part. */ + vlgvb %r6,%v18,7 /* Load byte index of zero or 16 + if there is no zero. */ + clr %r4,%r6 /* cc==1 if loaded byte count < zero-index. */ + locrl %r6,%r4 /* Load on cc==1. */ + j .Lslow_loop_acc + + /* Process s in 16byte aligned loop. */ +.Lslow_next_str: + vlr %v17,%v19 /* Load first part of accept (no zero). */ + algfr %r1,%r4 /* Add loaded byte count to current len. */ +.Lslow_loop_str: + vl %v16,0(%r1,%r2) /* Load search-string. */ + lghi %r4,16 /* Loaded byte count is 16. */ + vzero %v21 /* Zero out global mask. */ + lghi %r5,0 /* Set current len of accept-string to zero. */ + vfenezb %v18,%v16,%v16 /* Find zero in current string-part. */ + lghi %r8,0 /* There is no zero in first accept-part. */ + vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */ + +.Lslow_loop_acc: + vfaeb %v22,%v16,%v17,4 /* Create matching-mask (1 in mask -> + character matches any accepted character in + this accept-string-part) IN=0, RT=1. */ + vo %v21,%v21,%v22 /* global-mask = global- | matching-mask. */ + vfenezb %v18,%v21,%v21 /* Find first zero in global-mask. */ + vlgvb %r0,%v18,7 /* Get first found zero-index + (= first mismatch). */ + clrjl %r0,%r6,.Lslow_next_acc /* Mismatch-index < min(lbc,zero-index) + -> Process this string-part + with next acc-part. */ + clrjhe %r0,%r4,.Lslow_next_str /* Found-index >= loaded byte count + -> All loaded bytes are matching + any accept-character + and are not zero. */ + /* All bytes are matching any characters in accept-string + and search-string is fully processed (found-index == zero-index) */ +.Lslow_add_lbc_end: + algrk %r2,%r1,%r0 /* Add matching characters to current_len. */ + /* Restore registers. */ + vlgvg %r6,%v30,0 + vlgvg %r8,%v31,0 + vlgvg %r9,%v31,1 + br %r14 + + + +.Lslow_next_acc: + clijh %r8,0,.Lslow_add_lbc_end /* There was a zero in last acc-part + -> Add found index to current len + and end. */ + vlbb %v17,16(%r5,%r3),6 /* Load next accept part. */ + aghi %r5,16 /* Add current_len of accept-string. */ + lcbb %r9,0(%r5,%r3),6 /* Get loaded byte count of accept-string. */ + jo .Lslow_next_acc_onbb /* Jump away if accept-string is + on block-boundary. */ +.Lslow_next_acc_notonbb: + vistrbs %v17,%v17 /* Fill with zeros after first zero. */ + jo .Lslow_loop_acc /* No zero found -> no preparation needed. */ + +.Lslow_next_acc_prepare_zero: + /* Zero in accept-part: fill zeros with first-accept-character. */ + vlgvb %r8,%v17,0 /* Load first element of acc-part. */ + clije %r8,0,.Lslow_add_lbc_end /* End if zero is first character + in this part of accept-string. */ + /* r8>0 -> zero found in this acc-part. */ + vrepb %v18,%v17,0 /* Replicate first char accross all chars. */ + vceqb %v22,%v20,%v17 /* Create a mask (v22) of null chars + by comparing with 0 (v20). */ + vsel %v17,%v18,%v17,%v22 /* Replace null chars with first char. */ + j .Lslow_loop_acc /* Accept part is prepared -> process. */ + +.Lslow_next_acc_onbb: + vfenezb %v18,%v17,%v17 /* Find zero in loaded bytes of accept part. */ + vlgvb %r8,%v18,7 /* Load byte index of zero. */ + clrjl %r8,%r9,.Lslow_next_acc_notonbb /* Found a zero in loaded bytes + -> Prepare vr. */ + vl %v17,0(%r5,%r3) /* Load over boundary ... */ + lghi %r8,0 /* r8=0 -> no zero in this part of acc, + Check for zero is in jump-target. */ + j .Lslow_next_acc_notonbb /* ... and search for zero in + fully loaded vreg again. */ +END(__strspn_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/strspn.c b/sysdeps/s390/multiarch/strspn.c new file mode 100644 index 0000000000..7c26af8ced --- /dev/null +++ b/sysdeps/s390/multiarch/strspn.c @@ -0,0 +1,27 @@ +/* Multiple versions of strspn. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <string.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__strspn, strspn) + +#else +# include <string/strspn.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wcpcpy-c.c b/sysdeps/s390/multiarch/wcpcpy-c.c new file mode 100644 index 0000000000..b4849a3321 --- /dev/null +++ b/sysdeps/s390/multiarch/wcpcpy-c.c @@ -0,0 +1,25 @@ +/* Default wcslen implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WCPCPY __wcpcpy_c + +# include <wchar.h> +extern __typeof (__wcpcpy) __wcpcpy_c; +# include <wcsmbs/wcpcpy.c> +#endif diff --git a/sysdeps/s390/multiarch/wcpcpy-vx.S b/sysdeps/s390/multiarch/wcpcpy-vx.S new file mode 100644 index 0000000000..8a466c6a37 --- /dev/null +++ b/sysdeps/s390/multiarch/wcpcpy-vx.S @@ -0,0 +1,114 @@ +/* Vector optimized 32/64 bit S/390 version of wcpcpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* wchar_t * wcpcpy (const wchar_t *dest, const wchar_t *src) + Copy string src to dest returning a pointer to its end. + + Register usage: + -r0=border-len for switching to vector-instructions + -r1=tmp + -r2=dest and return value + -r3=src + -r4=tmp + -r5=current_len + -v16=part of src + -v17=index of zero + -v18=part of src +*/ +ENTRY(__wcpcpy_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */ + + tmll %r3,3 /* Test if s is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + + vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r5,%v17,7 /* Load zero index or 16 if not found. */ + clrjl %r5,%r1,.Lfound_align /* If found zero within loaded bytes, + copy bytes before and return. */ + + /* Align s to 16 byte. */ + risbgn %r4,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,15 /* current_len = 15. */ + slr %r5,%r4 /* Compute highest index to 16byte boundary. */ + + vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */ + ahi %r5,1 /* Start loop at next character. */ + + /* Find zero in 16byte aligned loop. */ +.Lloop: + vl %v16,0(%r5,%r3) /* Load s. */ + vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound_v16_0 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3) /* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */ + vfenezfs %v17,%v18,%v18 + je .Lfound_v18_16 + vl %v16,32(%r5,%r3) + vst %v18,16(%r5,%r2) + vfenezfs %v17,%v16,%v16 + je .Lfound_v16_32 + vl %v18,48(%r5,%r3) + vst %v16,32(%r5,%r2) + vfenezfs %v17,%v18,%v18 + je .Lfound_v18_48 + vst %v18,48(%r5,%r2) + + aghi %r5,64 + j .Lloop /* No zero found -> loop. */ + +.Lfound_v16_32: + aghi %r5,32 +.Lfound_v16_0: + la %r3,0(%r5,%r2) + vlgvb %r1,%v17,7 /* Load byte index of zero. */ + aghi %r1,3 /* Also copy remaining bytes of zero. */ + vstl %v16,%r1,0(%r3) /* Copy characters including zero. */ + lay %r2,-3(%r1,%r3) /* Return pointer to zero. */ + br %r14 + +.Lfound_v18_48: + aghi %r5,32 +.Lfound_v18_16: + la %r3,16(%r5,%r2) + vlgvb %r1,%v17,7 /* Load byte index of zero. */ + aghi %r1,3 /* Also copy remaining bytes of zero. */ + vstl %v18,%r1,0(%r3) /* Copy characters including zero. */ + lay %r2,-3(%r1,%r3) /* Return pointer to zero. */ + br %r14 + +.Lfound_align: + aghi %r5,3 /* Also copy remaining bytes of zero. */ + vstl %v16,%r5,0(%r2) /* Copy characters including zero. */ + lay %r2,-3(%r5,%r2) /* Return pointer to zero. */ + br %r14 + +.Lfallback: + jg __wcpcpy_c +END(__wcpcpy_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcpcpy.c b/sysdeps/s390/multiarch/wcpcpy.c new file mode 100644 index 0000000000..8afd98d7d4 --- /dev/null +++ b/sysdeps/s390/multiarch/wcpcpy.c @@ -0,0 +1,28 @@ +/* Multiple versions of wcpcpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc (__wcpcpy) +weak_alias (__wcpcpy, wcpcpy) + +#else +# include <wcsmbs/wcpcpy.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wcpncpy-c.c b/sysdeps/s390/multiarch/wcpncpy-c.c new file mode 100644 index 0000000000..86db27b525 --- /dev/null +++ b/sysdeps/s390/multiarch/wcpncpy-c.c @@ -0,0 +1,25 @@ +/* Default wcsncpy implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WCPNCPY __wcpncpy_c + +# include <wchar.h> +extern __typeof (__wcpncpy) __wcpncpy_c; +# include <wcsmbs/wcpncpy.c> +#endif diff --git a/sysdeps/s390/multiarch/wcpncpy-vx.S b/sysdeps/s390/multiarch/wcpncpy-vx.S new file mode 100644 index 0000000000..ca0203f451 --- /dev/null +++ b/sysdeps/s390/multiarch/wcpncpy-vx.S @@ -0,0 +1,222 @@ +/* Vector optimized 32/64 bit S/390 version of wcpncpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* wchar_t * wcpncpy (wchar_t *dest, const wchar_t *src, size_t n) + Copies at most n characters of string src to dest + returning a pointer to its end or dest+n + if src is smaller than n. + + Register usage: + -%r0 = return value + -%r1 = zero byte index + -%r2 = curr dst pointer + -%r3 = curr src pointer + -%r4 = n + -%r5 = current_len + -%r6 = loaded bytes + -%r7 = border, tmp +*/ +ENTRY(__wcpncpy_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + +# if !defined __s390x__ + llgfr %r4,%r4 +# endif /* !defined __s390x__ */ + + clgfi %r4,0 + ber %r14 /* Nothing to do, if n == 0. */ + + vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */ + + tmll %r3,3 /* Test if s is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + + vlvgp %v31,%r6,%r7 /* Save registers. */ + lghi %r5,0 /* current_len = 0. */ + + lcbb %r6,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */ + llgfr %r6,%r6 /* Convert 32bit to 64bit. */ + + /* Check range of maxlen and convert to byte-count. */ +# ifdef __s390x__ + tmhh %r4,49152 /* Test bit 0 or 1 of maxlen. */ + lghi %r1,-4 /* Max byte-count is 18446744073709551612. */ +# else + tmlh %r4,49152 /* Test bit 0 or 1 of maxlen. */ + llilf %r1,4294967292 /* Max byte-count is 4294967292. */ +# endif /* !__s390x__ */ + sllg %r4,%r4,2 /* Convert character-count to byte-count. */ + locgrne %r4,%r1 /* Use max byte-count, if bit 0/1 was one. */ + + la %r0,0(%r4,%r2) /* Save destination pointer + n for return. */ + + clgrjle %r4,%r6,.Lremaining_v16 /* If n <= loaded-bytes + -> process remaining. */ + + /* n > loaded-byte-count */ + vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */ + aghi %r1,3 /* Also copy remaining bytes of zero. */ + clrjl %r1,%r6,.Lfound_v16_store /* Found zero within loaded bytes, + copy and return. */ + + /* Align s to 16 byte. */ + risbgn %r7,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,15 /* current_len = 15. */ + slr %r5,%r7 /* Compute highest index to 16byte boundary. */ + + /* Zero not found and n > loaded-byte-count. */ + vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */ + ahi %r5,1 /* Start loop at next character. */ + + /* Now we are 16byte aligned, so we can load a full vreg + without page fault. */ + lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */ + aghi %r1,64 + clgrjl %r1,%r4,.Lloop64 + + vl %v16,0(%r5,%r3) /* Load s. */ + clgijl %r4,17,.Lremaining_v16 /* If n <=16, + process remaining bytes. */ +.Llt64: + lgr %r7,%r4 + slgfi %r7,16 /* border_len = n - 16. */ + + clgrjhe %r5,%r7,.Lremaining_v16 /* If current_len >= border + then process remaining bytes. */ + vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound_v16 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3) /* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */ + aghi %r5,16 + + clgrjhe %r5,%r7,.Lremaining_v18 + vfenezfs %v17,%v18,%v18 + je .Lfound_v18 + vl %v16,16(%r5,%r3) + vst %v18,0(%r5,%r2) + aghi %r5,16 + + clgrjhe %r5,%r7,.Lremaining_v16 + vfenezfs %v17,%v16,%v16 + je .Lfound_v16 + vl %v18,16(%r5,%r3) + vst %v16,0(%r5,%r2) + aghi %r5,16 + +.Lremaining_v18: + vlr %v16,%v18 +.Lremaining_v16: + /* v16 contains the remaining bytes [1...16]. + Store remaining bytes and append string-termination. */ + vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */ + slgrk %r7,%r4,%r5 /* Remaining bytes = maxlen - current_len */ + aghi %r7,-1 /* vstl needs highest index. */ + la %r2,0(%r5,%r2) /* vstl has no index register. */ + vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */ + aghi %r1,3 /* Also copy remaining bytes of zero. */ + /* Zero in remaining bytes? -> jump away (zero-index <= max-index). */ + clrjle %r1,%r7,.Lfound_v16_store + vstl %v16,%r7,0(%r2) /* Store remaining bytes without null + termination! */ +.Lend: + /* Restore saved registers. */ + vlgvg %r6,%v31,0 + vlgvg %r7,%v31,1 + lgr %r2,%r0 /* Load saved dest-ptr. */ + br %r14 + +.Lfound_v16_32: + aghi %r5,32 + j .Lfound_v16 +.Lfound_v18_48: + aghi %r5,32 +.Lfound_v18_16: + aghi %r5,16 +.Lfound_v18: + vlr %v16,%v18 +.Lfound_v16: + /* v16 contains a zero. Store remaining bytes to zero. current_len + has not reached border, thus checking for n is not needed! */ + vlgvb %r1,%v17,7 /* Load byte index of zero. */ + la %r2,0(%r5,%r2) /* vstl has no support for index-register. */ + aghi %r1,3 /* Also copy remaining bytes of zero. */ +.Lfound_v16_store: + vstl %v16,%r1,0(%r2) /* Copy characters including zero. */ + /* Fill remaining bytes with zero - remaining byte count always > 0. */ + algr %r5,%r1 /* Remaining bytes (=%r4) = ... */ + slgr %r4,%r5 /* = n - (currlen + zero_index + 1) */ + la %r2,0(%r1,%r2) /* Pointer to zero. start filling beyond. */ + lay %r0,-3(%r2) /* Save return-pointer to found zero. */ + clgije %r4,1,.Lend /* Skip zero-filling, if found-zero is last + possible character. + (1 is substracted from r4 below!). */ + aghi %r4,-2 /* mvc with exrl needs count - 1. + (additional -1, see remaining bytes above) */ + srlg %r6,%r4,8 /* Split into 256 byte blocks. */ + ltgr %r6,%r6 + je .Lzero_lt256 +.Lzero_loop256: + mvc 1(256,%r2),0(%r2) /* Fill 256 zeros at once. */ + la %r2,256(%r2) + brctg %r6,.Lzero_loop256 /* Loop until all blocks are processed. */ +.Lzero_lt256: + exrl %r4,.Lmvc_lt256 + j .Lend +.Lmvc_lt256: + mvc 1(1,%r2),0(%r2) + + /* Find zero in 16byte aligned loop. */ +.Lloop64: + vl %v16,0(%r5,%r3) + vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound_v16 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3) /* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */ + vfenezfs %v17,%v18,%v18 + je .Lfound_v18_16 + vl %v16,32(%r5,%r3) + vst %v18,16(%r5,%r2) + vfenezfs %v17,%v16,%v16 + je .Lfound_v16_32 + vl %v18,48(%r5,%r3) + vst %v16,32(%r5,%r2) + vfenezfs %v17,%v18,%v18 + je .Lfound_v18_48 + vst %v18,48(%r5,%r2) + + aghi %r5,64 + lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */ + aghi %r1,64 + clgrjl %r1,%r4,.Lloop64 + + vl %v16,0(%r5,%r3) /* Load s. */ + j .Llt64 + +.Lfallback: + jg __wcpncpy_c +END(__wcpncpy_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcpncpy.c b/sysdeps/s390/multiarch/wcpncpy.c new file mode 100644 index 0000000000..13bc543a8a --- /dev/null +++ b/sysdeps/s390/multiarch/wcpncpy.c @@ -0,0 +1,28 @@ +/* Multiple versions of wcpncpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc (__wcpncpy) +weak_alias (__wcpncpy, wcpncpy) + +#else +# include <wcsmbs/wcpncpy.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wcscat-c.c b/sysdeps/s390/multiarch/wcscat-c.c new file mode 100644 index 0000000000..bceec55408 --- /dev/null +++ b/sysdeps/s390/multiarch/wcscat-c.c @@ -0,0 +1,25 @@ +/* Default wcscat implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WCSCAT __wcscat_c + +# include <wchar.h> +extern __typeof (__wcscat) __wcscat_c; +# include <wcsmbs/wcscat.c> +#endif diff --git a/sysdeps/s390/multiarch/wcscat-vx.S b/sysdeps/s390/multiarch/wcscat-vx.S new file mode 100644 index 0000000000..8353caafa9 --- /dev/null +++ b/sysdeps/s390/multiarch/wcscat-vx.S @@ -0,0 +1,175 @@ +/* Vector optimized 32/64 bit S/390 version of wcscat. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* wchar_t * wcscat (wchar_t *dest, const wchar_t *src) + Concatenate two strings. + + Register usage: + -r0=saved dest pointer for return + -r1=tmp + -r2=dest + -r3=src + -r4=tmp + -r5=current_len + -v16=part of src + -v17=index of zero + -v18=part of src +*/ +ENTRY(__wcscat_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + /* __wcslen_c can handle non 4byte aligned pointers, + but __wcscpy_c not. Thus if either src or dest is + not 4byte aligned, use __wcscat_c. */ + tmll %r2,3 /* Test if s is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + tmll %r3,3 /* Test if src is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + + lgr %r0,%r2 /* Save destination pointer for return. */ + + /* WCSLEN + r1 = loaded bytes (tmp) + r4 = zero byte index (tmp) + r2 = dst + */ + + vfenezf %v16,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r5,%v16,7 /* Load zero index or 16 if not found. */ + clrjl %r5,%r1,.Llen_end /* Found zero within loaded bytes, end. */ + + /* Align s to 16 byte. */ + risbgn %r1,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,16 /* current_len = 16. */ + slr %r5,%r1 /* Compute bytes to 16bytes boundary. */ + + /* Find zero in 16byte aligned loop. */ +.Llen_loop: + vl %v16,0(%r5,%r2) /* Load s. */ + vfenezfs %v16,%v16,%v16 /* Find element not equal with zero search. */ + je .Llen_found /* Jump away if zero was found. */ + vl %v16,16(%r5,%r2) + vfenezfs %v16,%v16,%v16 + je .Llen_found16 + vl %v16,32(%r5,%r2) + vfenezfs %v16,%v16,%v16 + je .Llen_found32 + vl %v16,48(%r5,%r2) + vfenezfs %v16,%v16,%v16 + je .Llen_found48 + + aghi %r5,64 + j .Llen_loop /* No zero -> loop. */ + +.Llen_found48: + aghi %r5,16 +.Llen_found32: + aghi %r5,16 +.Llen_found16: + aghi %r5,16 +.Llen_found: + vlgvb %r4,%v16,7 /* Load byte index of zero. */ + algr %r5,%r4 + +.Llen_end: + /* WCSCPY + %r1 = loaded bytes (tmp) + %r4 = zero byte index (tmp) + %r3 = curr src pointer + %r2 = curr dst pointer + */ + la %r2,0(%r5,%r2) /* strcpy at end of dst-string. */ + + vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */ + + vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r5,%v17,7 /* Load zero index or 16 if not found. */ + clrjl %r5,%r1,.Lcpy_found_align /* If found zero within loaded bytes, + copy bytes before and return. */ + + /* Align s to 16 byte. */ + risbgn %r4,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,15 /* current_len = 15. */ + slr %r5,%r4 /* Compute highest index to 16byte boundary. */ + + vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */ + ahi %r5,1 /* Start loop at next character. */ + + /* Find zero in 16byte aligned loop. */ +.Lcpy_loop: + vl %v16,0(%r5,%r3) /* Load s. */ + vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lcpy_found_v16_0 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3) /* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Save previous part without zero to dst. */ + vfenezfs %v17,%v18,%v18 + je .Lcpy_found_v18_16 + vl %v16,32(%r5,%r3) + vst %v18,16(%r5,%r2) + vfenezfs %v17,%v16,%v16 + je .Lcpy_found_v16_32 + vl %v18,48(%r5,%r3) + vst %v16,32(%r5,%r2) + vfenezfs %v17,%v18,%v18 + je .Lcpy_found_v18_48 + vst %v18,48(%r5,%r2) + + aghi %r5,64 + j .Lcpy_loop /* No zero -> loop. */ + +.Lcpy_found_v16_32: + aghi %r5,32 +.Lcpy_found_v16_0: + la %r4,0(%r5,%r2) + vlgvb %r1,%v17,7 /* Load byte index of zero. */ + aghi %r1,3 /* Also copy remaining bytes of zero. */ + vstl %v16,%r1,0(%r4) /* Copy characters including zero. */ + lgr %r2,%r0 /* Load saved dest-ptr. */ + br %r14 + +.Lcpy_found_v18_48: + aghi %r5,32 +.Lcpy_found_v18_16: + la %r4,16(%r5,%r2) + vlgvb %r1,%v17,7 /* Load byte index of zero. */ + aghi %r1,3 /* Also copy remaining bytes of zero. */ + vstl %v18,%r1,0(%r4) /* Copy characters including zero. */ + lgr %r2,%r0 /* Load saved dest-ptr. */ + br %r14 + +.Lcpy_found_align: + aghi %r5,3 /* Also copy remaining bytes of found zero. */ + vstl %v16,%r5,0(%r2) /* Copy characters including zero. */ + lgr %r2,%r0 /* Load saved dest-ptr. */ + br %r14 +.Lfallback: + jg __wcscat_c +END(__wcscat_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcscat.c b/sysdeps/s390/multiarch/wcscat.c new file mode 100644 index 0000000000..8d71c2f1b9 --- /dev/null +++ b/sysdeps/s390/multiarch/wcscat.c @@ -0,0 +1,28 @@ +/* Multiple versions of wcscat. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc (__wcscat) +weak_alias (__wcscat, wcscat) + +#else +# include <wcsmbs/wcscat.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wcschr-c.c b/sysdeps/s390/multiarch/wcschr-c.c new file mode 100644 index 0000000000..9ba1d5f861 --- /dev/null +++ b/sysdeps/s390/multiarch/wcschr-c.c @@ -0,0 +1,37 @@ +/* Default wcschr implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WCSCHR __wcschr_c + +# include <wchar.h> +extern __typeof (__wcschr) __wcschr_c; +# undef weak_alias +# define weak_alias(name, alias) +# ifdef SHARED +# undef libc_hidden_def +# define libc_hidden_def(name) \ + __hidden_ver1 (__wcschr_c, __GI_wcschr, __wcschr_c); \ + strong_alias (__wcschr_c, __wcschr_c_1); \ + __hidden_ver1 (__wcschr_c_1, __GI___wcschr, __wcschr_c_1); +# undef libc_hidden_weak +# define libc_hidden_weak(name) +# endif /* SHARED */ + +# include <wcsmbs/wcschr.c> +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcschr-vx.S b/sysdeps/s390/multiarch/wcschr-vx.S new file mode 100644 index 0000000000..ff7d1c4b4e --- /dev/null +++ b/sysdeps/s390/multiarch/wcschr-vx.S @@ -0,0 +1,103 @@ +/* Vector optimized 32/64 bit S/390 version of wcschr. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* wchar_t *wcschr (const wchar_t *s, wchar_t c) + Locate character in string. + + Register usage: + -r1=tmp + -r2=s + -r3=c + -r4=tmp + -r5=current_len + -v16=part of s + -v17=index of unequal + -v18=replicated c +*/ +ENTRY(__wcschr_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + tmll %r2,3 /* Test if s is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + + lghi %r5,0 /* current_len = 0. */ + + vlvgf %v18,%r3,0 /* Generate vector which elements are all c. */ + vrepf %v18,%v18,0 + + vfeezfs %v16,%v16,%v18 /* Find element equal with zero search. */ + vlgvb %r4,%v16,7 /* Load byte index of character or zero. */ + clrjl %r4,%r1,.Lfound /* Return if c/zero is in loaded bytes. */ + + /* Align s to 16 byte. */ + risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,16 /* current_len = 16. */ + slr %r5,%r4 /* Compute bytes to 16bytes boundary. */ + + /* Find c/zero in 16byte aligned loop */ +.Lloop: + vl %v16,0(%r5,%r2) /* Load s. */ + vfeezfs %v16,%v16,%v18 /* Find element equal with zero search. */ + jno .Lfound /* Found c/zero (cc=0|1|2). */ + vl %v16,16(%r5,%r2) + vfeezfs %v16,%v16,%v18 + jno .Lfound16 + vl %v16,32(%r5,%r2) + vfeezfs %v16,%v16,%v18 + jno .Lfound32 + vl %v16,48(%r5,%r2) + vfeezfs %v16,%v16,%v18 + jno .Lfound48 + + aghi %r5,64 + j .Lloop /* No character and no zero -> loop. */ + +.Lfound48: + la %r5,16(%r5) /* Use la since aghi would clobber cc. */ +.Lfound32: + la %r5,16(%r5) +.Lfound16: + la %r5,16(%r5) +.Lfound: + je .Lzero /* Found zero, but no c before that zero. */ + +.Lcharacter: + vlgvb %r4,%v16,7 /* Load byte index of character. */ + algr %r5,%r4 + la %r2,0(%r5,%r2) /* Return pointer to character. */ + br %r14 + +.Lzero: + clije %r3,0,.Lcharacter /* Found zero and c is zero. */ + lghi %r2,0 /* Return null if character not found. */ + br %r14 +.Lfallback: + jg __wcschr_c +END(__wcschr_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcschr.c b/sysdeps/s390/multiarch/wcschr.c new file mode 100644 index 0000000000..fb51097cd6 --- /dev/null +++ b/sysdeps/s390/multiarch/wcschr.c @@ -0,0 +1,29 @@ +/* Multiple versions of wcschr. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc (__wcschr) +weak_alias (__wcschr, wcschr) +libc_hidden_weak (wcschr) + +#else +# include <wcsmbs/wcschr.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wcschrnul-c.c b/sysdeps/s390/multiarch/wcschrnul-c.c new file mode 100644 index 0000000000..bbee3288fe --- /dev/null +++ b/sysdeps/s390/multiarch/wcschrnul-c.c @@ -0,0 +1,25 @@ +/* Default wcschrnul implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WCSCHRNUL __wcschrnul_c + +# include <wchar.h> +extern __typeof (__wcschrnul) __wcschrnul_c; +# include <wcsmbs/wcschrnul.c> +#endif diff --git a/sysdeps/s390/multiarch/wcschrnul-vx.S b/sysdeps/s390/multiarch/wcschrnul-vx.S new file mode 100644 index 0000000000..e54e48d894 --- /dev/null +++ b/sysdeps/s390/multiarch/wcschrnul-vx.S @@ -0,0 +1,97 @@ +/* Vector optimized 32/64 bit S/390 version of wcschrnul. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* wchar_t* wcschrnul (const wchar_t *s, wchar_t c) + Returns pointer to first c or to \0 if c not found. + + Register usage: + -r1=tmp + -r2=s and return pointer + -r3=c + -r4=tmp + -r5=current_len + -v16=part of s + -v18=vector with c replicated in every byte +*/ +ENTRY(__wcschrnul_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + tmll %r2,3 /* Test if s is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + + lghi %r5,0 /* current_len = 0. */ + + vlvgf %v18,%r3,0 /* Generate vector which elements are all c. */ + vrepf %v18,%v18,0 + + vfeezfs %v16,%v16,%v18 /* Find element equal with zero search. */ + vlgvb %r4,%v16,7 /* Load byte index of character or zero. */ + clrjl %r4,%r1,.Lfound /* Return if c/zero is in loaded bytes. */ + + /* Align s to 16 byte. */ + risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,16 /* current_len = 16. */ + slr %r5,%r4 /* Compute bytes to 16bytes boundary. */ + + /* Find c/zero in 16byte aligned loop */ +.Lloop: + vl %v16,0(%r5,%r2) /* Load s. */ + vfeezfs %v16,%v16,%v18 /* Find element equal with zero search. */ + jno .Lfound /* Found c/zero (cc=0|1|2). */ + vl %v16,16(%r5,%r2) + vfeezfs %v16,%v16,%v18 + jno .Lfound16 + vl %v16,32(%r5,%r2) + vfeezfs %v16,%v16,%v18 + jno .Lfound32 + vl %v16,48(%r5,%r2) + vfeezfs %v16,%v16,%v18 + jno .Lfound48 + + aghi %r5,64 + j .Lloop /* No character and no zero -> loop. */ + + /* Found character or zero */ +.Lfound48: + aghi %r5,16 +.Lfound32: + aghi %r5,16 +.Lfound16: + aghi %r5,16 +.Lfound: + vlgvb %r1,%v16,7 /* Load byte index of character. */ + algr %r5,%r1 + la %r2,0(%r5,%r2) /* Return pointer to character. */ + +.Lend: + br %r14 +.Lfallback: + jg __wcschrnul_c +END(__wcschrnul_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcschrnul.c b/sysdeps/s390/multiarch/wcschrnul.c new file mode 100644 index 0000000000..7436a596bd --- /dev/null +++ b/sysdeps/s390/multiarch/wcschrnul.c @@ -0,0 +1,28 @@ +/* Multiple versions of wcschrnul. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc (__wcschrnul) +weak_alias (__wcschrnul, wcschrnul) + +#else +# include <wcsmbs/wcschrnul.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wcscmp-c.c b/sysdeps/s390/multiarch/wcscmp-c.c new file mode 100644 index 0000000000..3add8e4095 --- /dev/null +++ b/sysdeps/s390/multiarch/wcscmp-c.c @@ -0,0 +1,32 @@ +/* Default wcscmp implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WCSCMP __wcscmp_c + +# include <wchar.h> +extern __typeof (wcscmp) __wcscmp_c; +# undef weak_alias +# define weak_alias(name, alias) +# ifdef SHARED +# undef libc_hidden_def +# define libc_hidden_def(name) \ + __hidden_ver1 (__wcscmp_c, __GI___wcscmp, __wcscmp_c); +# endif /* SHARED */ +# include <wcsmbs/wcscmp.c> +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcscmp-vx.S b/sysdeps/s390/multiarch/wcscmp-vx.S new file mode 100644 index 0000000000..549ae3c733 --- /dev/null +++ b/sysdeps/s390/multiarch/wcscmp-vx.S @@ -0,0 +1,131 @@ +/* Vector optimized 32/64 bit S/390 version of wcscmp. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* int wcscmp (const wchar_t *s1, const wchar_t *s2) + Compare two strings + + Register usage: + -r1=loaded byte count s1 + -r2=s1 + -r3=s2 + -r4=loaded byte coutn s2, tmp + -r5=current_len + -v16=part of s1 + -v17=part of s2 + -v18=index of unequal +*/ +ENTRY(__wcscmp_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + lghi %r5,0 /* current_len = 0. */ + +.Lloop: + vlbb %v16,0(%r5,%r2),6 /* Load s1 to block boundary. */ + vlbb %v17,0(%r5,%r3),6 /* Load s2 to block boundary. */ + lcbb %r1,0(%r5,%r2),6 /* Get loaded byte count of s1. */ + jo .Llt16_1 /* Jump away if vr is not fully loaded. */ + lcbb %r4,0(%r5,%r3),6 + jo .Llt16_2 /* Jump away if vr is not fully loaded. */ + /* Both vrs are fully loaded. */ + aghi %r5,16 + vfenezfs %v18,%v16,%v17 /* Compare not equal with zero search. */ + jno .Lfound + + vlbb %v16,0(%r5,%r2),6 + vlbb %v17,0(%r5,%r3),6 + lcbb %r1,0(%r5,%r2),6 + jo .Llt16_1 + lcbb %r4,0(%r5,%r3),6 + jo .Llt16_2 + aghi %r5,16 + vfenezfs %v18,%v16,%v17 + jno .Lfound + + vlbb %v16,0(%r5,%r2),6 + vlbb %v17,0(%r5,%r3),6 + lcbb %r1,0(%r5,%r2),6 + jo .Llt16_1 + lcbb %r4,0(%r5,%r3),6 + jo .Llt16_2 + aghi %r5,16 + vfenezfs %v18,%v16,%v17 + jno .Lfound + + vlbb %v16,0(%r5,%r2),6 + vlbb %v17,0(%r5,%r3),6 + lcbb %r1,0(%r5,%r2),6 + jo .Llt16_1 + lcbb %r4,0(%r5,%r3),6 + jo .Llt16_2 + aghi %r5,16 + vfenezfs %v18,%v16,%v17 + jno .Lfound + j .Lloop + +.Lcmp_one_char: + /* At least one of both strings is not 4-byte aligned + and there is no full character before next block-boundary. + Compare one character to get over the boundary and + proceed with normal loop! */ + vlef %v16,0(%r5,%r2),0 /* Load one character. */ + vlef %v17,0(%r5,%r3),0 + lghi %r1,4 /* Loaded byte count is 4. */ + j .Llt_cmp /* Proceed with comparision. */ + +.Llt16_1: + lcbb %r4,0(%r5,%r3),6 /* Get loaded byte count of s2. */ +.Llt16_2: + clr %r1,%r4 + locrh %r1,%r4 /* Get minimum of bytes loaded in s1/2. */ + nill %r1,65532 /* Align bytes loaded to full characters. */ + jz .Lcmp_one_char /* Jump away if no full char is available. */ +.Llt_cmp: + algfr %r5,%r1 /* Add smallest loaded bytes to current_len. */ + vfenezfs %v18,%v16,%v17 /* Compare not equal with zero search. */ + vlgvb %r4,%v18,7 /* Get not equal index or 16 if all equal. */ + clrjl %r4,%r1,.Lfound /* Jump away if miscompare is within loaded + bytes. */ + j .Lloop + +.Lfound: + /* vfenezf found an unequal element or zero. + This instruction compares unsigned words, but wchar_t is signed. + Thus we have to compare the found element again. */ + vlgvb %r4,%v18,7 /* Extract not equal byte-index, */ + srl %r4,2 /* Convert it to character-index. */ + vlgvf %r3,%v16,0(%r4) /* Load character-values. */ + vlgvf %r4,%v17,0(%r4) + cr %r3,%r4 + je .Lend_equal + lghi %r2,1 + lghi %r1,-1 + locgrl %r2,%r1 + br %r14 +.Lend_equal: + lghi %r2,0 + br %r14 +END(__wcscmp_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcscmp.c b/sysdeps/s390/multiarch/wcscmp.c new file mode 100644 index 0000000000..705ef4596e --- /dev/null +++ b/sysdeps/s390/multiarch/wcscmp.c @@ -0,0 +1,28 @@ +/* Multiple versions of wcscmp. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc (__wcscmp) +weak_alias (__wcscmp, wcscmp) + +#else +# include <wcsmbs/wcscmp.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wcscpy-c.c b/sysdeps/s390/multiarch/wcscpy-c.c new file mode 100644 index 0000000000..3450c00048 --- /dev/null +++ b/sysdeps/s390/multiarch/wcscpy-c.c @@ -0,0 +1,25 @@ +/* Default wcscpy implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WCSCPY __wcscpy_c + +# include <wchar.h> +extern __typeof (wcscpy) __wcscpy_c; +# include <wcsmbs/wcscpy.c> +#endif diff --git a/sysdeps/s390/multiarch/wcscpy-vx.S b/sysdeps/s390/multiarch/wcscpy-vx.S new file mode 100644 index 0000000000..2077893130 --- /dev/null +++ b/sysdeps/s390/multiarch/wcscpy-vx.S @@ -0,0 +1,111 @@ +/* Vector optimized 32/64 bit S/390 version of wcscpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* char * wcscpy (const wchar_t *dest, const wchar_t *src) + Copy string src to dest. + + Register usage: + -r0=border-len for switching to vector-instructions + -r1=tmp + -r2=dest and return value + -r3=src + -r4=tmp + -r5=current_len + -v16=part of src + -v17=index of zero + -v18=part of src +*/ +ENTRY(__wcscpy_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */ + + tmll %r3,3 /* Test if s is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + + vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r5,%v17,7 /* Load zero index or 16 if not found. */ + clrjl %r5,%r1,.Lfound_align /* If found zero within loaded bytes, + copy bytes before and return. */ + + /* Align s to 16 byte. */ + risbgn %r4,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,15 /* current_len = 15. */ + slr %r5,%r4 /* Compute highest index to 16byte boundary. */ + + vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */ + ahi %r5,1 /* Start loop at next character. */ + + /* Find zero in 16byte aligned loop. */ +.Lloop: + vl %v16,0(%r5,%r3) /* Load s. */ + vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound_v16_0 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3) /* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */ + vfenezfs %v17,%v18,%v18 + je .Lfound_v18_16 + vl %v16,32(%r5,%r3) + vst %v18,16(%r5,%r2) + vfenezfs %v17,%v16,%v16 + je .Lfound_v16_32 + vl %v18,48(%r5,%r3) + vst %v16,32(%r5,%r2) + vfenezfs %v17,%v18,%v18 + je .Lfound_v18_48 + vst %v18,48(%r5,%r2) + + aghi %r5,64 + j .Lloop /* No zero found -> loop. */ + +.Lfound_v16_32: + aghi %r5,32 +.Lfound_v16_0: + la %r3,0(%r5,%r2) + vlgvb %r1,%v17,7 /* Load byte index of zero. */ + aghi %r1,3 /* Also copy remaining bytes of zero. */ + vstl %v16,%r1,0(%r3) /* Copy characters including zero. */ + br %r14 + +.Lfound_v18_48: + aghi %r5,32 +.Lfound_v18_16: + la %r3,16(%r5,%r2) + vlgvb %r1,%v17,7 /* Load byte index of zero. */ + aghi %r1,3 /* Also copy remaining bytes of zero. */ + vstl %v18,%r1,0(%r3) /* Copy characters including zero. */ + br %r14 + +.Lfound_align: + aghi %r5,3 /* Also copy remaining bytes of zero. */ + vstl %v16,%r5,0(%r2) /* Copy characters including zero. */ + br %r14 + +.Lfallback: + jg __wcscpy_c +END(__wcscpy_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcscpy.c b/sysdeps/s390/multiarch/wcscpy.c new file mode 100644 index 0000000000..8c5f54910b --- /dev/null +++ b/sysdeps/s390/multiarch/wcscpy.c @@ -0,0 +1,27 @@ +/* Multiple versions of wcscpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__wcscpy, wcscpy) + +#else +# include <wcsmbs/wcscpy.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wcscspn-c.c b/sysdeps/s390/multiarch/wcscspn-c.c new file mode 100644 index 0000000000..e8fd2a53d9 --- /dev/null +++ b/sysdeps/s390/multiarch/wcscspn-c.c @@ -0,0 +1,26 @@ +/* Default wcscscpn implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WCSCSPN __wcscspn_c + +# include <wchar.h> +extern __typeof (wcscspn) __wcscspn_c; + +# include <wcsmbs/wcscspn.c> +#endif diff --git a/sysdeps/s390/multiarch/wcscspn-vx.S b/sysdeps/s390/multiarch/wcscspn-vx.S new file mode 100644 index 0000000000..b0b1066658 --- /dev/null +++ b/sysdeps/s390/multiarch/wcscspn-vx.S @@ -0,0 +1,293 @@ +/* Vector optimized 32/64 bit S/390 version of wcscspn. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* size_t wcscspn (const wchar_t *s, const wchar_t * reject) + The wcscspn() function calculates the length of the initial segment + of s which consists entirely of characters not in reject. + + This method checks the length of reject string. If it fits entirely + in one vector register, a fast algorithm is used, which does not need + to check multiple parts of accept-string. Otherwise a slower full + check of accept-string is used. + + register overview: + r3: pointer to start of reject-string + r2: pointer to start of search-string + r0: loaded byte count of vlbb search-string + r4: found byte index + r1: current return len + v16: search-string + v17: reject-string + v18: temp-vreg + + ONLY FOR SLOW: + v19: first reject-string + v20: zero for preparing acc-vector + v21: global mask; 1 indicates a match between + search-string-vreg and any reject-character + v22: current mask; 1 indicates a match between + search-string-vreg and any reject-character in current acc-vreg + v30, v31: for re-/storing registers r6, r8, r9 + r5: current len of reject-string + r6: zero-index in search-string or 16 if no zero + or min(zero-index, loaded byte count) + r8: >0, if former reject-string-part contains a zero, + otherwise =0; + r9: loaded byte count of vlbb reject-string +*/ +ENTRY(__wcscspn_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + tmll %r2,3 /* Test if s is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + + /* + Check if reject-string fits in one vreg: + ---------------------------------------- + */ + vlbb %v17,0(%r3),0 /* Load reject. */ + lcbb %r0,0(%r3),0 + jo .Lcheck_onbb /* Special case if reject + lays on block-boundary. */ + +.Lcheck_notonbb: + lghi %r1,0 /* Zero out current len. */ + vistrfs %v17,%v17 /* Fill with zeros after first zero. */ + je .Lfast /* Zero found -> reject fits in one vreg. */ + j .Lslow /* No zero -> reject exceeds one vreg. */ + + +.Lcheck_onbb: + /* Reject lays on block-boundary. */ + nill %r0,65532 /* Recognize only fully loaded characters. */ + je .Lcheck_onbb2 /* Reload vr, if we loaded no full wchar_t. */ + vfenezf %v18,%v17,%v17 /* Search zero in loaded reject bytes. */ + vlgvb %r4,%v18,7 /* Get index of zero or 16 if not found. */ + clrjl %r4,%r0,.Lcheck_notonbb /* Zero index < loaded bytes count -> + Reject fits in one vreg; + Fill with zeros and proceed + with FAST. */ +.Lcheck_onbb2: + vl %v17,0(%r3) /* Load reject, which exceeds loaded bytes. */ + j .Lcheck_notonbb /* Check if reject fits in one vreg. */ + + + /* + Search s for reject in one vreg + ------------------------------- + */ +.Lfast: + /* Complete reject-string in v17 and remaining bytes are zero. */ + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r0,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + vfaezfs %v18,%v16,%v17,0 /* Find first element in v16 + unequal to any in v17 + or first zero element. */ + vlgvb %r4,%v18,7 /* Load byte index of found element. */ + clrjl %r4,%r0,.Lfast_loop_found2 /* If found index is within loaded + bytes, return with found element + index (=equal count). */ + + /* Align s to 16 byte. */ + risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r1,16 /* current_len = 16. */ + slr %r1,%r4 /* Compute bytes to 16bytes boundary. */ + + /* Process s in 16byte aligned loop. */ +.Lfast_loop: + vl %v16,0(%r1,%r2) /* Load search-string. */ + vfaezfs %v18,%v16,%v17,0 /* Find first element in v16 equal to any + in v17 or first zero element. */ + jno .Lfast_loop_found + + vl %v16,16(%r1,%r2) + vfaezfs %v18,%v16,%v17,0 + jno .Lfast_loop_found16 + + vl %v16,32(%r1,%r2) + vfaezfs %v18,%v16,%v17,0 + jno .Lfast_loop_found32 + + vl %v16,48(%r1,%r2) + vfaezfs %v18,%v16,%v17,0 + jno .Lfast_loop_found48 + + aghi %r1,64 + j .Lfast_loop /* Loop if no element was unequal to reject + and not zero. */ + + /* Found equal or zero element. */ +.Lfast_loop_found48: + aghi %r1,16 +.Lfast_loop_found32: + aghi %r1,16 +.Lfast_loop_found16: + aghi %r1,16 +.Lfast_loop_found: + vlgvb %r4,%v18,7 /* Load byte index of found element or zero. */ +.Lfast_loop_found2: + algrk %r2,%r1,%r4 /* Add found index to current len. */ + srlg %r2,%r2,2 /* Convert byte-count to character-count. */ + br %r14 + + + + /* + Search s for reject in multiple vregs + ------------------------------------- + */ +.Lslow: + /* Save registers. */ + vlvgg %v30,%r6,0 + vlvgp %v31,%r8,%r9 + + /* Reject in v17 without zero. */ + vlr %v19,%v17 /* Save first acc-part for a fast reload. */ + vzero %v20 /* Zero for preparing acc-vector. */ + vone %v24 /* One for checking result of former + string-part. */ + + /* Align s to 16 byte. */ + risbg %r4,%r2,60,128+63,0 /* Test if s is aligned and + %r4 = bits 60-63 'and' 15. */ + je .Lslow_loop_str /* If s is aligned, loop aligned. */ + lghi %r0,15 + slr %r0,%r4 /* Compute highest index to load (15-x). */ + vll %v16,%r0,0(%r2) /* Load up to 16byte boundary (vll needs + highest index, remaining bytes are 0). */ + ahi %r0,1 /* Work with loaded byte count. */ + vzero %v21 /* Zero out global mask. */ + lghi %r5,0 /* Set current len of reject-string to zero. */ + vfenezf %v18,%v16,%v16 /* Find zero in current string-part. */ + lghi %r8,0 /* There is no zero in first reject-part. */ + vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */ + clije %r6,0,.Lslow_end /* If first element is zero -> return 0. */ + clr %r0,%r6 /* cc==1 if loaded byte count < zero-index. */ + locrl %r6,%r0 /* Load on cc==1; zero-index = lbc. */ + j .Lslow_loop_acc + + + /* Process s in 16byte aligned loop. */ +.Lslow_next_str: + /* Check results of former processed str-part. */ + vfeef %v18,%v21,%v24 /* Find first equal match in global mask + (ones in element). */ + vlgvb %r4,%v18,7 /* Get index of first one (=equal) or 16. */ + /* Equal-index < min(zero-index, loaded byte count) + -> Return pointer to equal element. */ + clrjl %r4,%r6,.Lslow_index_found + /* Zero-index < loaded byte count + -> Former str-part was last str-part + -> Return null */ + clrjl %r6,%r0,.Lslow_end_not_found + + /* All elements are zero (=no match) -> proceed with next str-part. */ + vlr %v17,%v19 /* Load first part of reject (no zero). */ + algfr %r1,%r0 /* Add loaded byte count to current len. */ + +.Lslow_loop_str: + vl %v16,0(%r1,%r2) /* Load search-string. */ + lghi %r0,16 /* Loaded byte count is 16. */ + vzero %v21 /* Zero out global mask. */ + lghi %r5,0 /* Set current len of reject to zero. */ + vfenezf %v18,%v16,%v16 /* Find zero in current string-part. */ + lghi %r8,0 /* There is no zero in first reject-part. */ + vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */ + clije %r6,0,.Lslow_end /* If first element is zero (end of string) + -> Return current length. */ + +.Lslow_loop_acc: + vfaef %v22,%v16,%v17,4 /* Create matching-mask (1 in mask -> + Character matches any rejected character in + this reject-string-part) IN=0, RT=1. */ + vlgvf %r4,%v22,0 /* Get result of first element. */ + /* First element is equal to any rejected characters? + (All other parts of reject cannot lead to a match before this one) + -> Return current len, which is pointing to this element. */ + clijh %r4,0,.Lslow_end + vo %v21,%v21,%v22 /* Global-mask = global-|matching-mask. */ + /* Proceed with next acc until end of acc is reached. */ + + +.Lslow_next_acc: + clijh %r8,0,.Lslow_next_str /* There was a zero in last reject-part + -> Add found index to current len + and end. */ + vlbb %v17,16(%r5,%r3),6 /* Load next reject part. */ + aghi %r5,16 /* Increment current len of reject-string. */ + lcbb %r9,0(%r5,%r3),6 /* Get loaded byte count of reject-string. */ + jo .Lslow_next_acc_onbb /* Jump away if reject-string is + on block-boundary. */ +.Lslow_next_acc_notonbb: + vistrfs %v17,%v17 /* Fill with zeros after first zero. */ + jo .Lslow_loop_acc /* No zero found -> no preparation needed. */ + +.Lslow_next_acc_prepare_zero: + /* Zero in reject-part: fill zeros with first-reject-character. */ + vlgvf %r8,%v17,0 /* Load first element of reject-part. */ + clije %r8,0,.Lslow_next_str /* Process next str-part if first + character in this part of reject + is a zero. */ + /* r8>0 -> zero found in this acc-part. */ + vrepf %v18,%v17,0 /* Replicate first char accross all chars. */ + vceqf %v22,%v20,%v17 /* Create a mask (v22) of null chars + by comparing with 0 (v20). */ + vsel %v17,%v18,%v17,%v22 /* Replace null chars with first char. */ + j .Lslow_loop_acc /* Reject-string part is prepared. */ + +.Lslow_next_acc_onbb: + nill %r9,65532 /* Recognize only fully loaded characters. */ + je .Lslow_next_acc_onbb2 /* Reload vr, if no full wchar_t + loaded. */ + vfenezf %v18,%v17,%v17 /* Find zero in loaded bytes of reject part. */ + vlgvb %r8,%v18,7 /* Load byte index of zero. */ + clrjl %r8,%r9,.Lslow_next_acc_notonbb /* Found a zero in loaded bytes + -> Prepare vreg. */ +.Lslow_next_acc_onbb2: + vl %v17,0(%r5,%r3) /* Load over boundary ... */ + lghi %r8,0 /* r8=0 -> no zero in this part of acc, + check for zero is in jump-target. */ + j .Lslow_next_acc_notonbb /* ... and search for zero in + fully loaded vreg again. */ + +.Lslow_end_not_found: + algfr %r1,%r6 /* Add zero-index to current len. */ + j .Lslow_end +.Lslow_index_found: + algfr %r1,%r4 /* Add found index of char to current len. */ +.Lslow_end: + srlg %r2,%r1,2 /* Convert byte-count to character-count. */ + /* Restore registers. */ + vlgvg %r6,%v30,0 + vlgvg %r8,%v31,0 + vlgvg %r9,%v31,1 + br %r14 +.Lfallback: + jg __wcscspn_c +END(__wcscspn_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcscspn.c b/sysdeps/s390/multiarch/wcscspn.c new file mode 100644 index 0000000000..ebd77734ac --- /dev/null +++ b/sysdeps/s390/multiarch/wcscspn.c @@ -0,0 +1,27 @@ +/* Multiple versions of wcscspn. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__wcscspn, wcscspn) + +#else +# include <wcsmbs/wcscspn.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wcslen-c.c b/sysdeps/s390/multiarch/wcslen-c.c new file mode 100644 index 0000000000..dcbe3094d9 --- /dev/null +++ b/sysdeps/s390/multiarch/wcslen-c.c @@ -0,0 +1,25 @@ +/* Default wcslen implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WCSLEN __wcslen_c + +# include <wchar.h> +extern __typeof (__wcslen) __wcslen_c; +# include <wcsmbs/wcslen.c> +#endif diff --git a/sysdeps/s390/multiarch/wcslen-vx.S b/sysdeps/s390/multiarch/wcslen-vx.S new file mode 100644 index 0000000000..dafb7b799d --- /dev/null +++ b/sysdeps/s390/multiarch/wcslen-vx.S @@ -0,0 +1,91 @@ +/* Vector optimized 32/64 bit S/390 version of wcslen. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* size_t wcslen (const wchar_t *s) + Returns length of string s. + + Register usage: + -r1=bytes to 4k-byte boundary + -r2=s + -r3=tmp + -r4=tmp + -r5=current_len and return_value + -v16=part of s +*/ +ENTRY(__wcslen_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + tmll %r2,3 /* Test if s is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + + vfenezf %v16,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r4,%v16,7 /* Load zero index or 16 if not found. */ + clr %r4,%r1 /* If found zero within loaded bytes? */ + locgrl %r2,%r4 /* Then copy return value. */ + jl .Lend /* And return. */ + + /* Align s to 16 byte. */ + risbgn %r3,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,16 /* current_len = 16. */ + slr %r5,%r3 /* Compute bytes to 16bytes boundary. */ + + /* Find zero in 16byte aligned loop. */ +.Lloop: + vl %v16,0(%r5,%r2) /* Load s. */ + vfenezfs %v16,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound /* Jump away if zero was found. */ + vl %v16,16(%r5,%r2) + vfenezfs %v16,%v16,%v16 + je .Lfound16 + vl %v16,32(%r5,%r2) + vfenezfs %v16,%v16,%v16 + je .Lfound32 + vl %v16,48(%r5,%r2) + vfenezfs %v16,%v16,%v16 + je .Lfound48 + + aghi %r5,64 + j .Lloop /* No zero found -> loop. */ + +.Lfound48: + aghi %r5,16 +.Lfound32: + aghi %r5,16 +.Lfound16: + aghi %r5,16 +.Lfound: + vlgvb %r2,%v16,7 /* Load byte index of zero. */ + algr %r2,%r5 +.Lend: + srlg %r2,%r2,2 /* Convert byte-count to character-count. */ + br %r14 +.Lfallback: + jg __wcslen_c +END(__wcslen_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcslen.c b/sysdeps/s390/multiarch/wcslen.c new file mode 100644 index 0000000000..540845f70a --- /dev/null +++ b/sysdeps/s390/multiarch/wcslen.c @@ -0,0 +1,28 @@ +/* Multiple versions of wcslen. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc (__wcslen) +weak_alias (__wcslen, wcslen) + +#else +# include <wcsmbs/wcslen.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wcsncat-c.c b/sysdeps/s390/multiarch/wcsncat-c.c new file mode 100644 index 0000000000..e8cc219eac --- /dev/null +++ b/sysdeps/s390/multiarch/wcsncat-c.c @@ -0,0 +1,25 @@ +/* Default wcsncat implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WCSNCAT __wcsncat_c + +# include <wchar.h> +extern __typeof (wcsncat) __wcsncat_c; +# include <wcsmbs/wcsncat.c> +#endif diff --git a/sysdeps/s390/multiarch/wcsncat-vx.S b/sysdeps/s390/multiarch/wcsncat-vx.S new file mode 100644 index 0000000000..4264f6d21d --- /dev/null +++ b/sysdeps/s390/multiarch/wcsncat-vx.S @@ -0,0 +1,265 @@ +/* Vector optimized 32/64 bit S/390 version of wcsncat. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* wchar_t * wcsncat (wchar_t *dest, const wchar_t *src, size_t n) + Concatenate two strings - at most n characters of src. + + Register usage: + -r0=saved dest pointer for return + -r1=tmp + -r2=dest + -r3=src + -r4=n + -r5=current_len + -r6=tmp + -r7=tmp + -v16=part of src + -v17=index of zero + -v18=part of src + -v31=register save area for r6, r7 +*/ +ENTRY(__wcsncat_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + +# if !defined __s390x__ + llgfr %r4,%r4 +# endif /* !defined __s390x__ */ + + clgfi %r4,0 + ber %r14 /* Nothing to do, if n == 0. */ + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + /* If either src or dest is not 4byte aligned, use __wcsncat_c. */ + tmll %r2,3 /* Test if s is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + tmll %r3,3 /* Test if src is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + + lgr %r0,%r2 /* Save destination pointer for return. */ + vlvgp %v31,%r6,%r7 /* Save registers. */ + + /* WCSLEN + %r1 = loaded bytes (tmp) + %r6 = zero byte index (tmp) + %r2 = dst + */ + vfenezf %v16,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r5,%v16,7 /* Load zero index or 16 if not found. */ + clrjl %r5,%r1,.Llen_end /* Found zero within loaded bytes, end. */ + + /* Align s to 16 byte. */ + risbgn %r1,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,16 /* current_len = 16. */ + slr %r5,%r1 /* Compute bytes to 16bytes boundary. */ + + /* Find zero in 16byte aligned loop. */ +.Llen_loop: + vl %v16,0(%r5,%r2) /* Load s. */ + vfenezfs %v16,%v16,%v16 /* Find element not equal with zero search. */ + je .Llen_found /* Jump away if zero was found. */ + vl %v16,16(%r5,%r2) + vfenezfs %v16,%v16,%v16 + je .Llen_found16 + vl %v16,32(%r5,%r2) + vfenezfs %v16,%v16,%v16 + je .Llen_found32 + vl %v16,48(%r5,%r2) + vfenezfs %v16,%v16,%v16 + je .Llen_found48 + + aghi %r5,64 + j .Llen_loop /* No zero -> loop. */ + +.Llen_found48: + aghi %r5,16 +.Llen_found32: + aghi %r5,16 +.Llen_found16: + aghi %r5,16 +.Llen_found: + vlgvb %r1,%v16,7 /* Load byte index of zero. */ + algr %r5,%r1 + +.Llen_end: + /* WCSNCPY + %r1 = zero byte index (tmp) + %r6 = loaded bytes (tmp) + %r3 = curr src pointer + %r2 = curr dst pointer + %r7 = border, tmp + */ + la %r2,0(%r5,%r2) /* strcpy at end of dst-string. */ + + vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */ + lcbb %r6,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */ + llgfr %r6,%r6 /* Convert 32bit to 64bit. */ + + lghi %r5,0 /* current_len = 0. */ + + /* Check range of maxlen and convert to byte-count. */ +# ifdef __s390x__ + tmhh %r4,49152 /* Test bit 0 or 1 of maxlen. */ + lghi %r1,-4 /* Max byte-count is 18446744073709551612. */ +# else + tmlh %r4,49152 /* Test bit 0 or 1 of maxlen. */ + llilf %r1,4294967292 /* Max byte-count is 4294967292. */ +# endif /* !__s390x__ */ + sllg %r4,%r4,2 /* Convert character-count to byte-count. */ + locgrne %r4,%r1 /* Use max byte-count, if bit 0/1 was one. */ + + clgrjle %r4,%r6,.Lcpy_remaining_v16 /* If n <= loaded-bytes + -> process remaining. */ + + /* n > loaded-byte-count. */ + vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */ + clrjl %r1,%r6,.Lcpy_found_v16_store /* Found zero within loaded bytes, + copy and return. */ + + /* Align s to 16 byte. */ + risbgn %r1,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,15 /* current_len = 15. */ + slr %r5,%r1 /* Compute highest index to 16byte boundary. * + + /* Zero not found and maxlen > loaded-byte-count. */ + vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */ + ahi %r5,1 /* Start loop at next character. */ + + /* + Now we are 16byte aligned, so we can load a full vreg + without page fault. + */ + lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */ + aghi %r1,64 + clgrjl %r1,%r4,.Lcpy_loop64 + + vl %v16,0(%r5,%r3) /* Load s. */ + clgijl %r4,17,.Lcpy_remaining_v16 /* If n <=16, + process remaining bytes. */ +.Lcpy_lt64: + lgr %r7,%r4 + slgfi %r7,16 /* border_len = n - 16. */ + + clgrjhe %r5,%r7,.Lcpy_remaining_v16 + vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lcpy_found_v16 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3) /* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Save previous part without zero to dst. */ + aghi %r5,16 + + clgrjhe %r5,%r7,.Lcpy_remaining_v18 + vfenezfs %v17,%v18,%v18 + je .Lcpy_found_v18 + vl %v16,16(%r5,%r3) + vst %v18,0(%r5,%r2) + aghi %r5,16 + + clgrjhe %r5,%r7,.Lcpy_remaining_v16 + vfenezfs %v17,%v16,%v16 + je .Lcpy_found_v16 + vl %v18,16(%r5,%r3) + vst %v16,0(%r5,%r2) + aghi %r5,16 + +.Lcpy_remaining_v18: + vlr %v16,%v18 +.Lcpy_remaining_v16: + /* v16 contains the remaining bytes [1...16]. + Store remaining bytes and append string-termination. */ + vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */ + slgrk %r7,%r4,%r5 /* Remaining bytes = maxlen - current_len. */ + aghi %r7,-1 /* vstl needs highest index. */ + vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */ + la %r2,0(%r5,%r2) /* vstl has no index register. */ + /* Zero-index within remaining-bytes, store up to zero and end. */ + clgrjle %r1,%r7,.Lcpy_found_v16_store + vstl %v16,%r7,0(%r2) /* Store remaining bytes. */ + lghi %r1,0 + st %r1,1(%r7,%r2) /* Store string-null-termination beyond n. */ +.Lcpy_end: + /* Restore saved registers. */ + vlgvg %r6,%v31,0 + vlgvg %r7,%v31,1 + lgr %r2,%r0 /* Load saved dest-ptr. */ + br %r14 + +.Lcpy_found_v16_32: + aghi %r5,32 + j .Lcpy_found_v16 +.Lcpy_found_v18_48: + aghi %r5,32 +.Lcpy_found_v18_16: + aghi %r5,16 +.Lcpy_found_v18: + vlr %v16,%v18 +.Lcpy_found_v16: + /* v16 contains a zero. Store remaining bytes to zero. current_len + has not reached border, thus checking for n is not needed! */ + vlgvb %r1,%v17,7 /* Load byte index of zero. */ + la %r2,0(%r5,%r2) +.Lcpy_found_v16_store: + aghi %r1,3 /* Also copy remaining bytes of zero. */ + vstl %v16,%r1,0(%r2) /* Copy characters including zero. */ + j .Lcpy_end + + /* Find zero in 16byte aligned loop. */ +.Lcpy_loop2: + vl %v16,16(%r5,%r3) + vst %v18,0(%r5,%r2) + aghi %r5,16 + +.Lcpy_loop64: + vl %v16,0(%r5,%r3) + vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lcpy_found_v16 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3) /* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Save previous part without zero to dst. */ + vfenezfs %v17,%v18,%v18 + je .Lcpy_found_v18_16 + vl %v16,32(%r5,%r3) + vst %v18,16(%r5,%r2) + vfenezfs %v17,%v16,%v16 + je .Lcpy_found_v16_32 + vl %v18,48(%r5,%r3) + vst %v16,32(%r5,%r2) + vfenezfs %v17,%v18,%v18 + je .Lcpy_found_v18_48 + vst %v18,48(%r5,%r2) + + aghi %r5,64 + lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */ + aghi %r1,64 + clgrjl %r1,%r4,.Lcpy_loop64 + + vl %v16,0(%r5,%r3) /* Load s. */ + j .Lcpy_lt64 + +.Lfallback: + jg __wcsncat_c +END(__wcsncat_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcsncat.c b/sysdeps/s390/multiarch/wcsncat.c new file mode 100644 index 0000000000..62073321e8 --- /dev/null +++ b/sysdeps/s390/multiarch/wcsncat.c @@ -0,0 +1,27 @@ +/* Multiple versions of wcsncat. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__wcsncat, wcsncat) + +#else +# include <wcsmbs/wcsncat.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wcsncmp-c.c b/sysdeps/s390/multiarch/wcsncmp-c.c new file mode 100644 index 0000000000..8f2573810d --- /dev/null +++ b/sysdeps/s390/multiarch/wcsncmp-c.c @@ -0,0 +1,25 @@ +/* Default wcsncmp implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WCSNCMP __wcsncmp_c + +# include <wchar.h> +extern __typeof (wcsncmp) __wcsncmp_c; +# include <wcsmbs/wcsncmp.c> +#endif diff --git a/sysdeps/s390/multiarch/wcsncmp-vx.S b/sysdeps/s390/multiarch/wcsncmp-vx.S new file mode 100644 index 0000000000..e77f17dcaf --- /dev/null +++ b/sysdeps/s390/multiarch/wcsncmp-vx.S @@ -0,0 +1,177 @@ +/* Vector optimized 32/64 bit S/390 version of wcsncmp. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* int wcsncmp (const wchar_t *s1, const wchar_t *s2, size_t n) + Compare at most n characters of two strings. + + Register usage: + -r0=tmp + -r1=tmp + -r2=s1 + -r3=s2 + -r4=n + -r5=current_len + -v16=part of s1 + -v17=part of s2 + -v18=index of unequal +*/ +ENTRY(__wcsncmp_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + +# if !defined __s390x__ + llgfr %r4,%r4 +# endif /* !defined __s390x__ */ + + clgije %r4,0,.Lend_equal /* Nothing to do if n == 0. */ + + /* Check range of n and convert to byte-count. */ +# ifdef __s390x__ + tmhh %r4,49152 /* Test bit 0 or 1 of maxlen. */ + lghi %r1,-4 /* Max byte-count is 18446744073709551612. */ +# else + tmlh %r4,49152 /* Test bit 0 or 1 of maxlen. */ + llilf %r1,4294967292 /* Max byte-count is 4294967292. */ +# endif /* !__s390x__ */ + sllg %r4,%r4,2 /* Convert character-count to byte-count. */ + locgrne %r4,%r1 /* Use max byte-count, if bit 0/1 was one. */ + + /* Check first character without vector load. */ + lghi %r5,4 /* current_len = 4 bytes. */ + /* Check s1/2[0]. */ + lt %r0,0(%r2) + l %r1,0(%r3) + je .Lend_cmp_one_char + crjne %r0,%r1,.Lend_cmp_one_char + +.Lloop: + vlbb %v17,0(%r5,%r3),6 /* Load s2 to block boundary. */ + vlbb %v16,0(%r5,%r2),6 /* Load s1 to block boundary. */ + lcbb %r0,0(%r5,%r2),6 /* Get loaded byte count of s1. */ + jo .Llt16_1 /* Jump away if vector not fully loaded. */ + lcbb %r1,0(%r5,%r3),6 /* Get loaded byte count of s2. */ + jo .Llt16_2 /* Jump away if vector not fully loaded. */ + aghi %r5,16 /* Both vectors are fully loaded. */ + vfenezfs %v18,%v16,%v17 /* Compare not equal with zero search. */ + clgrjhe %r5,%r4,.Llastcmp /* If current_len >= n ->last compare. */ + jno .Lfound + + vlbb %v17,0(%r5,%r3),6 + vlbb %v16,0(%r5,%r2),6 + lcbb %r0,0(%r5,%r2),6 + jo .Llt16_1 + lcbb %r1,0(%r5,%r3),6 + jo .Llt16_2 + aghi %r5,16 + vfenezfs %v18,%v16,%v17 + clgrjhe %r5,%r4,.Llastcmp + jno .Lfound + + vlbb %v17,0(%r5,%r3),6 + vlbb %v16,0(%r5,%r2),6 + lcbb %r0,0(%r5,%r2),6 + jo .Llt16_1 + lcbb %r1,0(%r5,%r3),6 + jo .Llt16_2 + aghi %r5,16 + vfenezfs %v18,%v16,%v17 + clgrjhe %r5,%r4,.Llastcmp + jno .Lfound + + vlbb %v17,0(%r5,%r3),6 + vlbb %v16,0(%r5,%r2),6 + lcbb %r0,0(%r5,%r2),6 + jo .Llt16_1 + lcbb %r1,0(%r5,%r3),6 + jo .Llt16_2 + aghi %r5,16 + vfenezfs %v18,%v16,%v17 + clgrjhe %r5,%r4,.Llastcmp + jno .Lfound + + j .Lloop + +.Llt16_1: + lcbb %r1,0(%r5,%r3),6 /* Get loaded byte count of s2. */ +.Llt16_2: + clr %r0,%r1 /* Compare logical. */ + locrh %r0,%r1 /* Compute minimum of bytes loaded. */ + nill %r0,65532 /* Align bytes loaded to full characters. */ + jz .Lcmp_one_char /* Jump away if no full char is available. */ +.Llt_cmp: + algfr %r5,%r0 /* Add smallest loaded bytes to current_len. */ + vfenezfs %v18,%v16,%v17 /* Compare not equal with zero search. */ + clgrj %r5,%r4,10,.Llastcmp /* If current_len >= n -> last compare */ + vlgvb %r1,%v18,7 /* Get not equal index or 16 if all equal. */ + clrjl %r1,%r0,.Lfound /* Jump away if miscompare is within + loaded bytes; (index < loaded-bytes) */ + j .Lloop + +.Lcmp_one_char: + /* At least one of both strings is not 4-byte aligned + and there is no full character before next block-boundary. + Compare one character to get over the boundary and + proceed with normal loop! */ + vlef %v16,0(%r5,%r2),0 /* Load one character. */ + lghi %r0,4 /* Loaded byte count is 4. */ + vlef %v17,0(%r5,%r3),0 + j .Llt_cmp /* Proceed with comparision. */ + +.Llastcmp: + /* Use comparision result only if located within first n characters. + %r0: loaded byte count in vreg; + %r5: current_len; + %r4: n; + (current_len - n): [0...16[ + First ignored match index: loaded bytes - (current_len-n): ]0...16] + */ + slgr %r5,%r4 /* %r5 = current_len - n. */ + slr %r0,%r5 /* %r0 = first ignored match index. */ + vlgvb %r4,%v18,7 /* Get not equal index or 16 if all equal. */ + clrjl %r4,%r0,.Lfound2 /* Jump away if miscompare is within + loaded bytes and below n bytes. */ +.Lend_equal: + lghi %r2,0 + br %r14 + +.Lfound: + /* Difference or end of string. */ + /* vfenezf found an unequal element or zero. + This instruction compares unsigned words, but wchar_t is signed. + Thus we have to compare the found element again. */ + vlgvb %r4,%v18,7 /* Extract not equal byte-index. */ +.Lfound2: + srl %r4,2 /* And convert it to character-index. */ + vlgvf %r0,%v16,0(%r4) /* Load character-values. */ + vlgvf %r1,%v17,0(%r4) +.Lend_cmp_one_char: + cr %r0,%r1 + je .Lend_equal + lghi %r2,1 + lghi %r1,-1 + locgrl %r2,%r1 + br %r14 +END(__wcsncmp_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcsncmp.c b/sysdeps/s390/multiarch/wcsncmp.c new file mode 100644 index 0000000000..3482d90e4e --- /dev/null +++ b/sysdeps/s390/multiarch/wcsncmp.c @@ -0,0 +1,27 @@ +/* Multiple versions of wcsncmp. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__wcsncmp, wcsncmp) + +#else +# include <wcsmbs/wcsncmp.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wcsncpy-c.c b/sysdeps/s390/multiarch/wcsncpy-c.c new file mode 100644 index 0000000000..b63d86ef5f --- /dev/null +++ b/sysdeps/s390/multiarch/wcsncpy-c.c @@ -0,0 +1,25 @@ +/* Default wcsncpy implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WCSNCPY __wcsncpy_c + +# include <wchar.h> +extern __typeof (__wcsncpy) __wcsncpy_c; +# include <wcsmbs/wcsncpy.c> +#endif diff --git a/sysdeps/s390/multiarch/wcsncpy-vx.S b/sysdeps/s390/multiarch/wcsncpy-vx.S new file mode 100644 index 0000000000..33cc33f28b --- /dev/null +++ b/sysdeps/s390/multiarch/wcsncpy-vx.S @@ -0,0 +1,223 @@ +/* Vector optimized 32/64 bit S/390 version of wcsncpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* wchar_t *wcsncpy (const wchar_t *dest, const wchar_t *src, size_t n) + Copy at most n characters of string src to dest. + + Register usage: + -r0=dest pointer for return + -r1=tmp, zero byte index + -r2=dest + -r3=src + -r4=n + -r5=current_len + -r6=tmp, loaded bytes + -r7=tmp, border + -v16=part of src + -v17=index of zero + -v18=part of src + -v31=register save area for r6, r7 +*/ +ENTRY(__wcsncpy_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + +# if !defined __s390x__ + llgfr %r4,%r4 +# endif /* !defined __s390x__ */ + + clgfi %r4,0 + ber %r14 /* Nothing to do, if n == 0. */ + + vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */ + + tmll %r3,3 /* Test if s is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + + vlvgp %v31,%r6,%r7 /* Save registers. */ + lgr %r0,%r2 /* Save destination pointer for return. */ + + lcbb %r6,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */ + llgfr %r6,%r6 /* Convert 32bit to 64bit. */ + + lghi %r5,0 /* current_len = 0. */ + + /* Check range of maxlen and convert to byte-count. */ +# ifdef __s390x__ + tmhh %r4,49152 /* Test bit 0 or 1 of n. */ + lghi %r1,-4 /* Max byte-count is 18446744073709551612. */ +# else + tmlh %r4,49152 /* Test bit 0 or 1 of n. */ + llilf %r1,4294967292 /* Max byte-count is 4294967292. */ +# endif /* !__s390x__ */ + sllg %r4,%r4,2 /* Convert character-count to byte-count. */ + locgrne %r4,%r1 /* Use max byte-count, if bit 0/1 was one. */ + + clgrjle %r4,%r6,.Lremaining_v16 /* If n <= loaded-bytes + -> process remaining. */ + + /* n > loaded-byte-count. */ + vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */ + aghi %r1,3 /* Also copy remaining bytes of zero. */ + clrjl %r1,%r6,.Lfound_v16_store /* Found zero within loaded bytes, + copy and return. */ + + /* Align s to 16 byte. */ + risbgn %r7,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,15 /* current_len = 15. */ + slr %r5,%r7 /* Compute highest index to 16byte boundary. */ + + /* Zero not found and n > loaded-byte-count. */ + vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */ + ahi %r5,1 /* Start loop at next character. */ + + /* Now we are 16byte aligned, so we can load + a full vreg without page fault. */ + lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */ + aghi %r1,64 + clgrjl %r1,%r4,.Lloop64 + + vl %v16,0(%r5,%r3) /* Load s. */ + clgijl %r4,17,.Lremaining_v16 /* If n <=16, process remaining + bytes. */ +.Llt64: + lgr %r7,%r4 + slgfi %r7,16 /* border_len = maxlen - 16. */ + + clgrjhe %r5,%r7,.Lremaining_v16 /* If current_len >= border + then process remaining bytes. */ + vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound_v16 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3) /* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */ + aghi %r5,16 + + clgrjhe %r5,%r7,.Lremaining_v18 + vfenezfs %v17,%v18,%v18 + je .Lfound_v18 + vl %v16,16(%r5,%r3) + vst %v18,0(%r5,%r2) + aghi %r5,16 + + clgrjhe %r5,%r7,.Lremaining_v16 + vfenezfs %v17,%v16,%v16 + je .Lfound_v16 + vl %v18,16(%r5,%r3) + vst %v16,0(%r5,%r2) + aghi %r5,16 + +.Lremaining_v18: + vlr %v16,%v18 +.Lremaining_v16: + /* v16 contains the remaining bytes [1...16]. + Store remaining bytes and append string-termination. */ + vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */ + slgrk %r7,%r4,%r5 /* Remaining bytes = maxlen - current_len. */ + aghi %r7,-1 /* vstl needs highest index. */ + la %r2,0(%r5,%r2) /* vstl has no index register. */ + vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */ + aghi %r1,3 /* Also copy remaining bytes of zero. */ + /* Zero in remaining bytes? -> jump away (zero-index < max-index) + Do not jump away if zero-index == max-index, + but simply copy zero with vstl below. */ + clrjl %r1,%r7,.Lfound_v16_store + vstl %v16,%r7,0(%r2) /* Store remaining bytes without null + termination!. */ +.Lend: + /* Restore saved registers. */ + vlgvg %r6,%v31,0 + vlgvg %r7,%v31,1 + lgr %r2,%r0 /* Load saved dest-ptr. */ + br %r14 + +.Lfound_v16_32: + aghi %r5,32 + j .Lfound_v16 +.Lfound_v18_48: + aghi %r5,32 +.Lfound_v18_16: + aghi %r5,16 +.Lfound_v18: + vlr %v16,%v18 +.Lfound_v16: + /* v16 contains a zero. Store remaining bytes to zero. current_len + has not reached border, thus checking for n is not needed! */ + vlgvb %r1,%v17,7 /* Load byte index of zero. */ + la %r2,0(%r5,%r2) /* vstl has no support for index-register. */ + aghi %r1,3 /* Also copy remaining bytes of zero. */ +.Lfound_v16_store: + vstl %v16,%r1,0(%r2) /* Copy characters including zero. */ + /* Fill remaining bytes with zero - remaining count always > 0. */ + algr %r5,%r1 /* Remaining bytes (=%r4) = ... */ + slgr %r4,%r5 /* = maxlen - (currlen + zero_index + 1). */ + la %r2,0(%r1,%r2) /* Pointer to zero. start filling beyond. */ + aghi %r4,-2 /* mvc with exrl needs count - 1. + (additional -1, see remaining bytes above) */ + srlg %r6,%r4,8 /* Split into 256 byte blocks. */ + ltgr %r6,%r6 + je .Lzero_lt256 +.Lzero_loop256: + mvc 1(256,%r2),0(%r2) /* Fill 256 zeros at once. */ + la %r2,256(%r2) + brctg %r6,.Lzero_loop256 /* Loop until all blocks are processed. */ +.Lzero_lt256: + exrl %r4,.Lmvc_lt256 + j .Lend +.Lmvc_lt256: + mvc 1(1,%r2),0(%r2) + + /* Find zero in 16byte aligned loop. */ +.Lloop64: + vl %v16,0(%r5,%r3) /* Load s. */ + vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound_v16 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3) /* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */ + vfenezfs %v17,%v18,%v18 + je .Lfound_v18_16 + vl %v16,32(%r5,%r3) + vst %v18,16(%r5,%r2) + vfenezfs %v17,%v16,%v16 + je .Lfound_v16_32 + vl %v18,48(%r5,%r3) + vst %v16,32(%r5,%r2) + vfenezfs %v17,%v18,%v18 + je .Lfound_v18_48 + vst %v18,48(%r5,%r2) + + aghi %r5,64 + lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */ + aghi %r1,64 + clgrjl %r1,%r4,.Lloop64 + + vl %v16,0(%r5,%r3) /* Load s. */ + j .Llt64 + +.Lfallback: + jg __wcsncpy_c +END(__wcsncpy_vx) + +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcsncpy.c b/sysdeps/s390/multiarch/wcsncpy.c new file mode 100644 index 0000000000..eb225a97b4 --- /dev/null +++ b/sysdeps/s390/multiarch/wcsncpy.c @@ -0,0 +1,28 @@ +/* Multiple versions of wcsncpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc (__wcsncpy) +weak_alias (__wcsncpy, wcsncpy) + +#else +# include <wcsmbs/wcsncpy.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wcsnlen-c.c b/sysdeps/s390/multiarch/wcsnlen-c.c new file mode 100644 index 0000000000..89984e9f18 --- /dev/null +++ b/sysdeps/s390/multiarch/wcsnlen-c.c @@ -0,0 +1,25 @@ +/* Default wcsnlen implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WCSNLEN __wcsnlen_c + +# include <wchar.h> +extern __typeof (__wcsnlen) __wcsnlen_c; +# include <wcsmbs/wcsnlen.c> +#endif diff --git a/sysdeps/s390/multiarch/wcsnlen-vx.S b/sysdeps/s390/multiarch/wcsnlen-vx.S new file mode 100644 index 0000000000..1ba00c3cae --- /dev/null +++ b/sysdeps/s390/multiarch/wcsnlen-vx.S @@ -0,0 +1,151 @@ +/* Vector optimized 32/64 bit S/390 version of wcsnlen. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* size_t wcsnlen (const wchar_t *s, size_t maxlen) + Returns the number of characters in s or at most maxlen. + + Register usage: + -r1=tmp + -r2=address of string + -r3=maxlen (number of characters to be read) + -r4=tmp + -r5=current_len and return_value + -v16=part of s +*/ +ENTRY(__wcsnlen_vx) + + .machine "z13" + .machinemode "zarch_nohighgprs" + +# if !defined __s390x__ + llgfr %r3,%r3 +# endif /* !defined __s390x__ */ + + clgfi %r3,0 /* if maxlen == 0, return 0. */ + locgre %r2,%r3 + ber %r14 + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + llgfr %r1,%r1 /* Convert 32bit to 64bit. */ + + tmll %r2,3 /* Test if s is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + + /* Check range of maxlen and convert to byte-count. */ +# ifdef __s390x__ + tmhh %r3,49152 /* Test bit 0 or 1 of maxlen. */ + lghi %r4,-4 /* Max byte-count is 18446744073709551612. */ +# else + tmlh %r3,49152 /* Test bit 0 or 1 of maxlen. */ + llilf %r4,4294967292 /* Max byte-count is 4294967292. */ +# endif /* !__s390x__ */ + sllg %r3,%r3,2 /* Convert character-count to byte-count. */ + locgrne %r3,%r4 /* Use max byte-count, if bit 0/1 was one. */ + + vfenezf %v16,%v16,%v16 /* Find element not equal with zero search. */ + clgr %r1,%r3 + locgrh %r1,%r3 /* loaded_byte_count + = min (loaded_byte_count, maxlen) */ + + vlgvb %r5,%v16,7 /* Load zero index or 16 if not found. */ + clrjl %r5,%r1,.Lend /* Found zero within loaded bytes -> return. */ + + clgr %r1,%r3 /* If loaded_byte_count == maxlen -> end. */ + locgre %r5,%r3 + je .Lend + + /* Align s to 16 byte. */ + risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,16 /* current_len = 16. */ + slr %r5,%r4 /* Compute bytes to 16bytes boundary. */ + + lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */ + aghi %r1,64 + clgrjl %r1,%r3,.Lloop64 + + /* Find zero in max 64byte with aligned s. */ +.Llt64: + vl %v16,0(%r5,%r2) /* Load s. */ + vfenezfs %v16,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound /* Jump away if zero was found. */ + aghi %r5,16 + clgrjhe %r5,%r3,.Lfound /* If current_len >= maxlen -> end. */ + vl %v16,0(%r5,%r2) + vfenezfs %v16,%v16,%v16 + je .Lfound + aghi %r5,16 + clgrjhe %r5,%r3,.Lfound + vl %v16,0(%r5,%r2) + vfenezfs %v16,%v16,%v16 + je .Lfound + aghi %r5,16 + clgrjhe %r5,%r3,.Lfound + vl %v16,0(%r5,%r2) + vfenezfs %v16,%v16,%v16 + j .Lfound + +.Lfound48: + aghi %r5,16 +.Lfound32: + aghi %r5,16 +.Lfound16: + aghi %r5,16 +.Lfound: + vlgvb %r4,%v16,7 /* Load byte index of zero or 16 if no zero. */ + algr %r5,%r4 + + clgr %r5,%r3 + locgrh %r5,%r3 /* Return min (current_len, maxlen). */ +.Lend: + srlg %r2,%r5,2 /* Convert byte-count to character-count. */ + br %r14 + + /* Find zero in 16byte aligned loop. */ +.Lloop64: + vl %v16,0(%r5,%r2) /* Load s. */ + vfenezfs %v16,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound /* Jump away if zero was found. */ + vl %v16,16(%r5,%r2) + vfenezfs %v16,%v16,%v16 + je .Lfound16 + vl %v16,32(%r5,%r2) + vfenezfs %v16,%v16,%v16 + je .Lfound32 + vl %v16,48(%r5,%r2) + vfenezfs %v16,%v16,%v16 + je .Lfound48 + + aghi %r5,64 + lgr %r1,%r5 /* If %r5 + 64 < maxlen? -> loop64. */ + aghi %r1,64 + clgrjl %r1,%r3,.Lloop64 + + j .Llt64 + +.Lfallback: + jg __wcsnlen_c +END(__wcsnlen_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcsnlen.c b/sysdeps/s390/multiarch/wcsnlen.c new file mode 100644 index 0000000000..4308472a81 --- /dev/null +++ b/sysdeps/s390/multiarch/wcsnlen.c @@ -0,0 +1,28 @@ +/* Multiple versions of wcsnlen. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc (__wcsnlen) +weak_alias (__wcsnlen, wcsnlen) + +#else +# include <wcsmbs/wcsnlen.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wcspbrk-c.c b/sysdeps/s390/multiarch/wcspbrk-c.c new file mode 100644 index 0000000000..8b74eaf017 --- /dev/null +++ b/sysdeps/s390/multiarch/wcspbrk-c.c @@ -0,0 +1,31 @@ +/* Default wcspbrk implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WCSPBRK __wcspbrk_c + +# include <wchar.h> +extern __typeof (wcspbrk) __wcspbrk_c; +# ifdef SHARED +# undef libc_hidden_def +# define libc_hidden_def(name) \ + __hidden_ver1 (__wcspbrk_c, __GI_wcspbrk, __wcspbrk_c); +# endif /* SHARED */ + +# include <wcsmbs/wcspbrk.c> +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcspbrk-vx.S b/sysdeps/s390/multiarch/wcspbrk-vx.S new file mode 100644 index 0000000000..3e28e9aa90 --- /dev/null +++ b/sysdeps/s390/multiarch/wcspbrk-vx.S @@ -0,0 +1,315 @@ +/* Vector optimized 32/64 bit S/390 version of wcspbrk. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* wchar_t *wcspbrk (const wchar_t *s, const wchar_t * accept) + The wcspbrk() function locates the first occurrence in the string s + of any of the characters in the string accept and returns a pointer + to that character or NULL if not found. + + This method checks the length of accept string. If it fits entirely + in one vector register, a fast algorithm is used, which does not need + to check multiple parts of accept-string. Otherwise a slower full + check of accept-string is used. + + register overview: + r3: pointer to start of accept-string + r2: pointer to start of search-string + r0: loaded byte count of vlbb search-string (32bit unsigned) + r4: found byte index (32bit unsigned) + r1: current return len (64bit unsigned) + v16: search-string + v17: accept-string + v18: temp-vreg + + ONLY FOR SLOW: + v19: first accept-string + v20: zero for preparing acc-vector + v21: global mask; 1 indicates a match between + search-string-vreg and any accept-character + v22: current mask; 1 indicates a match between + search-string-vreg and any accept-character in current acc-vreg + v24: one for result-checking of former string-part + v30, v31: for re-/storing registers r6, r8, r9 + r5: current len of accept-string + r6: zero-index in search-string or 16 if no zero + or min(zero-index, loaded byte count) + r8: >0, if former accept-string-part contains a zero, + otherwise =0; + r9: loaded byte count of vlbb accept-string +*/ +ENTRY(__wcspbrk_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + tmll %r2,3 /* Test if s is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + + /* + Check if accept-string fits in one vreg: + ---------------------------------------- + */ + vlbb %v17,0(%r3),6 /* Load accept. */ + lcbb %r0,0(%r3),6 + jo .Lcheck_onbb /* Special case if accept lays + on block-boundary. */ + +.Lcheck_notonbb: + lghi %r1,0 /* Zero out current len. */ + vlgvf %r0,%v17,0 /* Get first element. */ + clije %r0,0,.Lfast_end_null /* Return null if accept is empty. */ + + vistrfs %v17,%v17 /* Fill with zeros after first zero. */ + je .Lfast /* Zero found -> accept fits in one vreg. */ + j .Lslow /* No zero -> accept exceeds one vreg */ + + +.Lcheck_onbb: + /* Accept lays on block-boundary. */ + nill %r0,65532 /* Recognize only fully loaded characters. */ + je .Lcheck_onbb2 /* Reload vr, if we loaded no full wchar_t. */ + vfenezf %v18,%v17,%v17 /* Search zero in loaded accept bytes. */ + vlgvb %r4,%v18,7 /* Get index of zero or 16 if not found. */ + clrjl %r4,%r0,.Lcheck_notonbb /* Zero index < loaded bytes count -> + accept fits in one vreg; + Fill with zeros and proceed + with FAST. */ +.Lcheck_onbb2: + vl %v17,0(%r3) /* Load accept, which exceeds loaded bytes. */ + j .Lcheck_notonbb /* Check if accept fits in one vreg. */ + + + /* + Search s for accept in one vreg + ------------------------------- + */ +.Lfast: + /* Complete accept-string in v17 and remaining bytes are zero. */ + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r0,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + vfaezfs %v18,%v16,%v17,0 /* Find first element in v16 unequal to any + in v17 or first zero element. */ + vlgvb %r4,%v18,7 /* Load byte index of found element. */ + /* If found index is within loaded bytes, return with found + element index (=equal count). */ + clrjl %r4,%r0,.Lfast_loop_found2 + + /* Align s to 16 byte. */ + risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r1,16 /* current_len = 16. */ + slr %r1,%r4 /* Compute bytes to 16bytes boundary. */ + +.Lfast_loop: + vl %v16,0(%r1,%r2) /* Load search-string. */ + vfaezfs %v18,%v16,%v17,0 /* Find first element in v16 equal to any + in v17 or first zero element. */ + jno .Lfast_loop_found + + vl %v16,16(%r1,%r2) + vfaezfs %v18,%v16,%v17,0 + jno .Lfast_loop_found16 + + vl %v16,32(%r1,%r2) + vfaezfs %v18,%v16,%v17,0 + jno .Lfast_loop_found32 + + vl %v16,48(%r1,%r2) + vfaezfs %v18,%v16,%v17,0 + jno .Lfast_loop_found48 + + aghi %r1,64 + j .Lfast_loop /* Loop if no element was unequal to accept + and not zero. */ + + /* Found equal or zero element. */ +.Lfast_loop_found48: + aghi %r1,16 +.Lfast_loop_found32: + aghi %r1,16 +.Lfast_loop_found16: + aghi %r1,16 +.Lfast_loop_found: + vlgvb %r4,%v18,7 /* Load byte index of found element. */ +.Lfast_loop_found2: + srlg %r5,%r4,2 /* Convert byte-index to character-index. */ + vlgvf %r0,%v16,0(%r5) /* Get found element. */ + clije %r0,0,.Lfast_end_null /* Return null if no accept-char found */ + algfr %r1,%r4 /* Add found index of char to current len. */ + la %r2,0(%r1,%r2) /* And return pointer to first equal char. */ + br %r14 + +.Lfast_end_null: + lghi %r2,0 /* Return null if no character is equal. */ + br %r14 + + + + + /* + Search s for accept in multiple vregs + ------------------------------------- + */ +.Lslow: + /* Save registers. */ + vlvgg %v30,%r6,0 + vlvgp %v31,%r8,%r9 + + /* Accept in v17 without zero */ + vlr %v19,%v17 /* Save first acc-part for a fast reload. */ + vzero %v20 /* Zero for preparing acc-vector. */ + vone %v24 /* One for checking result of former string. */ + + /* Align s to 16 byte. */ + risbg %r4,%r2,60,128+63,0 /* Test if s is aligned and + %r4 = bits 60-63 'and' 15. */ + je .Lslow_loop_str /* If s is aligned, loop aligned. */ + lghi %r0,15 + slr %r0,%r4 /* Compute highest index to load (15-x). */ + vll %v16,%r0,0(%r2) /* Load up to 16byte boundary; + needs highest index, left bytes are 0. */ + ahi %r0,1 /* Work with loaded byte count. */ + vzero %v21 /* Zero out global mask. */ + lghi %r5,0 /* Set current len of accept-string to zero. */ + vfenezf %v18,%v16,%v16 /* Find zero in current string-part. */ + lghi %r8,0 /* There is no zero in first accept-part. */ + vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */ + clije %r6,0,.Lslow_end_null /* If first element is zero + (end of string) -> return null */ + clr %r0,%r6 /* cc==1 if loaded byte count < zero-index. */ + locrl %r6,%r0 /* Load on cc==1; zero-index = lbc. */ + j .Lslow_loop_acc + + + /* Process s in 16byte aligned loop. */ +.Lslow_next_str: + /* Check results of former processed str-part. */ + vfeef %v18,%v21,%v24 /* Find first equal match in global mask + (ones in element). */ + vlgvb %r4,%v18,7 /* Get index of first one (=equal) + or 16 if no match. */ + /* Equal-index < min(zero-index, loaded byte count) + -> return pointer to equal element. */ + clrjl %r4,%r6,.Lslow_index_found + /* Zero-index < loaded byte count + -> former str-part was last str-part + -> return null */ + clrjl %r6,%r0,.Lslow_end_null + /* All elements are zero (=no match) -> proceed with next str-part. */ + + vlr %v17,%v19 /* Load first part of accept (no zero). */ + algfr %r1,%r0 /* Add loaded byte count to current len. */ + +.Lslow_loop_str: + vl %v16,0(%r1,%r2) /* Load search-string */ + lghi %r0,16 /* Loaded byte count is 16. */ + vzero %v21 /* Zero out global mask. */ + lghi %r5,0 /* Set current len of accept to zero. */ + vfenezf %v18,%v16,%v16 /* Find zero in current string-part. */ + lghi %r8,0 /* There is no zero in first accept-part. */ + vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */ + clije %r6,0,.Lslow_end_null /* If first element is zero + (end of string) -> return null. */ + +.Lslow_loop_acc: + vfaef %v22,%v16,%v17,4 /* Create matching-mask (1 in mask -> + Character matches any accepted character in + this accept-string-part) IN=0, RT=1. */ + vlgvf %r4,%v22,0 /* Get result of first element. */ + /* First element is equal to any accepted characters + (all other parts of accept cannot lead to a match before this one) + -> current len is pointing to first element + -> return found */ + clijh %r4,0,.Lslow_end_found + vo %v21,%v21,%v22 /* Global-mask = global-|matching-mask. */ + /* Proceed with next acc until end of acc is reached. */ + + +.Lslow_next_acc: + clijh %r8,0,.Lslow_next_str /* There was a zero in the last acc-part + -> add index to current len and + end. */ + vlbb %v17,16(%r5,%r3),6 /* Load next accept part. */ + aghi %r5,16 /* Increment current len of accept-string. */ + lcbb %r9,0(%r5,%r3),6 /* Get loaded byte count of accept-string. */ + jo .Lslow_next_acc_onbb /* Jump away ifaccept-string is + on block-boundary. */ +.Lslow_next_acc_notonbb: + vistrfs %v17,%v17 /* Fill with zeros after first zero. */ + jo .Lslow_loop_acc /* No zero found -> no preparation needed. */ + +.Lslow_next_acc_prepare_zero: + /* Zero in accept-part: fill zeros with first-accept-character. */ + vlgvf %r8,%v17,0 /* Load first element of acc-part. */ + clije %r8,0,.Lslow_next_str /* Proceed with next string-part, + If first char in this part of accept + is a zero. */ + /* r8>0 -> zero found in this acc-part. */ + vrepf %v18,%v17,0 /* Replicate first char accross all chars. */ + vceqf %v22,%v20,%v17 /* Create a mask (v22) of null chars + by comparing with 0 (v20). */ + vsel %v17,%v18,%v17,%v22 /* Replace null chars with first char. */ + j .Lslow_loop_acc /* Accept part is prepared -> process. */ + +.Lslow_next_acc_onbb: + nill %r9,65532 /* Recognize only fully loaded characters. */ + je .Lslow_next_acc_onbb2 /* Reload vr, if no full wchar_t. */ + vfenezf %v18,%v17,%v17 /* Find zero in loaded bytes of accept part. */ + vlgvb %r8,%v18,7 /* Load byte index of zero. */ + clrjl %r8,%r9,.Lslow_next_acc_notonbb /* Found a zero in loaded bytes + -> Prepare vreg. */ +.Lslow_next_acc_onbb2: + vl %v17,0(%r5,%r3) /* Load over boundary ... */ + lghi %r8,0 /* r8=0 -> no zero in this part of acc, + check for zero is in jump-target. */ + j .Lslow_next_acc_notonbb /* ... and search for zero in + fully loaded vreg again. */ + +.Lslow_end_null: + lghi %r1,0 /* Return null if no character is equal. */ + j .Lslow_end + +.Lslow_loop_found: + vlgvb %r4,%v18,7 /* Load byte index of found element. */ + srlg %r5,%r4,2 /* Convert byte-index to character-index. */ + vlgvf %r0,%v16,0(%r5) /* Get found element. */ + clije %r0,0,.Lslow_end_null /* Return null if no acc-char found. */ + +.Lslow_index_found: + algfr %r1,%r4 /* Add found index of char to current len. */ +.Lslow_end_found: + la %r1,0(%r1,%r2) /* And return pointer to first equal char. */ + +.Lslow_end: + /* Restore registers. */ + vlgvg %r6,%v30,0 + vlgvg %r8,%v31,0 + vlgvg %r9,%v31,1 + lgr %r2,%r1 + br %r14 +.Lfallback: + jg __wcspbrk_c +END(__wcspbrk_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcspbrk.c b/sysdeps/s390/multiarch/wcspbrk.c new file mode 100644 index 0000000000..198144d2c5 --- /dev/null +++ b/sysdeps/s390/multiarch/wcspbrk.c @@ -0,0 +1,27 @@ +/* Multiple versions of wcspbrk. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__wcspbrk, wcspbrk) + +#else +# include <wcsmbs/wcspbrk.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wcsrchr-c.c b/sysdeps/s390/multiarch/wcsrchr-c.c new file mode 100644 index 0000000000..eac588b79e --- /dev/null +++ b/sysdeps/s390/multiarch/wcsrchr-c.c @@ -0,0 +1,25 @@ +/* Default wcsrchr implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WCSRCHR __wcsrchr_c + +# include <wchar.h> +extern __typeof (wcsrchr) __wcsrchr_c; +# include <wcsmbs/wcsrchr.c> +#endif diff --git a/sysdeps/s390/multiarch/wcsrchr-vx.S b/sysdeps/s390/multiarch/wcsrchr-vx.S new file mode 100644 index 0000000000..0b99edc7a5 --- /dev/null +++ b/sysdeps/s390/multiarch/wcsrchr-vx.S @@ -0,0 +1,190 @@ +/* Vector optimized 32/64 bit S/390 version of wcsrchr. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* wchar_t *wcsrchr (const wchar_t *s, wchar_t c) + Locate the last character c in string. + + Register usage: + -r0=loaded bytes in first part of s. + -r1=pointer to last occurence of c or NULL if not found. + -r2=s + -r3=c + -r4=tmp + -r5=current_len + -v16=part of s + -v17=index of found element + -v18=replicated c + -v19=part of s with last occurence of c. + -v20=permute pattern +*/ +ENTRY(__wcsrchr_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r0,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + tmll %r2,3 /* Test if s is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + + vlvgf %v18,%r3,0 /* Generate vector which elements are all c. */ + vrepf %v18,%v18,0 + + lghi %r1,-1 /* Currently no c found. */ + lghi %r5,0 /* current_len = 0. */ + + vfeezfs %v17,%v16,%v18 /* Find element equal or zero. */ + vlgvb %r4,%v17,7 /* Load byte index of c/zero or 16. */ + clrjl %r4,%r0,.Lfound_first_part /* Found c/zero in loaded bytes. */ +.Lalign: + /* Align s to 16 byte. */ + risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,16 /* current_len = 16. */ + slr %r5,%r4 /* Compute bytes to 16bytes boundary. */ + +.Lloop: + vl %v16,0(%r5,%r2) /* Load s. */ + vfeezfs %v17,%v16,%v18 /* Find element equal with zero search. */ + jno .Lfound /* Found c/zero (cc=0|1|2). */ + vl %v16,16(%r5,%r2) + vfeezfs %v17,%v16,%v18 + jno .Lfound16 + vl %v16,32(%r5,%r2) + vfeezfs %v17,%v16,%v18 + jno .Lfound32 + vl %v16,48(%r5,%r2) + vfeezfs %v17,%v16,%v18 + jno .Lfound48 + + aghi %r5,64 + j .Lloop /* No character and no zero -> loop. */ + +.Lfound48: + la %r5,16(%r5) /* Use la since aghi would clobber cc. */ +.Lfound32: + la %r5,16(%r5) +.Lfound16: + la %r5,16(%r5) +.Lfound: + je .Lzero /* Found zero, but no c before that zero. */ + /* Save this part of s to check for further matches after reaching + the end of the complete string. */ + vlr %v19,%v16 + lgr %r1,%r5 + + jh .Lzero /* Found a zero after the found c. */ + aghi %r5,16 /* Start search of next part of s. */ + j .Lloop + +.Lfound_first_part: + /* This code is only executed if the found c/zero is whithin loaded + bytes. If no c/zero was found (cc==3) the found index = 16, thus + this code is not called. + Resulting condition code of vector find element equal: + cc==0: no c, found zero + cc==1: c found, no zero + cc==2: c found, found zero after c + cc==3: no c, no zero (this case can be ignored). */ + je .Lzero /* Found zero, but no c before that zero. */ + + locgrne %r1,%r5 /* Mark c as found in first part of s. */ + vlr %v19,%v16 + + jl .Lalign /* No zero (e.g. if vr was fully loaded) + -> Align and loop afterwards. */ + + /* Found a zero in vr. If vr was not fully loaded due to block + boundary, the remaining bytes are filled with zero and we can't + rely on zero indication of condition code here! */ + + vfenezf %v17,%v16,%v16 + vlgvb %r4,%v17,7 /* Load byte index of zero or 16. */ + clrjl %r4,%r0,.Lzero /* Zero within loaded bytes -> end. */ + j .Lalign /* Align and loop afterwards. */ + +.Lend_searched_zero: + vlgvb %r4,%v17,7 /* Load byte index of zero. */ + algr %r5,%r4 + la %r2,0(%r5,%r2) /* Return pointer to zero. */ + br %r14 + +.Lzero: + /* Reached end of string. Check if one c was found before. */ + clije %r3,0,.Lend_searched_zero /* Found zero and c is zero. */ + + cgfi %r1,-1 /* No c found -> return NULL. */ + locghie %r2,0 + ber %r14 + + larl %r3,.Lpermute_mask /* Load permute mask. */ + vl %v20,0(%r3) + + /* c was found and is part of v19. */ + vfenezf %v17,%v19,%v19 /* Find zero. */ + vlgvb %r4,%v17,7 /* Load byte index of zero or 16. */ + ahi %r4,3 /* Found zero index is first byte, + thus highest byte index is last byte of + wchar_t zero. */ + + clgfi %r5,0 /* Loaded byte count in v19 is 16, ... */ + lochine %r0,16 /* ... if v19 is not the first part of s. */ + ahi %r0,-1 /* Convert byte count to highest index. */ + + clr %r0,%r4 + locrl %r4,%r0 /* r4 = min (zero-index, highest-index). */ + + /* Right-shift of v19 to mask bytes after zero. */ + clije %r4,15,.Lzero_permute /* No shift is needed if highest index + in vr is 15. */ + lhi %r0,15 + slr %r0,%r4 /* Compute byte count for vector shift left. */ + sll %r0,3 /* Convert to bit count. */ + vlvgb %v17,%r0,7 + vsrlb %v19,%v19,%v17 /* Vector shift right by byte by number of bytes + specified in bits 1-4 of byte 7 in v17. */ + + /* Reverse bytes in v19. */ +.Lzero_permute: + vperm %v19,%v19,%v19,%v20 /* Permute v19 to reversed order. */ + + /* Find c in reversed v19. */ + vfeef %v19,%v19,%v18 /* Find c. */ + la %r2,0(%r1,%r2) + vlgvb %r3,%v19,7 /* Load byte index of c. */ + + /* Compute index in real s and return. */ + slgr %r4,%r3 + lay %r2,-3(%r4,%r2) /* Return pointer to zero. -3 is needed, + because the found byte index is reversed in + vector-register. Thus point to first byte of + wchar_t. */ + br %r14 +.Lpermute_mask: + .byte 0x0C,0x0D,0x0E,0x0F,0x08,0x09,0x0A,0x0B + .byte 0x04,0x05,0x06,0x07,0x00,0x01,0x02,0x03 +.Lfallback: + jg __wcsrchr_c +END(__wcsrchr_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcsrchr.c b/sysdeps/s390/multiarch/wcsrchr.c new file mode 100644 index 0000000000..9281e12898 --- /dev/null +++ b/sysdeps/s390/multiarch/wcsrchr.c @@ -0,0 +1,27 @@ +/* Multiple versions of wcsrchr. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__wcsrchr, wcsrchr) + +#else +# include <wcsmbs/wcsrchr.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wcsspn-c.c b/sysdeps/s390/multiarch/wcsspn-c.c new file mode 100644 index 0000000000..54c2698bd8 --- /dev/null +++ b/sysdeps/s390/multiarch/wcsspn-c.c @@ -0,0 +1,31 @@ +/* Default wcsspn implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WCSSPN __wcsspn_c + +# include <wchar.h> +extern __typeof (wcsspn) __wcsspn_c; +# ifdef SHARED +# undef libc_hidden_def +# define libc_hidden_def(name) \ + __hidden_ver1 (__wcsspn_c, __GI_wcsspn, __wcsspn_c); +# endif /* SHARED */ + +# include <wcsmbs/wcsspn.c> +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcsspn-vx.S b/sysdeps/s390/multiarch/wcsspn-vx.S new file mode 100644 index 0000000000..e1785ea7cf --- /dev/null +++ b/sysdeps/s390/multiarch/wcsspn-vx.S @@ -0,0 +1,270 @@ +/* Vector optimized 32/64 bit S/390 version of wcsspn. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* size_t wcsspn (const wchar_t *s, const wchar_t * accept) + The wcsspn() function calculates the length of the initial segment + of s which consists entirely of characters in accept. + + This method checks the length of accept string. If it fits entirely + in one vector register, a fast algorithm is used, which does not need + to check multiple parts of accept-string. Otherwise a slower full + check of accept-string is used. + + register overview: + r3: pointer to start of accept-string + r2: pointer to start of search-string + r4: loaded byte count of vl search-string + r0: found byte index + r1: current return len of s + v16: search-string + v17: accept-string + v18: temp-vreg + + ONLY FOR SLOW: + v19: first accept-string + v20: zero for preparing acc-vector + v21: global mask; 1 indicates a match between + search-string-vreg and any accept-character + v22: current mask; 1 indicates a match between + search-string-vreg and any accept-character in current acc-vreg + v30, v31: for re-/storing registers r6, r8, r9 + r5: current len of accept-string + r6: zero-index in search-string or 16 if no zero + or min(zero-index, loaded byte count) + r8: >0, if former accept-string-part contains a zero, + otherwise =0; + r9: loaded byte count of vlbb accept-string +*/ +ENTRY(__wcsspn_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + + tmll %r2,3 /* Test if s is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + + /* + Check if accept-string fits in one vreg: + ---------------------------------------- + */ + vlbb %v17,0(%r3),6 /* Load accept. */ + lcbb %r4,0(%r3),6 + jo .Lcheck_onbb /* Special case if accept lays + on block-boundary. */ +.Lcheck_notonbb: + vistrfs %v17,%v17 /* Fill with zeros after first zero. */ + je .Lfast /* Zero found -> accept fits in one vreg. */ + j .Lslow /* No zero -> accept exceeds one vreg. */ + +.Lcheck_onbb: + /* Accept lays on block-boundary. */ + nill %r4,65532 /* Recognize only fully loaded characters. */ + je .Lcheck_onbb2 /* Reload vr if no full wchar_t. */ + vfenezf %v18,%v17,%v17 /* Search zero in loaded accept bytes. */ + vlgvb %r0,%v18,7 /* Get index of zero or 16 if not found. */ + clrjl %r0,%r4,.Lcheck_notonbb /* Zero index < loaded bytes count -> + Accept fits in one vreg; + Fill with zeros and proceed + with FAST. */ +.Lcheck_onbb2: + vl %v17,0(%r3) /* Load accept, which exceeds loaded bytes. */ + j .Lcheck_notonbb /* Check if accept fits in one vreg. */ + + + /* + Search s for accept in one vreg + ------------------------------- + */ +.Lfast: + /* Complete accept-string in v17 and remaining bytes are zero. */ + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r1,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + + vfaezfs %v16,%v16,%v17,8 /* Find first element in v16 + unequal to any in v17 + or first zero element. */ + + vlgvb %r0,%v16,7 /* Load byte index of found element. */ + /* If found index is within loaded bytes (%r0 < %r1), + return with found element index (=equal count). */ + clr %r0,%r1 + srlg %r0,%r0,2 /* Convert byte-count to character-count. */ + locgrl %r2,%r0 + blr %r14 + + /* Align s to 16 byte. */ + risbgn %r4,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r1,16 /* current_len = 16. */ + slr %r1,%r4 /* Compute bytes to 16bytes boundary. */ + +.Lfast_loop: + vl %v16,0(%r1,%r2) /* Load search-string. */ + vfaezfs %v16,%v16,%v17,8 /* Find first element in v16 + unequal to any in v17 + or first zero element. */ + jno .Lfast_loop_found + vl %v16,16(%r1,%r2) + vfaezfs %v16,%v16,%v17,8 + jno .Lfast_loop_found16 + vl %v16,32(%r1,%r2) + vfaezfs %v16,%v16,%v17,8 + jno .Lfast_loop_found32 + vl %v16,48(%r1,%r2) + vfaezfs %v16,%v16,%v17,8 + jno .Lfast_loop_found48 + + aghi %r1,64 + j .Lfast_loop /* Loop if no element was unequal to accept + and not zero. */ + + /* Found unequal or zero element. */ +.Lfast_loop_found48: + aghi %r1,16 +.Lfast_loop_found32: + aghi %r1,16 +.Lfast_loop_found16: + aghi %r1,16 +.Lfast_loop_found: + vlgvb %r0,%v16,7 /* Load byte index of found element. */ + algrk %r2,%r1,%r0 /* And add it to current len. */ + srlg %r2,%r2,2 /* Convert byte-count to character-count. */ + br %r14 + + + /* + Search s for accept in multiple vregs + ------------------------------------- + */ +.Lslow: + /* Save registers. */ + vlvgg %v30,%r6,0 + vlvgp %v31,%r8,%r9 + lghi %r1,0 /* Zero out current len. */ + + /* accept in v17 without zero. */ + vlr %v19,%v17 /* Save first acc-part for a fast reload. */ + vzero %v20 /* Zero for preparing acc-vector. */ + + /* Align s to 16 byte. */ + risbg %r0,%r2,60,128+63,0 /* Test if s is aligned and + %r0 = bits 60-63 'and' 15. */ + je .Lslow_loop_str /* If s is aligned, loop aligned */ + lghi %r4,15 + slr %r4,%r0 /* Compute highest index to load (15-x). */ + vll %v16,%r4,0(%r2) /* Load up to 16byte boundary (vll needs + highest index, remaining bytes are 0). */ + aghi %r4,1 /* Work with loaded byte count. */ + vzero %v21 /* Zero out global mask. */ + lghi %r5,0 /* Set current len of accept-string to zero. */ + vfenezf %v18,%v16,%v16 /* Find zero in current string-part. */ + lghi %r8,0 /* There is no zero in first accept-part. */ + vlgvb %r6,%v18,7 /* Load byte index of zero or 16 + if there is no zero. */ + clr %r4,%r6 /* cc==1 if loaded byte count < zero-index. */ + locrl %r6,%r4 /* Load on cc==1. */ + j .Lslow_loop_acc + + /* Process s in 16byte aligned loop. */ +.Lslow_next_str: + vlr %v17,%v19 /* Load first part of accept (no zero). */ + algfr %r1,%r4 /* Add loaded byte count to current len. */ +.Lslow_loop_str: + vl %v16,0(%r1,%r2) /* Load search-string. */ + lghi %r4,16 /* Loaded byte count is 16. */ + vzero %v21 /* Zero out global mask. */ + lghi %r5,0 /* Set current len of accept-string to zero. */ + vfenezf %v18,%v16,%v16 /* Find zero in current string-part. */ + lghi %r8,0 /* There is no zero in first accept-part. */ + vlgvb %r6,%v18,7 /* Load byte index of zero or 16 if no zero. */ + +.Lslow_loop_acc: + vfaef %v22,%v16,%v17,4 /* Create matching-mask (1 in mask -> + character matches any accepted character in + this accept-string-part) IN=0, RT=1. */ + vo %v21,%v21,%v22 /* global-mask = global- | matching-mask. */ + vfenezf %v18,%v21,%v21 /* Find first zero in global-mask. */ + vlgvb %r0,%v18,7 /* Get first found zero-index + (= first mismatch). */ + clrjl %r0,%r6,.Lslow_next_acc /* Mismatch-index < min(lbc,zero-index) + -> Process this string-part + with next acc-part. */ + clrjhe %r0,%r4,.Lslow_next_str /* Found-index >= loaded byte count + -> All loaded bytes are matching + any accept-character + and are not zero. */ + /* All bytes are matching any characters in accept-string + and search-string is fully processed (found-index == zero-index). */ +.Lslow_add_lbc_end: + algrk %r2,%r1,%r0 /* Add matching characters to current len. */ + srlg %r2,%r2,2 /* Convert byte-count to character-count. */ + /* Restore registers. */ + vlgvg %r6,%v30,0 + vlgvg %r8,%v31,0 + vlgvg %r9,%v31,1 + br %r14 + +.Lslow_next_acc: + clijh %r8,0,.Lslow_add_lbc_end /* There was a zero in last acc-part + -> Add found index to current len + and end. */ + vlbb %v17,16(%r5,%r3),6 /* Load next accept part. */ + aghi %r5,16 /* Increment current len of accept-string. */ + lcbb %r9,0(%r5,%r3),6 /* Get loaded byte count of accept-string. */ + jo .Lslow_next_acc_onbb /* Jump away if accept-string is + on block-boundary. */ +.Lslow_next_acc_notonbb: + vistrfs %v17,%v17 /* Fill with zeros after first zero. */ + jo .Lslow_loop_acc /* No zero found -> no preparation needed. */ + +.Lslow_next_acc_prepare_zero: + /* Zero in accept-part: fill zeros with first-accept-character. */ + vlgvf %r8,%v17,0 /* Load first element of acc-part. */ + clije %r8,0,.Lslow_add_lbc_end /* End if zero is first character + in this part of accept-string. */ + /* r8>0 -> zero found in this acc-part. */ + vrepf %v18,%v17,0 /* Replicate first char accross all chars. */ + vceqf %v22,%v20,%v17 /* Create a mask (v22) of null chars + by comparing with 0 (v20). */ + vsel %v17,%v18,%v17,%v22 /* Replace null chars with first char. */ + j .Lslow_loop_acc /* Accept part is prepared -> process. */ + +.Lslow_next_acc_onbb: + nill %r9,65532 /* Recognize only fully loaded characters. */ + je .Lslow_next_acc_onbb2 /* Reload vr, if we loaded no full + wchar_t. */ + vfenezf %v18,%v17,%v17 /* Find zero in loaded bytes of accept part. */ + vlgvb %r8,%v18,7 /* Load byte index of zero. */ + clrjl %r8,%r9,.Lslow_next_acc_notonbb /* Found a zero in loaded bytes + -> Prepare vreg. */ +.Lslow_next_acc_onbb2: + vl %v17,0(%r5,%r3) /* Load over boundary ... */ + lghi %r8,0 /* r8=0 -> no zero in this part of acc, + check for zero is in jump-target. */ + j .Lslow_next_acc_notonbb /* ... and search for zero in + fully loaded vreg again. */ +.Lfallback: + jg __wcsspn_c +END(__wcsspn_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcsspn.c b/sysdeps/s390/multiarch/wcsspn.c new file mode 100644 index 0000000000..167a881d13 --- /dev/null +++ b/sysdeps/s390/multiarch/wcsspn.c @@ -0,0 +1,27 @@ +/* Multiple versions of wcsspn. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__wcsspn, wcsspn) + +#else +# include <wcsmbs/wcsspn.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wmemchr-c.c b/sysdeps/s390/multiarch/wmemchr-c.c new file mode 100644 index 0000000000..32dddc6c3d --- /dev/null +++ b/sysdeps/s390/multiarch/wmemchr-c.c @@ -0,0 +1,37 @@ +/* Default wmemchr implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WMEMCHR __wmemchr_c + +# include <wchar.h> +extern __typeof (wmemchr) __wmemchr_c; +# undef weak_alias +# define weak_alias(name, alias) +# ifdef SHARED +# undef libc_hidden_def +# define libc_hidden_def(name) \ + __hidden_ver1 (__wmemchr_c, __GI___wmemchr, __wmemchr_c); +# undef libc_hidden_weak +# define libc_hidden_weak(name) \ + strong_alias (__wmemchr_c, __wmemchr_c_1); \ + __hidden_ver1 (__wmemchr_c_1, __GI_wmemchr, __wmemchr_c_1); +# endif /* SHARED */ + +# include <wcsmbs/wmemchr.c> +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wmemchr-vx.S b/sysdeps/s390/multiarch/wmemchr-vx.S new file mode 100644 index 0000000000..a729681341 --- /dev/null +++ b/sysdeps/s390/multiarch/wmemchr-vx.S @@ -0,0 +1,166 @@ +/* Vector optimized 32/64 bit S/390 version of wmemchr. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* wchar_t *wmemchr (const wchar_t *s, wchar_t c, size_t n) + Scans memory for character c + and returns pointer to first c. + + Register usage: + -r0=tmp + -r1=tmp + -r2=s + -r3=c + -r4=n + -r5=current_len + -v16=part of s + -v17=index of found c + -v18=c replicated +*/ +ENTRY(__wmemchr_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + +# if !defined __s390x__ + llgfr %r4,%r4 +# endif /* !defined __s390x__ */ + + clgije %r4,0,.Lnf_end /* If len == 0 then exit. */ + + vlbb %v16,0(%r2),6 /* Load s until next 4k-byte boundary. */ + lcbb %r0,0(%r2),6 /* Get bytes to 4k-byte boundary or 16. */ + llgfr %r0,%r0 /* Convert 32bit to 64bit. */ + + tmll %r2,3 /* Test if s is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + + vlvgf %v18,%r3,0 /* Generate vector which elements are all c. */ + vrepf %v18,%v18,0 + lghi %r5,16 /* current_len = 16. */ + + /* Check range of maxlen and convert to byte-count. */ +# ifdef __s390x__ + tmhh %r4,49152 /* Test bit 0 or 1 of maxlen. */ + lghi %r1,-4 /* Max byte-count is 18446744073709551612. */ +# else + tmlh %r4,49152 /* Test bit 0 or 1 of maxlen. */ + llilf %r1,4294967292 /* Max byte-count is 4294967292. */ +# endif /* !__s390x__ */ + sllg %r4,%r4,2 /* Convert character-count to byte-count. */ + locgrne %r4,%r1 /* Use max byte-count, if bit 0/1 was one. */ + + clgrjhe %r0,%r4,.Llastcmp /* If (bytes to boundary) >= n, + jump to lastcmp. */ + + vfeefs %v17,%v16,%v18 /* Find c. */ + vlgvb %r1,%v17,7 /* Load byte index of c. */ + clgrjl %r1,%r0,.Lfound2 /* Found c is within loaded bytes. */ + + /* Align s to 16 byte. */ + risbgn %r1,%r2,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + slr %r5,%r1 /* Compute bytes to 16bytes boundary. */ + + lgr %r0,%r5 /* If %r5 + 64 < n? -> loop64. */ + aghi %r0,64 + clgrjl %r0,%r4,.Lloop64 +.Llt64: + vl %v16,0(%r5,%r2) + aghi %r5,16 + clgrjhe %r5,%r4,.Llastcmp /* Do last compare if curr-len >= n. */ + vfeefs %v17,%v16,%v18 /* Find c. */ + jl .Lfound /* Jump away if c was found. */ + + vl %v16,0(%r5,%r2) + aghi %r5,16 + clgrjhe %r5,%r4,.Llastcmp + vfeefs %v17,%v16,%v18 + jl .Lfound + + vl %v16,0(%r5,%r2) + aghi %r5,16 + clgrjhe %r5,%r4,.Llastcmp + vfeefs %v17,%v16,%v18 + jl .Lfound + + vl %v16,0(%r5,%r2) + aghi %r5,16 + +.Llastcmp: + /* Use comparision result only if located within first n characters. + %r5: current_len; + %r4: n; + (current_len - n): [0...16[ + first ignored match index = vr-width - (current_len - n) ]0...16] + */ + vfeefs %v17,%v16,%v18 /* Find c. */ + slgrk %r4,%r5,%r4 /* %r5 = current_len - n. */ + lghi %r0,16 /* Register width = 16. */ + vlgvb %r1,%v17,7 /* Extract found index or 16 if all equal. */ + slr %r0,%r4 /* %r0 = first ignored match index. */ + clrjl %r1,%r0,.Lfound2 /* Go away if miscompare is below n bytes. */ + /* c not found within n-bytes. */ +.Lnf_end: + lghi %r2,0 /* Return null. */ + br %r14 + +.Lfound48: + aghi %r5,16 +.Lfound32: + aghi %r5,16 +.Lfound16: + aghi %r5,16 +.Lfound0: + aghi %r5,16 +.Lfound: + vlgvb %r1,%v17,7 /* Load byte index of c. */ +.Lfound2: + slgfi %r5,16 /* current_len -=16 */ + algr %r5,%r1 /* Zero byte index is added to current len. */ + la %r2,0(%r5,%r2) /* Return pointer to c. */ + br %r14 + +.Lloop64: + vl %v16,0(%r5,%r2) + vfeefs %v17,%v16,%v18 /* Find c. */ + jl .Lfound0 /* Jump away if c was found. */ + vl %v16,16(%r5,%r2) + vfeefs %v17,%v16,%v18 + jl .Lfound16 + vl %v16,32(%r5,%r2) + vfeefs %v17,%v16,%v18 + jl .Lfound32 + vl %v16,48(%r5,%r2) + vfeefs %v17,%v16,%v18 + jl .Lfound48 + + aghi %r5,64 + lgr %r0,%r5 /* If %r5 + 64 < n? -> loop64. */ + aghi %r0,64 + clgrjl %r0,%r4,.Lloop64 + + j .Llt64 +.Lfallback: + jg __wmemchr_c +END(__wmemchr_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wmemchr.c b/sysdeps/s390/multiarch/wmemchr.c new file mode 100644 index 0000000000..f2bfe3c7a5 --- /dev/null +++ b/sysdeps/s390/multiarch/wmemchr.c @@ -0,0 +1,29 @@ +/* Multiple versions of wmemchr. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc (__wmemchr) +weak_alias (__wmemchr, wmemchr) +libc_hidden_weak (wmemchr) + +#else +# include <wcsmbs/wmemchr.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wmemcmp-c.c b/sysdeps/s390/multiarch/wmemcmp-c.c new file mode 100644 index 0000000000..683385431e --- /dev/null +++ b/sysdeps/s390/multiarch/wmemcmp-c.c @@ -0,0 +1,26 @@ +/* Default wmemcmp implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WMEMCMP __wmemcmp_c + +# include <wchar.h> +extern __typeof (wmemcmp) __wmemcmp_c; + +# include <wcsmbs/wmemcmp.c> +#endif diff --git a/sysdeps/s390/multiarch/wmemcmp-vx.S b/sysdeps/s390/multiarch/wmemcmp-vx.S new file mode 100644 index 0000000000..761cc17771 --- /dev/null +++ b/sysdeps/s390/multiarch/wmemcmp-vx.S @@ -0,0 +1,149 @@ +/* Vector Optimized 32/64 bit S/390 version of wmemcmp. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* int wmemcmp (const wchar_t *s1, const wchar_t *s2, size_t n) + Compare at most n characters of two wchar_t-arrays. + + Register usage: + -r0=tmp + -r1=number of blocks + -r2=s1 + -r3=s2 + -r4=n + -r5=current_len + -v16=part of s1 + -v17=part of s2 + -v18=index of unequal +*/ +ENTRY(__wmemcmp_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + +# if !defined __s390x__ + llgfr %r4,%r4 +# endif /* !defined __s390x__ */ + clgije %r4,0,.Lend_equal /* Nothing to do if n == 0. */ + + /* Check range of maxlen and convert to byte-count. */ +# ifdef __s390x__ + tmhh %r4,49152 /* Test bit 0 or 1 of maxlen. */ + lghi %r1,-4 /* Max byte-count is 18446744073709551612. */ +# else + tmlh %r4,49152 /* Test bit 0 or 1 of maxlen. */ + llilf %r1,4294967292 /* Max byte-count is 4294967292. */ +# endif /* !__s390x__ */ + sllg %r4,%r4,2 /* Convert character-count to byte-count. */ + locgrne %r4,%r1 /* Use max byte-count, if bit 0/1 was one. */ + + lghi %r5,0 /* current_len = 0. */ + + clgijh %r4,16,.Lgt16 + +.Lremaining: + aghi %r4,-1 /* vstl needs highest index. */ + vll %v16,%r4,0(%r2) + vll %v17,%r4,0(%r3) + vfenef %v18,%v16,%v17 /* Compare not equal. */ + vlgvb %r1,%v18,7 /* Load unequal index or 16 if not found. */ + clrj %r1,%r4,12,.Lfound2 /* r1 <= r4 -> unequal within loaded + bytes. */ + +.Lend_equal: + lghi %r2,0 + br %r14 + +.Lfound: + /* vfenezf found an unequal element or zero. + This instruction compares unsigned words, but wchar_t is signed. + Thus we have to compare the found element again. */ + vlgvb %r1,%v18,7 /* Extract not equal byte-index. */ +.Lfound2: + srl %r1,2 /* And convert it to character-index. */ + vlgvf %r0,%v16,0(%r1) /* Load character-values. */ + vlgvf %r1,%v17,0(%r1) + cr %r0,%r1 + je .Lend_equal + lghi %r2,1 + lghi %r1,-1 + locgrl %r2,%r1 + br %r14 + +.Lgt16: + clgijh %r4,64,.Lpreloop64 + +.Lpreloop16: + srlg %r1,%r4,4 /* Split into 16byte blocks */ +.Lloop16: + vl %v16,0(%r5,%r2) + vl %v17,0(%r5,%r3) + aghi %r5,16 + vfenefs %v18,%v16,%v17 /* Compare not equal. */ + jno .Lfound + brctg %r1,.Lloop16 /* Loop until all blocks are processed. */ + + llgfr %r4,%r4 + nilf %r4,15 /* Get remaining bytes */ + locgre %r2,%r4 + ber %r14 + la %r2,0(%r5,%r2) + la %r3,0(%r5,%r3) + j .Lremaining + +.Lpreloop64: + srlg %r1,%r4,6 /* Split into 64byte blocks */ +.Lloop64: + vl %v16,0(%r5,%r2) + vl %v17,0(%r5,%r3) + vfenefs %v18,%v16,%v17 /* Compare not equal. */ + jno .Lfound + + vl %v16,16(%r5,%r2) + vl %v17,16(%r5,%r3) + vfenefs %v18,%v16,%v17 + jno .Lfound + + vl %v16,32(%r5,%r2) + vl %v17,32(%r5,%r3) + vfenefs %v18,%v16,%v17 + jno .Lfound + + vl %v16,48(%r5,%r2) + vl %v17,48(%r5,%r3) + aghi %r5,64 + vfenefs %v18,%v16,%v17 + jno .Lfound + + brctg %r1,.Lloop64 /* Loop until all blocks are processed. */ + + llgfr %r4,%r4 + nilf %r4,63 /* Get remaining bytes */ + locgre %r2,%r4 + ber %r14 + clgijh %r4,16,.Lpreloop16 + la %r2,0(%r5,%r2) + la %r3,0(%r5,%r3) + j .Lremaining +END(__wmemcmp_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wmemcmp.c b/sysdeps/s390/multiarch/wmemcmp.c new file mode 100644 index 0000000000..95106fcaf9 --- /dev/null +++ b/sysdeps/s390/multiarch/wmemcmp.c @@ -0,0 +1,27 @@ +/* Multiple versions of wmemcmp. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc2 (__wmemcmp, wmemcmp) + +#else +# include <wcsmbs/wmemcmp.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wmemset-c.c b/sysdeps/s390/multiarch/wmemset-c.c new file mode 100644 index 0000000000..61ccd8fc09 --- /dev/null +++ b/sysdeps/s390/multiarch/wmemset-c.c @@ -0,0 +1,37 @@ +/* Default wmemset implementation for S/390. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WMEMSET __wmemset_c + +# include <wchar.h> +extern __typeof (__wmemset) __wmemset_c; +# undef weak_alias +# define weak_alias(name, alias) +# ifdef SHARED +# undef libc_hidden_def +# define libc_hidden_def(name) \ + __hidden_ver1 (__wmemset_c, __GI___wmemset, __wmemset_c); +# undef libc_hidden_weak +# define libc_hidden_weak(name) \ + strong_alias (__wmemset_c, __wmemset_c_1); \ + __hidden_ver1 (__wmemset_c_1, __GI_wmemset, __wmemset_c_1); +# endif /* SHARED */ + +# include <wcsmbs/wmemset.c> +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wmemset-vx.S b/sysdeps/s390/multiarch/wmemset-vx.S new file mode 100644 index 0000000000..7a28bb4ca6 --- /dev/null +++ b/sysdeps/s390/multiarch/wmemset-vx.S @@ -0,0 +1,142 @@ +/* Vector Optimized 32/64 bit S/390 version of wmemset. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* wchar_t *wmemset(wchar_t *dest, wchar_t wc, size_t n) + Fill an array of wide-characters with a constant wide character + and returns dest. + + Register usage: + -r0=tmp + -r1=tmp + -r2=dest or current-pointer + -r3=wc + -r4=n + -r5=tmp + -v16=replicated wc + -v17,v18,v19=copy of v16 for vstm + -v31=saved dest for return +*/ +ENTRY(__wmemset_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + +# if !defined __s390x__ + llgfr %r4,%r4 +# endif /* !defined __s390x__ */ + + vlvgg %v31,%r2,0 /* Save destination pointer for return. */ + clgije %r4,0,.Lend + + vlvgf %v16,%r3,0 /* Generate vector with wchar_t wc. */ + vrepf %v16,%v16,0 + + /* Check range of maxlen and convert to byte-count. */ +# ifdef __s390x__ + tmhh %r4,49152 /* Test bit 0 or 1 of maxlen. */ + lghi %r5,-4 /* Max byte-count is 18446744073709551612. */ +# else + tmlh %r4,49152 /* Test bit 0 or 1 of maxlen. */ + llilf %r5,4294967292 /* Max byte-count is 4294967292. */ +# endif /* !__s390x__ */ + sllg %r4,%r4,2 /* Convert character-count to byte-count. */ + locgrne %r4,%r5 /* Use max byte-count, if bit 0/1 was one. */ + + /* Align dest to 16 byte. */ + risbg %r0,%r2,60,128+63,0 /* Test if s is aligned and + %r3 = bits 60-63 'and' 15. */ + je .Lpreloop /* If s is aligned, loop aligned. */ + tmll %r2,3 /* Test if s is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + lghi %r1,16 + slr %r1,%r0 /* Compute byte count to load (16-x). */ + clgr %r1,%r4 + locgrh %r1,%r4 /* min (byte count, n) */ + aghik %r5,%r1,-1 /* vstl needs highest index. */ + vstl %v16,%r5,0(%r2) /* Store remaining bytes. */ + clgrje %r1,%r4,.Lend /* Return if n bytes where set. */ + slgr %r4,%r1 /* Compute remaining byte count. */ + la %r2,0(%r1,%r2) + +.Lpreloop: + /* Now we are 16-byte aligned. */ + clgijl %r4,17,.Lremaining + srlg %r1,%r4,8 /* Split into 256byte blocks */ + clgije %r1,0,.Lpreloop64 + vlr %v17,%v16 + vlr %v18,%v16 + vlr %v19,%v16 + +.Lloop256: + vstm %v16,%v19,0(%r2) + vstm %v16,%v19,64(%r2) + vstm %v16,%v19,128(%r2) + vstm %v16,%v19,192(%r2) + la %r2,256(%r2) + brctg %r1,.Lloop256 /* Loop until all blocks are processed. */ + + llgfr %r4,%r4 + nilf %r4,255 /* Get remaining bytes */ + je .Lend /* Skip store remaining bytes if zero. */ + +.Lpreloop64: + clgijl %r4,17,.Lremaining + clgijl %r4,33,.Lpreloop16 + srlg %r1,%r4,5 /* Split into 32byte blocks */ + +.Lloop32: + vst %v16,0(%r2) + vst %v16,16(%r2) + la %r2,32(%r2) + brctg %r1,.Lloop32 /* Loop until all blocks are processed. */ + + llgfr %r4,%r4 + nilf %r4,31 /* Get remaining bytes */ + je .Lend /* Skip store remaining bytes if zero. */ + +.Lpreloop16: + clgijl %r4,17,.Lremaining + srlg %r1,%r4,4 /* Split into 16byte blocks */ + +.Lloop16: + vst %v16,0(%r2) + la %r2,16(%r2) + brctg %r1,.Lloop16 /* Loop until all blocks are processed. */ + + llgfr %r4,%r4 + nilf %r4,15 /* Get remaining bytes */ + je .Lend /* Skip store remaining bytes if zero. */ + +.Lremaining: + aghi %r4,-1 /* vstl needs highest index. */ + vstl %v16,%r4,0(%r2) + +.Lend: + vlgvg %r2,%v31,0 /* Load saved dest for return value. */ + br %r14 +.Lfallback: + srlg %r4,%r4,2 /* Convert byte-count to character-count. */ + jg __wmemset_c +END(__wmemset_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wmemset.c b/sysdeps/s390/multiarch/wmemset.c new file mode 100644 index 0000000000..e9e695fc0a --- /dev/null +++ b/sysdeps/s390/multiarch/wmemset.c @@ -0,0 +1,29 @@ +/* Multiple versions of wmemset. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include <wchar.h> +# include <ifunc-resolve.h> + +s390_vx_libc_ifunc (__wmemset) +weak_alias (__wmemset, wmemset) +libc_hidden_weak (wmemset) + +#else +# include <wcsmbs/wmemset.c> +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/nptl/Makefile b/sysdeps/s390/nptl/Makefile index 13332608c4..5734b983b0 100644 --- a/sysdeps/s390/nptl/Makefile +++ b/sysdeps/s390/nptl/Makefile @@ -1,4 +1,4 @@ -# Copyright (C) 2003-2015 Free Software Foundation, Inc. +# Copyright (C) 2003-2016 Free Software Foundation, Inc. # This file is part of the GNU C Library. # The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/nptl/bits/pthreadtypes.h b/sysdeps/s390/nptl/bits/pthreadtypes.h index 1f3bb14abe..40d10fea59 100644 --- a/sysdeps/s390/nptl/bits/pthreadtypes.h +++ b/sysdeps/s390/nptl/bits/pthreadtypes.h @@ -1,4 +1,4 @@ -/* Copyright (C) 2003-2015 Free Software Foundation, Inc. +/* Copyright (C) 2003-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/nptl/bits/semaphore.h b/sysdeps/s390/nptl/bits/semaphore.h index 9ae0c7f38d..0d756abc42 100644 --- a/sysdeps/s390/nptl/bits/semaphore.h +++ b/sysdeps/s390/nptl/bits/semaphore.h @@ -1,4 +1,4 @@ -/* Copyright (C) 2003-2015 Free Software Foundation, Inc. +/* Copyright (C) 2003-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Martin Schwidefsky <schwidefsky@de.ibm.com>, 2003. diff --git a/sysdeps/s390/nptl/pthread_spin_init.c b/sysdeps/s390/nptl/pthread_spin_init.c index 902ece1b21..7d3568fd6f 100644 --- a/sysdeps/s390/nptl/pthread_spin_init.c +++ b/sysdeps/s390/nptl/pthread_spin_init.c @@ -1,4 +1,4 @@ -/* Copyright (C) 2003-2015 Free Software Foundation, Inc. +/* Copyright (C) 2003-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Martin Schwidefsky <schwidefsky@de.ibm.com>, 2003. diff --git a/sysdeps/s390/nptl/pthread_spin_lock.c b/sysdeps/s390/nptl/pthread_spin_lock.c index b255b38975..def6a24275 100644 --- a/sysdeps/s390/nptl/pthread_spin_lock.c +++ b/sysdeps/s390/nptl/pthread_spin_lock.c @@ -1,4 +1,4 @@ -/* Copyright (C) 2003-2015 Free Software Foundation, Inc. +/* Copyright (C) 2003-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Martin Schwidefsky <schwidefsky@de.ibm.com>, 2003. @@ -19,15 +19,14 @@ #include "pthreadP.h" int -pthread_spin_lock (lock) - pthread_spinlock_t *lock; +pthread_spin_lock (pthread_spinlock_t *lock) { int oldval; - __asm __volatile ("0: lhi %0,0\n" - " cs %0,%2,%1\n" - " jl 0b" - : "=&d" (oldval), "=Q" (*lock) - : "d" (1), "m" (*lock) : "cc" ); + __asm__ __volatile__ ("0: lhi %0,0\n" + " cs %0,%2,%1\n" + " jl 0b" + : "=&d" (oldval), "=Q" (*lock) + : "d" (1), "m" (*lock) : "cc" ); return 0; } diff --git a/sysdeps/s390/nptl/pthread_spin_trylock.c b/sysdeps/s390/nptl/pthread_spin_trylock.c index 144cc77051..4c00e0833f 100644 --- a/sysdeps/s390/nptl/pthread_spin_trylock.c +++ b/sysdeps/s390/nptl/pthread_spin_trylock.c @@ -1,4 +1,4 @@ -/* Copyright (C) 2003-2015 Free Software Foundation, Inc. +/* Copyright (C) 2003-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Martin Schwidefsky <schwidefsky@de.ibm.com>, 2003. @@ -20,14 +20,13 @@ #include "pthreadP.h" int -pthread_spin_trylock (lock) - pthread_spinlock_t *lock; +pthread_spin_trylock (pthread_spinlock_t *lock) { int old; - __asm __volatile ("cs %0,%3,%1" - : "=d" (old), "=Q" (*lock) - : "0" (0), "d" (1), "m" (*lock) : "cc" ); + __asm__ __volatile__ ("cs %0,%3,%1" + : "=d" (old), "=Q" (*lock) + : "0" (0), "d" (1), "m" (*lock) : "cc" ); return old != 0 ? EBUSY : 0; } diff --git a/sysdeps/s390/nptl/pthread_spin_unlock.c b/sysdeps/s390/nptl/pthread_spin_unlock.c index a34ebccdce..0dcc2d0cb5 100644 --- a/sysdeps/s390/nptl/pthread_spin_unlock.c +++ b/sysdeps/s390/nptl/pthread_spin_unlock.c @@ -1,4 +1,4 @@ -/* Copyright (C) 2003-2015 Free Software Foundation, Inc. +/* Copyright (C) 2003-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Martin Schwidefsky <schwidefsky@de.ibm.com>, 2003. @@ -24,9 +24,9 @@ int pthread_spin_unlock (pthread_spinlock_t *lock) { - __asm __volatile (" xc %O0(4,%R0),%0\n" - " bcr 15,0" - : "=Q" (*lock) : "m" (*lock) : "cc" ); + __asm__ __volatile__ (" xc %O0(4,%R0),%0\n" + " bcr 15,0" + : "=Q" (*lock) : "m" (*lock) : "cc" ); return 0; } strong_alias (pthread_spin_unlock, pthread_spin_init) diff --git a/sysdeps/s390/nptl/pthreaddef.h b/sysdeps/s390/nptl/pthreaddef.h index 80f9a81558..d483f11103 100644 --- a/sysdeps/s390/nptl/pthreaddef.h +++ b/sysdeps/s390/nptl/pthreaddef.h @@ -1,4 +1,4 @@ -/* Copyright (C) 2003-2015 Free Software Foundation, Inc. +/* Copyright (C) 2003-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/nptl/tls.h b/sysdeps/s390/nptl/tls.h index e6f8a47ecc..e4c3ec7830 100644 --- a/sysdeps/s390/nptl/tls.h +++ b/sysdeps/s390/nptl/tls.h @@ -1,5 +1,5 @@ /* Definition for thread-local data handling. NPTL/s390 version. - Copyright (C) 2003-2015 Free Software Foundation, Inc. + Copyright (C) 2003-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -53,7 +53,11 @@ typedef struct int gscope_flag; #ifndef __ASSUME_PRIVATE_FUTEX int private_futex; +#else + int __glibc_reserved1; #endif + /* GCC split stack support. */ + void *__private_ss; } tcbhead_t; # ifndef __s390x__ @@ -159,9 +163,9 @@ typedef struct /* Set the stack guard field in TCB head. */ #define THREAD_SET_STACK_GUARD(value) \ - do \ + do \ { \ - __asm __volatile ("" : : : "a0", "a1"); \ + __asm__ __volatile__ ("" : : : "a0", "a1"); \ THREAD_SETMEM (THREAD_SELF, header.stack_guard, value); \ } \ while (0) diff --git a/sysdeps/s390/s390-32/__longjmp.c b/sysdeps/s390/s390-32/__longjmp.c index b253934083..2631cfd32f 100644 --- a/sysdeps/s390/s390-32/__longjmp.c +++ b/sysdeps/s390/s390-32/__longjmp.c @@ -1,4 +1,4 @@ -/* Copyright (C) 2000-2015 Free Software Foundation, Inc. +/* Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). @@ -37,46 +37,46 @@ __longjmp (__jmp_buf env, int val) #elif defined CHECK_SP CHECK_SP (env, 0); #endif - register int r2 __asm ("%r2") = val == 0 ? 1 : val; + register int r2 __asm__ ("%r2") = val == 0 ? 1 : val; #ifdef PTR_DEMANGLE - register uintptr_t r3 __asm ("%r3") = guard; - register void *r1 __asm ("%r1") = (void *) env; + register uintptr_t r3 __asm__ ("%r3") = guard; + register void *r1 __asm__ ("%r1") = (void *) env; #endif /* Restore registers and jump back. */ - asm volatile ( + __asm__ __volatile__ ( /* longjmp probe expects longjmp first argument, second argument and target address. */ #ifdef PTR_DEMANGLE - "lm %%r4,%%r5,32(%1)\n\t" - "xr %%r4,%2\n\t" - "xr %%r5,%2\n\t" - LIBC_PROBE_ASM (longjmp, 4@%1 -4@%0 4@%%r4) + "lm %%r4,%%r5,32(%1)\n\t" + "xr %%r4,%2\n\t" + "xr %%r5,%2\n\t" + LIBC_PROBE_ASM (longjmp, 4@%1 -4@%0 4@%%r4) #else - LIBC_PROBE_ASM (longjmp, 4@%1 -4@%0 4@%%r14) + LIBC_PROBE_ASM (longjmp, 4@%1 -4@%0 4@%%r14) #endif - /* restore fpregs */ - "ld %%f6,48(%1)\n\t" - "ld %%f4,40(%1)\n\t" + /* restore fpregs */ + "ld %%f6,48(%1)\n\t" + "ld %%f4,40(%1)\n\t" - /* restore gregs and return to jmp_buf target */ + /* restore gregs and return to jmp_buf target */ #ifdef PTR_DEMANGLE - "lm %%r6,%%r13,0(%1)\n\t" - "lr %%r15,%%r5\n\t" - LIBC_PROBE_ASM (longjmp_target, 4@%1 -4@%0 4@%%r4) - "br %%r4" + "lm %%r6,%%r13,0(%1)\n\t" + "lr %%r15,%%r5\n\t" + LIBC_PROBE_ASM (longjmp_target, 4@%1 -4@%0 4@%%r4) + "br %%r4" #else - "lm %%r6,%%r15,0(%1)\n\t" - LIBC_PROBE_ASM (longjmp_target, 4@%1 -4@%0 4@%%r14) - "br %%r14" + "lm %%r6,%%r15,0(%1)\n\t" + LIBC_PROBE_ASM (longjmp_target, 4@%1 -4@%0 4@%%r14) + "br %%r14" #endif - : : "r" (r2), + : : "r" (r2), #ifdef PTR_DEMANGLE - "r" (r1), "r" (r3) + "r" (r1), "r" (r3) #else - "a" (env) + "a" (env) #endif - ); + ); /* Avoid `volatile function does return' warnings. */ for (;;); diff --git a/sysdeps/s390/s390-32/add_n.S b/sysdeps/s390/s390-32/add_n.S index 0e0927837f..b8e915712e 100644 --- a/sysdeps/s390/s390-32/add_n.S +++ b/sysdeps/s390/s390-32/add_n.S @@ -1,6 +1,6 @@ /* Add two limb vectors of the same length > 0 and store sum in a third limb vector. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU MP Library. diff --git a/sysdeps/s390/s390-32/addmul_1.S b/sysdeps/s390/s390-32/addmul_1.S index f6575234ea..160c599d16 100644 --- a/sysdeps/s390/s390-32/addmul_1.S +++ b/sysdeps/s390/s390-32/addmul_1.S @@ -1,6 +1,6 @@ /* S390 __mpn_addmul_1 -- Multiply a limb vector with a limb and add the result to a second limb vector. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU MP Library. diff --git a/sysdeps/s390/s390-32/backtrace.c b/sysdeps/s390/s390-32/backtrace.c index 73db65275d..a8290ed86a 100644 --- a/sysdeps/s390/s390-32/backtrace.c +++ b/sysdeps/s390/s390-32/backtrace.c @@ -1,5 +1,5 @@ /* Return backtrace of current program state. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky <schwidefsky@de.ibm.com>. This file is part of the GNU C Library. @@ -17,7 +17,7 @@ License along with the GNU C Library; if not, see <http://www.gnu.org/licenses/>. */ -#include <bits/libc-lock.h> +#include <libc-lock.h> #include <dlfcn.h> #include <execinfo.h> #include <stddef.h> @@ -85,7 +85,7 @@ __backchain_backtrace (void **array, int size) struct layout *stack; int cnt = 0; - asm ("LR %0,%%r15" : "=d" (stack) ); + __asm__ ("LR %0,%%r15" : "=d" (stack) ); /* We skip the call to this function, it makes no sense to record it. */ stack = (struct layout *) stack->back_chain; while (cnt < size) @@ -126,6 +126,10 @@ int __backtrace (void **array, int size) { struct trace_arg arg = { .array = array, .size = size, .cnt = -1 }; + + if (size <= 0) + return 0; + #ifdef SHARED __libc_once_define (static, once); @@ -135,8 +139,7 @@ __backtrace (void **array, int size) return __backchain_backtrace (array, size); #endif - if (size >= 1) - unwind_backtrace (backtrace_helper, &arg); + unwind_backtrace (backtrace_helper, &arg); return arg.cnt != -1 ? arg.cnt : 0; } diff --git a/sysdeps/s390/s390-32/bcopy.S b/sysdeps/s390/s390-32/bcopy.S index 0a42f93f23..cc64cb9aa7 100644 --- a/sysdeps/s390/s390-32/bcopy.S +++ b/sysdeps/s390/s390-32/bcopy.S @@ -1,6 +1,6 @@ /* bcopy -- copy a block from source to destination. S/390 version. This file is part of the GNU C Library. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/s390-32/bzero.S b/sysdeps/s390/s390-32/bzero.S index 7c03c3eb9d..4cbb62e06e 100644 --- a/sysdeps/s390/s390-32/bzero.S +++ b/sysdeps/s390/s390-32/bzero.S @@ -1,6 +1,6 @@ /* bzero -- set a block of memory to zero. IBM S390 version This file is part of the GNU C Library. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/s390-32/crti.S b/sysdeps/s390/s390-32/crti.S index 87444a9ffc..5db5b1e900 100644 --- a/sysdeps/s390/s390-32/crti.S +++ b/sysdeps/s390/s390-32/crti.S @@ -1,5 +1,5 @@ /* Special .init and .fini section support for S/390. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/s390-32/crtn.S b/sysdeps/s390/s390-32/crtn.S index fcf2788af0..73677917dc 100644 --- a/sysdeps/s390/s390-32/crtn.S +++ b/sysdeps/s390/s390-32/crtn.S @@ -1,5 +1,5 @@ /* Special .init and .fini section support for S/390. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/s390-32/dl-machine.h b/sysdeps/s390/s390-32/dl-machine.h index 119e7b568c..14bde3b58d 100644 --- a/sysdeps/s390/s390-32/dl-machine.h +++ b/sysdeps/s390/s390-32/dl-machine.h @@ -1,5 +1,5 @@ /* Machine-dependent ELF dynamic relocation inline functions. S390 Version. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Carl Pederson & Martin Schwidefsky. This file is part of the GNU C Library. @@ -55,10 +55,10 @@ elf_machine_dynamic (void) { register Elf32_Addr *got; - asm( " bras %0,2f\n" - "1: .long _GLOBAL_OFFSET_TABLE_-1b\n" - "2: al %0,0(%0)" - : "=&a" (got) : : "0" ); + __asm__( " bras %0,2f\n" + "1: .long _GLOBAL_OFFSET_TABLE_-1b\n" + "2: al %0,0(%0)" + : "=&a" (got) : : "0" ); return *got; } @@ -70,14 +70,14 @@ elf_machine_load_address (void) { Elf32_Addr addr; - asm( " bras 1,2f\n" - "1: .long _GLOBAL_OFFSET_TABLE_ - 1b\n" - " .long (_dl_start - 1b - 0x80000000) & 0x00000000ffffffff\n" - "2: l %0,4(1)\n" - " ar %0,1\n" - " al 1,0(1)\n" - " sl %0,_dl_start@GOT(1)" - : "=&d" (addr) : : "1" ); + __asm__( " bras 1,2f\n" + "1: .long _GLOBAL_OFFSET_TABLE_ - 1b\n" + " .long (_dl_start - 1b - 0x80000000) & 0x00000000ffffffff\n" + "2: l %0,4(1)\n" + " ar %0,1\n" + " al 1,0(1)\n" + " sl %0,_dl_start@GOT(1)" + : "=&d" (addr) : : "1" ); return addr; } @@ -141,7 +141,7 @@ elf_machine_runtime_setup (struct link_map *l, int lazy, int profile) The C function `_dl_start' is the real entry point; its return value is the user program's entry point. */ -#define RTLD_START asm ("\n\ +#define RTLD_START __asm__ ("\n\ .text\n\ .align 4\n\ .globl _start\n\ diff --git a/sysdeps/s390/s390-32/dl-sysdep.h b/sysdeps/s390/s390-32/dl-sysdep.h index d7a4e53ba3..d550d15985 100644 --- a/sysdeps/s390/s390-32/dl-sysdep.h +++ b/sysdeps/s390/s390-32/dl-sysdep.h @@ -1,5 +1,5 @@ /* System-specific settings for dynamic linker code. S/390 version. - Copyright (C) 2014-2015 Free Software Foundation, Inc. + Copyright (C) 2014-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/s390-32/dl-trampoline.S b/sysdeps/s390/s390-32/dl-trampoline.S index df3e7dbec2..1645610383 100644 --- a/sysdeps/s390/s390-32/dl-trampoline.S +++ b/sysdeps/s390/s390-32/dl-trampoline.S @@ -1,5 +1,5 @@ /* PLT trampolines. s390 version. - Copyright (C) 2005-2015 Free Software Foundation, Inc. + Copyright (C) 2005-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/s390-32/memchr.S b/sysdeps/s390/s390-32/memchr.S index af550ca372..5c82af4b90 100644 --- a/sysdeps/s390/s390-32/memchr.S +++ b/sysdeps/s390/s390-32/memchr.S @@ -1,5 +1,5 @@ /* Search a character in a block of memory. For IBM S390 - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). diff --git a/sysdeps/s390/s390-32/memcmp.S b/sysdeps/s390/s390-32/memcmp.S index 93f06d57a3..50ab61c77f 100644 --- a/sysdeps/s390/s390-32/memcmp.S +++ b/sysdeps/s390/s390-32/memcmp.S @@ -1,5 +1,5 @@ /* memcmp - compare two memory blocks. 32 bit S/390 version. - Copyright (C) 2012-2015 Free Software Foundation, Inc. + Copyright (C) 2012-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -27,38 +27,38 @@ .text #ifdef USE_MULTIARCH -ENTRY(__memcmp_g5) +ENTRY(__memcmp_default) #else ENTRY(memcmp) #endif .machine "g5" - basr %r5,0 + basr %r5,0 .L_G5_16: - ltr %r4,%r4 - je .L_G5_4 - ahi %r4,-1 - lr %r1,%r4 - srl %r1,8 - ltr %r1,%r1 - jne .L_G5_12 - ex %r4,.L_G5_17-.L_G5_16(%r5) + ltr %r4,%r4 + je .L_G5_4 + ahi %r4,-1 + lr %r1,%r4 + srl %r1,8 + ltr %r1,%r1 + jne .L_G5_12 + ex %r4,.L_G5_17-.L_G5_16(%r5) .L_G5_4: - ipm %r2 - sll %r2,2 - sra %r2,30 - br %r14 + ipm %r2 + sll %r2,2 + sra %r2,30 + br %r14 .L_G5_12: - clc 0(256,%r3),0(%r2) - jne .L_G5_4 - la %r3,256(%r3) - la %r2,256(%r2) - brct %r1,.L_G5_12 - ex %r4,.L_G5_17-.L_G5_16(%r5) - j .L_G5_4 + clc 0(256,%r3),0(%r2) + jne .L_G5_4 + la %r3,256(%r3) + la %r2,256(%r2) + brct %r1,.L_G5_12 + ex %r4,.L_G5_17-.L_G5_16(%r5) + j .L_G5_4 .L_G5_17: - clc 0(1,%r3),0(%r2) + clc 0(1,%r3),0(%r2) #ifdef USE_MULTIARCH -END(__memcmp_g5) +END(__memcmp_default) #else END(memcmp) libc_hidden_builtin_def (memcmp) diff --git a/sysdeps/s390/s390-32/memcpy.S b/sysdeps/s390/s390-32/memcpy.S index f26fd00ff8..62ecbbf619 100644 --- a/sysdeps/s390/s390-32/memcpy.S +++ b/sysdeps/s390/s390-32/memcpy.S @@ -1,5 +1,5 @@ /* memcpy - copy a block from source to destination. S/390 version. - Copyright (C) 2012-2015 Free Software Foundation, Inc. + Copyright (C) 2012-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -26,41 +26,41 @@ %r4 = number of bytes to copy. */ #ifdef USE_MULTIARCH -ENTRY(__memcpy_g5) +ENTRY(__memcpy_default) #else ENTRY(memcpy) #endif .machine "g5" - st %r13,52(%r15) - .cfi_offset 13, -44 - basr %r13,0 + st %r13,52(%r15) + .cfi_offset 13, -44 + basr %r13,0 .L_G5_16: - ltr %r4,%r4 - je .L_G5_4 - ahi %r4,-1 - lr %r5,%r4 - srl %r5,8 - ltr %r5,%r5 - lr %r1,%r2 - jne .L_G5_12 - ex %r4,.L_G5_17-.L_G5_16(%r13) + ltr %r4,%r4 + je .L_G5_4 + ahi %r4,-1 + lr %r5,%r4 + srl %r5,8 + ltr %r5,%r5 + lr %r1,%r2 + jne .L_G5_12 + ex %r4,.L_G5_17-.L_G5_16(%r13) .L_G5_4: - l %r13,52(%r15) - br %r14 + l %r13,52(%r15) + br %r14 .L_G5_13: - chi %r5,4096 # Switch to mvcle for copies >1MB - jh __memcpy_mvcle + chi %r5,4096 # Switch to mvcle for copies >1MB + jh __memcpy_mvcle .L_G5_12: - mvc 0(256,%r1),0(%r3) - la %r1,256(%r1) - la %r3,256(%r3) - brct %r5,.L_G5_12 - ex %r4,.L_G5_17-.L_G5_16(%r13) - j .L_G5_4 + mvc 0(256,%r1),0(%r3) + la %r1,256(%r1) + la %r3,256(%r3) + brct %r5,.L_G5_12 + ex %r4,.L_G5_17-.L_G5_16(%r13) + j .L_G5_4 .L_G5_17: - mvc 0(1,%r1),0(%r3) + mvc 0(1,%r1),0(%r3) #ifdef USE_MULTIARCH -END(__memcpy_g5) +END(__memcpy_default) #else END(memcpy) libc_hidden_builtin_def (memcpy) diff --git a/sysdeps/s390/s390-32/memset.S b/sysdeps/s390/s390-32/memset.S index 0abba61c22..eca65d4a49 100644 --- a/sysdeps/s390/s390-32/memset.S +++ b/sysdeps/s390/s390-32/memset.S @@ -1,5 +1,5 @@ /* Set a block of memory to some byte value. For IBM S390 - Copyright (C) 2012-2015 Free Software Foundation, Inc. + Copyright (C) 2012-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -28,37 +28,37 @@ .text #ifdef USE_MULTIARCH -ENTRY(__memset_g5) +ENTRY(__memset_default) #else ENTRY(memset) #endif .machine "g5" - basr %r5,0 + basr %r5,0 .L_G5_19: - ltr %r4,%r4 - je .L_G5_4 - stc %r3,0(%r2) - chi %r4,1 - lr %r1,%r2 - je .L_G5_4 - ahi %r4,-2 - lr %r3,%r4 - srl %r3,8 - ltr %r3,%r3 - jne .L_G5_14 - ex %r4,.L_G5_20-.L_G5_19(%r5) + ltr %r4,%r4 + je .L_G5_4 + stc %r3,0(%r2) + chi %r4,1 + lr %r1,%r2 + je .L_G5_4 + ahi %r4,-2 + lr %r3,%r4 + srl %r3,8 + ltr %r3,%r3 + jne .L_G5_14 + ex %r4,.L_G5_20-.L_G5_19(%r5) .L_G5_4: - br %r14 + br %r14 .L_G5_14: - mvc 1(256,%r1),0(%r1) - la %r1,256(%r1) - brct %r3,.L_G5_14 - ex %r4,.L_G5_20-.L_G5_19(%r5) - j .L_G5_4 + mvc 1(256,%r1),0(%r1) + la %r1,256(%r1) + brct %r3,.L_G5_14 + ex %r4,.L_G5_20-.L_G5_19(%r5) + j .L_G5_4 .L_G5_20: - mvc 1(1,%r1),0(%r1) + mvc 1(1,%r1),0(%r1) #ifdef USE_MULTIARCH -END(__memset_g5) +END(__memset_default) #else END(memset) libc_hidden_builtin_def (memset) diff --git a/sysdeps/s390/s390-32/mul_1.S b/sysdeps/s390/s390-32/mul_1.S index bd8b805fe0..50df39c17f 100644 --- a/sysdeps/s390/s390-32/mul_1.S +++ b/sysdeps/s390/s390-32/mul_1.S @@ -1,6 +1,6 @@ /* __mpn_mul_1 -- Multiply a limb vector with a limb and store the result in a second limb vector. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU MP Library. diff --git a/sysdeps/s390/s390-32/multiarch/Makefile b/sysdeps/s390/s390-32/multiarch/Makefile index 9baeecda64..f8aee14bbd 100644 --- a/sysdeps/s390/s390-32/multiarch/Makefile +++ b/sysdeps/s390/s390-32/multiarch/Makefile @@ -1,3 +1,4 @@ ifeq ($(subdir),string) -sysdep_routines += ifunc-resolve memset memcpy memcmp +sysdep_routines += memset memset-s390 memcpy memcpy-s390 \ + memcmp memcmp-s390 endif diff --git a/sysdeps/s390/s390-32/multiarch/ifunc-resolve.c b/sysdeps/s390/s390-32/multiarch/ifunc-resolve.c deleted file mode 100644 index 8e0cdd5df1..0000000000 --- a/sysdeps/s390/s390-32/multiarch/ifunc-resolve.c +++ /dev/null @@ -1,72 +0,0 @@ -/* IFUNC resolver function for CPU specific functions. - 32 bit S/390 version. - Copyright (C) 2012-2015 Free Software Foundation, Inc. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <unistd.h> -#include <dl-procinfo.h> - -#define STFLE_BITS_Z10 34 /* General instructions extension */ -#define STFLE_BITS_Z196 45 /* Distinct operands, pop ... */ - -#if IS_IN (libc) - -#define IFUNC_RESOLVE(FUNC) \ - asm (".globl " #FUNC "\n\t" \ - ".type " #FUNC ",@gnu_indirect_function\n\t" \ - ".set " #FUNC ",__resolve_" #FUNC "\n\t" \ - ".globl __GI_" #FUNC "\n\t" \ - ".set __GI_" #FUNC "," #FUNC "\n"); \ - \ - /* Make the declarations of the optimized functions hidden in order - to prevent GOT slots being generated for them. */ \ - extern void *__##FUNC##_z196 attribute_hidden; \ - extern void *__##FUNC##_z10 attribute_hidden; \ - extern void *__##FUNC##_g5 attribute_hidden; \ - \ - void *__resolve_##FUNC (unsigned long int dl_hwcap) \ - { \ - if ((dl_hwcap & HWCAP_S390_STFLE) \ - && (dl_hwcap & HWCAP_S390_ZARCH) \ - && (dl_hwcap & HWCAP_S390_HIGH_GPRS)) \ - { \ - /* We want just 1 double word to be returned. */ \ - register unsigned long reg0 asm("0") = 0; \ - unsigned long long stfle_bits; \ - \ - asm volatile(".insn s,0xb2b00000,%0" "\n\t" /* stfle */ \ - : "=QS" (stfle_bits), "+d" (reg0) \ - : : "cc"); \ - \ - if ((stfle_bits & (1ULL << (63 - STFLE_BITS_Z196))) != 0) \ - return &__##FUNC##_z196; \ - else if ((stfle_bits & (1ULL << (63 - STFLE_BITS_Z10))) != 0) \ - return &__##FUNC##_z10; \ - } \ - return &__##FUNC##_g5; \ - } - -IFUNC_RESOLVE(memset) -IFUNC_RESOLVE(memcmp) -asm(".weak bcmp ; bcmp = memcmp"); - -/* In the static lib memcpy is needed before the reloc is resolved. */ -#ifdef SHARED -IFUNC_RESOLVE(memcpy) -#endif - -#endif diff --git a/sysdeps/s390/s390-32/multiarch/memchr.c b/sysdeps/s390/s390-32/multiarch/memchr.c new file mode 100644 index 0000000000..2281e43056 --- /dev/null +++ b/sysdeps/s390/s390-32/multiarch/memchr.c @@ -0,0 +1,21 @@ +/* Multiple versions of memchr. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* This wrapper-file is needed, because otherwise file + sysdeps/s390/s390-[32|64]/memchr.S will be used. */ +#include <sysdeps/s390/multiarch/memchr.c> diff --git a/sysdeps/s390/s390-32/multiarch/memcmp.S b/sysdeps/s390/s390-32/multiarch/memcmp-s390.S index 584dc99792..e9ee6d2270 100644 --- a/sysdeps/s390/s390-32/multiarch/memcmp.S +++ b/sysdeps/s390/s390-32/multiarch/memcmp-s390.S @@ -1,5 +1,5 @@ /* CPU specific memcmp implementations. 32 bit S/390 version. - Copyright (C) 2012-2015 Free Software Foundation, Inc. + Copyright (C) 2012-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -32,73 +32,73 @@ ENTRY(__memcmp_z196) .machine "z196" .machinemode "zarch_nohighgprs" - ltr %r4,%r4 - je .L_Z196_4 - ahi %r4,-1 - srlk %r1,%r4,8 - ltr %r1,%r1 - jne .L_Z196_2 + ltr %r4,%r4 + je .L_Z196_4 + ahi %r4,-1 + srlk %r1,%r4,8 + ltr %r1,%r1 + jne .L_Z196_2 .L_Z196_3: - exrl %r4,.L_Z196_14 + exrl %r4,.L_Z196_14 .L_Z196_4: - ipm %r2 - sll %r2,2 - sra %r2,30 - br %r14 + ipm %r2 + sll %r2,2 + sra %r2,30 + br %r14 .L_Z196_17: - la %r3,256(%r3) - la %r2,256(%r2) - ahi %r1,-1 - je .L_Z196_3 + la %r3,256(%r3) + la %r2,256(%r2) + ahi %r1,-1 + je .L_Z196_3 .L_Z196_2: - pfd 1,512(%r3) - pfd 1,512(%r2) - clc 0(256,%r3),0(%r2) - je .L_Z196_17 - ipm %r2 - sll %r2,2 - sra %r2,30 - br %r14 + pfd 1,512(%r3) + pfd 1,512(%r2) + clc 0(256,%r3),0(%r2) + je .L_Z196_17 + ipm %r2 + sll %r2,2 + sra %r2,30 + br %r14 .L_Z196_14: - clc 0(1,%r3),0(%r2) + clc 0(1,%r3),0(%r2) END(__memcmp_z196) ENTRY(__memcmp_z10) .machine "z10" .machinemode "zarch_nohighgprs" - ltr %r4,%r4 - je .L_Z10_4 - ahi %r4,-1 - lr %r1,%r4 - srl %r1,8 - cijlh %r1,0,.L_Z10_12 + ltr %r4,%r4 + je .L_Z10_4 + ahi %r4,-1 + lr %r1,%r4 + srl %r1,8 + cijlh %r1,0,.L_Z10_12 .L_Z10_3: - exrl %r4,.L_Z10_15 + exrl %r4,.L_Z10_15 .L_Z10_4: - ipm %r2 - sll %r2,2 - sra %r2,30 - br %r14 + ipm %r2 + sll %r2,2 + sra %r2,30 + br %r14 .L_Z10_12: - pfd 1,512(%r3) - pfd 1,512(%r2) - clc 0(256,%r3),0(%r2) - jne .L_Z10_4 - la %r3,256(%r3) - la %r2,256(%r2) - brct %r1,.L_Z10_12 - j .L_Z10_3 + pfd 1,512(%r3) + pfd 1,512(%r2) + clc 0(256,%r3),0(%r2) + jne .L_Z10_4 + la %r3,256(%r3) + la %r2,256(%r2) + brct %r1,.L_Z10_12 + j .L_Z10_3 .L_Z10_15: - clc 0(1,%r3),0(%r2) + clc 0(1,%r3),0(%r2) END(__memcmp_z10) -#endif +#endif /* IS_IN (libc) */ #include "../memcmp.S" #if !IS_IN (libc) .globl memcmp -.set memcmp,__memcmp_g5 +.set memcmp,__memcmp_default .weak bcmp -.set bcmp,__memcmp_g5 +.set bcmp,__memcmp_default #endif diff --git a/sysdeps/s390/s390-32/multiarch/memcmp.c b/sysdeps/s390/s390-32/multiarch/memcmp.c new file mode 100644 index 0000000000..44f72dc8ca --- /dev/null +++ b/sysdeps/s390/s390-32/multiarch/memcmp.c @@ -0,0 +1,24 @@ +/* Multiple versions of memcmp. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) +# include <ifunc-resolve.h> + +s390_libc_ifunc (memcmp) +__asm__(".weak bcmp ; bcmp = memcmp"); +#endif diff --git a/sysdeps/s390/s390-32/multiarch/memcpy.S b/sysdeps/s390/s390-32/multiarch/memcpy-s390.S index 51f4fcff15..4e30cdf6c6 100644 --- a/sysdeps/s390/s390-32/multiarch/memcpy.S +++ b/sysdeps/s390/s390-32/multiarch/memcpy-s390.S @@ -1,5 +1,5 @@ /* CPU specific memcpy implementations. 32 bit S/390 version. - Copyright (C) 2012-2015 Free Software Foundation, Inc. + Copyright (C) 2012-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -32,67 +32,67 @@ ENTRY(__memcpy_z196) .machine "z196" .machinemode "zarch_nohighgprs" - llgfr %r4,%r4 - ltgr %r4,%r4 - je .L_Z196_4 - aghi %r4,-1 - lr %r1,%r2 - srlg %r5,%r4,8 - ltgr %r5,%r5 - jne .L_Z196_5 + llgfr %r4,%r4 + ltgr %r4,%r4 + je .L_Z196_4 + aghi %r4,-1 + lr %r1,%r2 + srlg %r5,%r4,8 + ltgr %r5,%r5 + jne .L_Z196_5 .L_Z196_3: - exrl %r4,.L_Z196_14 + exrl %r4,.L_Z196_14 .L_Z196_4: - br %r14 + br %r14 .L_Z196_5: - cgfi %r5,262144 # Switch to mvcle for copies >64MB - jh __memcpy_mvcle + cgfi %r5,262144 # Switch to mvcle for copies >64MB + jh __memcpy_mvcle .L_Z196_2: - pfd 1,768(%r3) - pfd 2,768(%r1) - mvc 0(256,%r1),0(%r3) - aghi %r5,-1 - la %r1,256(%r1) - la %r3,256(%r3) - jne .L_Z196_2 - j .L_Z196_3 + pfd 1,768(%r3) + pfd 2,768(%r1) + mvc 0(256,%r1),0(%r3) + aghi %r5,-1 + la %r1,256(%r1) + la %r3,256(%r3) + jne .L_Z196_2 + j .L_Z196_3 .L_Z196_14: - mvc 0(1,%r1),0(%r3) + mvc 0(1,%r1),0(%r3) END(__memcpy_z196) ENTRY(__memcpy_z10) .machine "z10" .machinemode "zarch_nohighgprs" - llgfr %r4,%r4 - cgije %r4,0,.L_Z10_4 - aghi %r4,-1 - lr %r1,%r2 - srlg %r5,%r4,8 - cgijlh %r5,0,.L_Z10_13 + llgfr %r4,%r4 + cgije %r4,0,.L_Z10_4 + aghi %r4,-1 + lr %r1,%r2 + srlg %r5,%r4,8 + cgijlh %r5,0,.L_Z10_13 .L_Z10_3: - exrl %r4,.L_Z10_15 + exrl %r4,.L_Z10_15 .L_Z10_4: - br %r14 + br %r14 .L_Z10_13: - cgfi %r5,65535 # Switch to mvcle for copies >16MB - jh __memcpy_mvcle + cgfi %r5,65535 # Switch to mvcle for copies >16MB + jh __memcpy_mvcle .L_Z10_12: - pfd 1,768(%r3) - pfd 2,768(%r1) - mvc 0(256,%r1),0(%r3) - la %r1,256(%r1) - la %r3,256(%r3) - brctg %r5,.L_Z10_12 - j .L_Z10_3 + pfd 1,768(%r3) + pfd 2,768(%r1) + mvc 0(256,%r1),0(%r3) + la %r1,256(%r1) + la %r3,256(%r3) + brctg %r5,.L_Z10_12 + j .L_Z10_3 .L_Z10_15: - mvc 0(1,%r1),0(%r3) + mvc 0(1,%r1),0(%r3) END(__memcpy_z10) -#endif +#endif /* SHARED && IS_IN (libc) */ #include "../memcpy.S" #if !defined SHARED || !IS_IN (libc) .globl memcpy -.set memcpy,__memcpy_g5 +.set memcpy,__memcpy_default #endif diff --git a/sysdeps/s390/s390-32/multiarch/memcpy.c b/sysdeps/s390/s390-32/multiarch/memcpy.c new file mode 100644 index 0000000000..2a98aa0b82 --- /dev/null +++ b/sysdeps/s390/s390-32/multiarch/memcpy.c @@ -0,0 +1,24 @@ +/* Multiple versions of memcpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* In the static lib memcpy is needed before the reloc is resolved. */ +#if defined SHARED && IS_IN (libc) +# include <ifunc-resolve.h> + +s390_libc_ifunc (memcpy) +#endif diff --git a/sysdeps/s390/s390-32/multiarch/memset.S b/sysdeps/s390/s390-32/multiarch/memset-s390.S index 1a7b45f369..47277c13a6 100644 --- a/sysdeps/s390/s390-32/multiarch/memset.S +++ b/sysdeps/s390/s390-32/multiarch/memset-s390.S @@ -1,5 +1,5 @@ /* Set a block of memory to some byte value. 32 bit S/390 version. - Copyright (C) 2012-2015 Free Software Foundation, Inc. + Copyright (C) 2012-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -32,61 +32,61 @@ ENTRY(__memset_z196) .machine "z196" .machinemode "zarch_nohighgprs" - llgfr %r4,%r4 - ltgr %r4,%r4 - je .L_Z196_4 - stc %r3,0(%r2) - lr %r1,%r2 - cghi %r4,1 - je .L_Z196_4 - aghi %r4,-2 - srlg %r5,%r4,8 - ltgr %r5,%r5 - jne .L_Z196_1 + llgfr %r4,%r4 + ltgr %r4,%r4 + je .L_Z196_4 + stc %r3,0(%r2) + lr %r1,%r2 + cghi %r4,1 + je .L_Z196_4 + aghi %r4,-2 + srlg %r5,%r4,8 + ltgr %r5,%r5 + jne .L_Z196_1 .L_Z196_3: - exrl %r4,.L_Z196_17 + exrl %r4,.L_Z196_17 .L_Z196_4: - br %r14 + br %r14 .L_Z196_1: cgfi %r5,1048576 jh __memset_mvcle # Switch to mvcle for >256MB .L_Z196_2: - pfd 2,1024(%r1) - mvc 1(256,%r1),0(%r1) - aghi %r5,-1 - la %r1,256(%r1) - jne .L_Z196_2 - j .L_Z196_3 + pfd 2,1024(%r1) + mvc 1(256,%r1),0(%r1) + aghi %r5,-1 + la %r1,256(%r1) + jne .L_Z196_2 + j .L_Z196_3 .L_Z196_17: - mvc 1(1,%r1),0(%r1) + mvc 1(1,%r1),0(%r1) END(__memset_z196) ENTRY(__memset_z10) .machine "z10" .machinemode "zarch_nohighgprs" - llgfr %r4,%r4 - cgije %r4,0,.L_Z10_4 - stc %r3,0(%r2) - lr %r1,%r2 - cgije %r4,1,.L_Z10_4 - aghi %r4,-2 - srlg %r5,%r4,8 - cgijlh %r5,0,.L_Z10_15 + llgfr %r4,%r4 + cgije %r4,0,.L_Z10_4 + stc %r3,0(%r2) + lr %r1,%r2 + cgije %r4,1,.L_Z10_4 + aghi %r4,-2 + srlg %r5,%r4,8 + cgijlh %r5,0,.L_Z10_15 .L_Z10_3: - exrl %r4,.L_Z10_18 + exrl %r4,.L_Z10_18 .L_Z10_4: - br %r14 + br %r14 .L_Z10_15: cgfi %r5,163840 # Switch to mvcle for >40MB jh __memset_mvcle .L_Z10_14: - pfd 2,1024(%r1) - mvc 1(256,%r1),0(%r1) - la %r1,256(%r1) - brctg %r5,.L_Z10_14 - j .L_Z10_3 + pfd 2,1024(%r1) + mvc 1(256,%r1),0(%r1) + la %r1,256(%r1) + brctg %r5,.L_Z10_14 + j .L_Z10_3 .L_Z10_18: - mvc 1(1,%r1),0(%r1) + mvc 1(1,%r1),0(%r1) END(__memset_z10) ENTRY(__memset_mvcle) @@ -103,11 +103,11 @@ ENTRY(__memset_mvcle) br %r14 END(__memset_mvcle) -#endif +#endif /* IS_IN (libc) */ #include "../memset.S" #if !IS_IN (libc) .globl memset -.set memset,__memset_g5 +.set memset,__memset_default #endif diff --git a/sysdeps/s390/s390-32/multiarch/memset.c b/sysdeps/s390/s390-32/multiarch/memset.c new file mode 100644 index 0000000000..89b8102f2a --- /dev/null +++ b/sysdeps/s390/s390-32/multiarch/memset.c @@ -0,0 +1,23 @@ +/* Multiple versions of memset. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) +# include <ifunc-resolve.h> + +s390_libc_ifunc (memset) +#endif diff --git a/sysdeps/s390/s390-32/multiarch/strcmp.c b/sysdeps/s390/s390-32/multiarch/strcmp.c new file mode 100644 index 0000000000..b7eebc017f --- /dev/null +++ b/sysdeps/s390/s390-32/multiarch/strcmp.c @@ -0,0 +1,21 @@ +/* Multiple versions of strcmp. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* This wrapper-file is needed, because otherwise file + sysdeps/s390/s390-[32|64]/strcmp.S will be used. */ +#include <sysdeps/s390/multiarch/strcmp.c> diff --git a/sysdeps/s390/s390-32/multiarch/strcpy.c b/sysdeps/s390/s390-32/multiarch/strcpy.c new file mode 100644 index 0000000000..ae140d22b7 --- /dev/null +++ b/sysdeps/s390/s390-32/multiarch/strcpy.c @@ -0,0 +1,21 @@ +/* Multiple versions of strcpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* This wrapper-file is needed, because otherwise file + sysdeps/s390/s390-[32|64]/strcpy.S will be used. */ +#include <sysdeps/s390/multiarch/strcpy.c> diff --git a/sysdeps/s390/s390-32/multiarch/strncpy.c b/sysdeps/s390/s390-32/multiarch/strncpy.c new file mode 100644 index 0000000000..28a2af72e4 --- /dev/null +++ b/sysdeps/s390/s390-32/multiarch/strncpy.c @@ -0,0 +1,21 @@ +/* Multiple versions of strncpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* This wrapper-file is needed, because otherwise file + sysdeps/s390/s390-[32|64]/strncpy.S will be used. */ +#include <sysdeps/s390/multiarch/strncpy.c> diff --git a/sysdeps/s390/s390-32/s390-mcount.S b/sysdeps/s390/s390-32/s390-mcount.S index cc70641621..a27f434fbf 100644 --- a/sysdeps/s390/s390-32/s390-mcount.S +++ b/sysdeps/s390/s390-32/s390-mcount.S @@ -1,5 +1,5 @@ /* S/390-specific implemetation of profiling support. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com) diff --git a/sysdeps/s390/s390-32/setjmp.S b/sysdeps/s390/s390-32/setjmp.S index e068a3cbfe..dbacb0fdf2 100644 --- a/sysdeps/s390/s390-32/setjmp.S +++ b/sysdeps/s390/s390-32/setjmp.S @@ -1,5 +1,5 @@ /* setjmp for s390, ELF version. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU C Library. @@ -27,21 +27,23 @@ #if !IS_IN (rtld) # if defined SHARED && SHLIB_COMPAT (libc, GLIBC_2_19, GLIBC_2_20) /* we need a unique name in case of symbol versioning. */ +# define setjmp __v1setjmp +# define _setjmp __v1_setjmp # define __sigsetjmp __v1__sigsetjmp + +# undef libc_hidden_def +# define libc_hidden_def(name) strong_alias(_setjmp, __GI__setjmp) # endif /* if defined SHARED && SHLIB_COMPAT (libc, GLIBC_2_19, GLIBC_2_20) */ #endif /* !IS_IN (rtld) */ - /* We include the BSD entry points here as well but we make - them weak. */ + /* We include the BSD entry points here as well. */ ENTRY (setjmp) - .weak C_SYMBOL_NAME (setjmp) lhi %r3,1 /* second argument of one */ j .Linternal_sigsetjmp /* branch relativ to __sigsetjmp */ END (setjmp) /* Binary compatibility entry point. */ ENTRY(_setjmp) - .weak C_SYMBOL_NAME (_setjmp) lhi %r3,0 /* second argument of zero */ j .Linternal_sigsetjmp /* branch relativ to __sigsetjmp */ END (_setjmp) @@ -96,15 +98,15 @@ END (__sigsetjmp) /* In glibc release 2.19 new versions of setjmp-functions were introduced, but were reverted before 2.20. Thus both versions are the same function. */ +# undef setjmp +# undef _setjmp # undef __sigsetjmp -weak_alias (setjmp, __v1setjmp); -weak_alias (setjmp, __v2setjmp); +strong_alias (__v1setjmp, __v2setjmp); versioned_symbol (libc, __v1setjmp, setjmp, GLIBC_2_0); compat_symbol (libc, __v2setjmp, setjmp, GLIBC_2_19); -weak_alias (_setjmp, __v1_setjmp); -weak_alias (_setjmp, __v2_setjmp); +strong_alias (__v1_setjmp, __v2_setjmp); versioned_symbol (libc, __v1_setjmp, _setjmp, GLIBC_2_0); compat_symbol (libc, __v2_setjmp, _setjmp, GLIBC_2_19); diff --git a/sysdeps/s390/s390-32/stackguard-macros.h b/sysdeps/s390/s390-32/stackguard-macros.h index 449e8d488f..46109744ed 100644 --- a/sysdeps/s390/s390-32/stackguard-macros.h +++ b/sysdeps/s390/s390-32/stackguard-macros.h @@ -1,15 +1,15 @@ #include <stdint.h> #define STACK_CHK_GUARD \ - ({ uintptr_t x; asm ("ear %0,%%a0; l %0,0x14(%0)" : "=a" (x)); x; }) + ({ uintptr_t x; __asm__ ("ear %0,%%a0; l %0,0x14(%0)" : "=a" (x)); x; }) /* On s390/s390x there is no unique pointer guard, instead we use the same value as the stack guard. */ #define POINTER_CHK_GUARD \ - ({ \ - uintptr_t x; \ - asm ("ear %0,%%a0; l %0,%1(%0)" \ - : "=a" (x) \ - : "i" (offsetof (tcbhead_t, stack_guard))); \ - x; \ - }) + ({ \ + uintptr_t x; \ + __asm__ ("ear %0,%%a0; l %0,%1(%0)" \ + : "=a" (x) \ + : "i" (offsetof (tcbhead_t, stack_guard))); \ + x; \ + }) diff --git a/sysdeps/s390/s390-32/start.S b/sysdeps/s390/s390-32/start.S index 7d3c6d9a89..1fbc64d2e4 100644 --- a/sysdeps/s390/s390-32/start.S +++ b/sysdeps/s390/s390-32/start.S @@ -1,5 +1,5 @@ /* Startup code compliant to the ELF s390 ABI. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU C Library. diff --git a/sysdeps/s390/s390-32/strcmp.S b/sysdeps/s390/s390-32/strcmp.S index 109de12deb..71f113ebab 100644 --- a/sysdeps/s390/s390-32/strcmp.S +++ b/sysdeps/s390/s390-32/strcmp.S @@ -1,6 +1,6 @@ /* strcmp - compare two string. S/390 version. This file is part of the GNU C Library. - Copyright (C) 2001-2015 Free Software Foundation, Inc. + Copyright (C) 2001-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/s390-32/strcpy.S b/sysdeps/s390/s390-32/strcpy.S index b8b9b5f34b..5cdc350f91 100644 --- a/sysdeps/s390/s390-32/strcpy.S +++ b/sysdeps/s390/s390-32/strcpy.S @@ -1,6 +1,6 @@ /* strcpy - copy a string from source to destination. For IBM S390 This file is part of the GNU C Library. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/s390-32/strncpy.S b/sysdeps/s390/s390-32/strncpy.S index b0e234e2da..75800b3ee6 100644 --- a/sysdeps/s390/s390-32/strncpy.S +++ b/sysdeps/s390/s390-32/strncpy.S @@ -1,7 +1,7 @@ /* strncpy - copy at most n characters from a string from source to destination. For IBM S390 This file is part of the GNU C Library. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/s390-32/sub_n.S b/sysdeps/s390/s390-32/sub_n.S index 3ade1fe86d..f8de2c2a5e 100644 --- a/sysdeps/s390/s390-32/sub_n.S +++ b/sysdeps/s390/s390-32/sub_n.S @@ -1,6 +1,6 @@ /* __mpn_sub_n -- Add two limb vectors of the same length > 0 and store sum in a third limb vector. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU MP Library. diff --git a/sysdeps/s390/s390-32/sysdep.h b/sysdeps/s390/s390-32/sysdep.h index 779f1799b3..26e9285dbd 100644 --- a/sysdeps/s390/s390-32/sysdep.h +++ b/sysdeps/s390/s390-32/sysdep.h @@ -1,5 +1,5 @@ /* Assembler macros for s390. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU C Library. diff --git a/sysdeps/s390/s390-32/tls-macros.h b/sysdeps/s390/s390-32/tls-macros.h index a592d81585..09b42aa37a 100644 --- a/sysdeps/s390/s390-32/tls-macros.h +++ b/sysdeps/s390/s390-32/tls-macros.h @@ -1,102 +1,102 @@ #define TLS_LE(x) \ ({ unsigned long __offset; \ - asm ("bras %0,1f\n" \ - "0:\t.long " #x "@ntpoff\n" \ - "1:\tl %0,0(%0)" \ - : "=a" (__offset) : : "cc" ); \ + __asm__ ("bras %0,1f\n" \ + "0:\t.long " #x "@ntpoff\n" \ + "1:\tl %0,0(%0)" \ + : "=a" (__offset) : : "cc" ); \ (int *) (__builtin_thread_pointer() + __offset); }) #ifdef PIC # define TLS_IE(x) \ ({ unsigned long __offset, __got; \ - asm ("bras %0,1f\n" \ - "0:\t.long _GLOBAL_OFFSET_TABLE_-0b\n\t" \ - ".long " #x "@gotntpoff\n" \ - "1:\tl %1,0(%0)\n\t" \ - "la %1,0(%1,%0)\n\t" \ - "l %0,4(%0)\n\t" \ - "l %0,0(%0,%1):tls_load:" #x "\n" \ - : "=&a" (__offset), "=&a" (__got) : : "cc" ); \ + __asm__ ("bras %0,1f\n" \ + "0:\t.long _GLOBAL_OFFSET_TABLE_-0b\n\t" \ + ".long " #x "@gotntpoff\n" \ + "1:\tl %1,0(%0)\n\t" \ + "la %1,0(%1,%0)\n\t" \ + "l %0,4(%0)\n\t" \ + "l %0,0(%0,%1):tls_load:" #x "\n" \ + : "=&a" (__offset), "=&a" (__got) : : "cc" ); \ (int *) (__builtin_thread_pointer() + __offset); }) #else # define TLS_IE(x) \ ({ unsigned long __offset; \ - asm ("bras %0,1f\n" \ - "0:\t.long " #x "@indntpoff\n" \ - "1:\t l %0,0(%0)\n\t" \ - "l %0,0(%0):tls_load:" #x \ - : "=&a" (__offset) : : "cc" ); \ + __asm__ ("bras %0,1f\n" \ + "0:\t.long " #x "@indntpoff\n" \ + "1:\t l %0,0(%0)\n\t" \ + "l %0,0(%0):tls_load:" #x \ + : "=&a" (__offset) : : "cc" ); \ (int *) (__builtin_thread_pointer() + __offset); }) #endif #ifdef PIC # define TLS_LD(x) \ ({ unsigned long __offset, __save12; \ - asm ("bras %0,1f\n" \ - "0:\t.long _GLOBAL_OFFSET_TABLE_-0b\n\t" \ - ".long __tls_get_offset@plt-0b\n\t" \ - ".long " #x "@tlsldm\n\t" \ - ".long " #x "@dtpoff\n" \ - "1:\tlr %1,%%r12\n\t" \ - "l %%r12,0(%0)\n\t" \ - "la %%r12,0(%%r12,%0)\n\t" \ - "l %%r1,4(%0)\n\t" \ - "l %%r2,8(%0)\n\t" \ - "bas %%r14,0(%%r1,%0):tls_ldcall:" #x "\n\t" \ - "l %0,12(%0)\n\t" \ - "alr %0,%%r2\n\t" \ - "lr %%r12,%1" \ - : "=&a" (__offset), "=&a" (__save12) \ - : : "cc", "0", "1", "2", "3", "4", "5" ); \ + __asm__ ("bras %0,1f\n" \ + "0:\t.long _GLOBAL_OFFSET_TABLE_-0b\n\t" \ + ".long __tls_get_offset@plt-0b\n\t" \ + ".long " #x "@tlsldm\n\t" \ + ".long " #x "@dtpoff\n" \ + "1:\tlr %1,%%r12\n\t" \ + "l %%r12,0(%0)\n\t" \ + "la %%r12,0(%%r12,%0)\n\t" \ + "l %%r1,4(%0)\n\t" \ + "l %%r2,8(%0)\n\t" \ + "bas %%r14,0(%%r1,%0):tls_ldcall:" #x "\n\t" \ + "l %0,12(%0)\n\t" \ + "alr %0,%%r2\n\t" \ + "lr %%r12,%1" \ + : "=&a" (__offset), "=&a" (__save12) \ + : : "cc", "0", "1", "2", "3", "4", "5" ); \ (int *) (__builtin_thread_pointer() + __offset); }) #else # define TLS_LD(x) \ ({ unsigned long __offset; \ - asm ("bras %0,1f\n" \ - "0:\t.long _GLOBAL_OFFSET_TABLE_\n\t" \ - ".long __tls_get_offset@plt\n\t" \ - ".long " #x "@tlsldm\n\t" \ - ".long " #x "@dtpoff\n" \ - "1:\tl %%r12,0(%0)\n\t" \ - "l %%r1,4(%0)\n\t" \ - "l %%r2,8(%0)\n\t" \ - "bas %%r14,0(%%r1):tls_ldcall:" #x "\n\t" \ - "l %0,12(%0)\n\t" \ - "alr %0,%%r2" \ - : "=&a" (__offset) : : "cc", "0", "1", "2", "3", "4", "5", "12" ); \ + __asm__ ("bras %0,1f\n" \ + "0:\t.long _GLOBAL_OFFSET_TABLE_\n\t" \ + ".long __tls_get_offset@plt\n\t" \ + ".long " #x "@tlsldm\n\t" \ + ".long " #x "@dtpoff\n" \ + "1:\tl %%r12,0(%0)\n\t" \ + "l %%r1,4(%0)\n\t" \ + "l %%r2,8(%0)\n\t" \ + "bas %%r14,0(%%r1):tls_ldcall:" #x "\n\t" \ + "l %0,12(%0)\n\t" \ + "alr %0,%%r2" \ + : "=&a" (__offset) : : "cc", "0", "1", "2", "3", "4", "5", "12" ); \ (int *) (__builtin_thread_pointer() + __offset); }) #endif #ifdef PIC # define TLS_GD(x) \ ({ unsigned long __offset, __save12; \ - asm ("bras %0,1f\n" \ - "0:\t.long _GLOBAL_OFFSET_TABLE_-0b\n\t" \ - ".long __tls_get_offset@plt-0b\n\t" \ - ".long " #x "@tlsgd\n" \ - "1:\tlr %1,%%r12\n\t" \ - "l %%r12,0(%0)\n\t" \ - "la %%r12,0(%%r12,%0)\n\t" \ - "l %%r1,4(%0)\n\t" \ - "l %%r2,8(%0)\n\t" \ - "bas %%r14,0(%%r1,%0):tls_gdcall:" #x "\n\t" \ - "lr %0,%%r2\n\t" \ - "lr %%r12,%1" \ - : "=&a" (__offset), "=&a" (__save12) \ - : : "cc", "0", "1", "2", "3", "4", "5" ); \ + __asm__ ("bras %0,1f\n" \ + "0:\t.long _GLOBAL_OFFSET_TABLE_-0b\n\t" \ + ".long __tls_get_offset@plt-0b\n\t" \ + ".long " #x "@tlsgd\n" \ + "1:\tlr %1,%%r12\n\t" \ + "l %%r12,0(%0)\n\t" \ + "la %%r12,0(%%r12,%0)\n\t" \ + "l %%r1,4(%0)\n\t" \ + "l %%r2,8(%0)\n\t" \ + "bas %%r14,0(%%r1,%0):tls_gdcall:" #x "\n\t" \ + "lr %0,%%r2\n\t" \ + "lr %%r12,%1" \ + : "=&a" (__offset), "=&a" (__save12) \ + : : "cc", "0", "1", "2", "3", "4", "5" ); \ (int *) (__builtin_thread_pointer() + __offset); }) #else # define TLS_GD(x) \ ({ unsigned long __offset; \ - asm ("bras %0,1f\n" \ - "0:\t.long _GLOBAL_OFFSET_TABLE_\n\t" \ - ".long __tls_get_offset@plt\n\t" \ - ".long " #x "@tlsgd\n" \ - "1:\tl %%r12,0(%0)\n\t" \ - "l %%r1,4(%0)\n\t" \ - "l %%r2,8(%0)\n\t" \ - "bas %%r14,0(%%r1):tls_gdcall:" #x "\n\t" \ - "lr %0,%%r2" \ - : "=&a" (__offset) : : "cc", "0", "1", "2", "3", "4", "5", "12" ); \ + __asm__ ("bras %0,1f\n" \ + "0:\t.long _GLOBAL_OFFSET_TABLE_\n\t" \ + ".long __tls_get_offset@plt\n\t" \ + ".long " #x "@tlsgd\n" \ + "1:\tl %%r12,0(%0)\n\t" \ + "l %%r1,4(%0)\n\t" \ + "l %%r2,8(%0)\n\t" \ + "bas %%r14,0(%%r1):tls_gdcall:" #x "\n\t" \ + "lr %0,%%r2" \ + : "=&a" (__offset) : : "cc", "0", "1", "2", "3", "4", "5", "12" ); \ (int *) (__builtin_thread_pointer() + __offset); }) #endif diff --git a/sysdeps/s390/s390-32/tst-audit.h b/sysdeps/s390/s390-32/tst-audit.h index 27bc11343f..8908602cff 100644 --- a/sysdeps/s390/s390-32/tst-audit.h +++ b/sysdeps/s390/s390-32/tst-audit.h @@ -1,6 +1,6 @@ /* Definitions for testing PLT entry/exit auditing. S/390 32-bit version. - Copyright (C) 2012-2015 Free Software Foundation, Inc. + Copyright (C) 2012-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. diff --git a/sysdeps/s390/s390-64/__longjmp.c b/sysdeps/s390/s390-64/__longjmp.c index e75e648a32..66005b82ac 100644 --- a/sysdeps/s390/s390-64/__longjmp.c +++ b/sysdeps/s390/s390-64/__longjmp.c @@ -1,4 +1,4 @@ -/* Copyright (C) 2001-2015 Free Software Foundation, Inc. +/* Copyright (C) 2001-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). @@ -37,52 +37,52 @@ __longjmp (__jmp_buf env, int val) #elif defined CHECK_SP CHECK_SP (env, 0); #endif - register long int r2 __asm ("%r2") = val == 0 ? 1 : val; + register long int r2 __asm__ ("%r2") = val == 0 ? 1 : val; #ifdef PTR_DEMANGLE - register uintptr_t r3 __asm ("%r3") = guard; - register void *r1 __asm ("%r1") = (void *) env; + register uintptr_t r3 __asm__ ("%r3") = guard; + register void *r1 __asm__ ("%r1") = (void *) env; #endif /* Restore registers and jump back. */ - asm volatile ( - /* longjmp probe expects longjmp first argument, second - argument and target address. */ + __asm__ __volatile__ ( + /* longjmp probe expects longjmp first argument, second + argument and target address. */ #ifdef PTR_DEMANGLE - "lmg %%r4,%%r5,64(%1)\n\t" - "xgr %%r4,%2\n\t" - "xgr %%r5,%2\n\t" - LIBC_PROBE_ASM (longjmp, 8@%1 -4@%0 8@%%r4) + "lmg %%r4,%%r5,64(%1)\n\t" + "xgr %%r4,%2\n\t" + "xgr %%r5,%2\n\t" + LIBC_PROBE_ASM (longjmp, 8@%1 -4@%0 8@%%r4) #else - LIBC_PROBE_ASM (longjmp, 8@%1 -4@%0 8@%%r14) + LIBC_PROBE_ASM (longjmp, 8@%1 -4@%0 8@%%r14) #endif - /* restore fpregs */ - "ld %%f8,80(%1)\n\t" - "ld %%f9,88(%1)\n\t" - "ld %%f10,96(%1)\n\t" - "ld %%f11,104(%1)\n\t" - "ld %%f12,112(%1)\n\t" - "ld %%f13,120(%1)\n\t" - "ld %%f14,128(%1)\n\t" - "ld %%f15,136(%1)\n\t" + /* restore fpregs */ + "ld %%f8,80(%1)\n\t" + "ld %%f9,88(%1)\n\t" + "ld %%f10,96(%1)\n\t" + "ld %%f11,104(%1)\n\t" + "ld %%f12,112(%1)\n\t" + "ld %%f13,120(%1)\n\t" + "ld %%f14,128(%1)\n\t" + "ld %%f15,136(%1)\n\t" - /* restore gregs and return to jmp_buf target */ + /* restore gregs and return to jmp_buf target */ #ifdef PTR_DEMANGLE - "lmg %%r6,%%r13,0(%1)\n\t" - "lgr %%r15,%%r5\n\t" - LIBC_PROBE_ASM (longjmp_target, 8@%1 -4@%0 8@%%r4) - "br %%r4" + "lmg %%r6,%%r13,0(%1)\n\t" + "lgr %%r15,%%r5\n\t" + LIBC_PROBE_ASM (longjmp_target, 8@%1 -4@%0 8@%%r4) + "br %%r4" #else - "lmg %%r6,%%r15,0(%1)\n\t" - LIBC_PROBE_ASM (longjmp_target, 8@%1 -4@%0 8@%%r14) - "br %%r14" + "lmg %%r6,%%r15,0(%1)\n\t" + LIBC_PROBE_ASM (longjmp_target, 8@%1 -4@%0 8@%%r14) + "br %%r14" #endif - : : "r" (r2), + : : "r" (r2), #ifdef PTR_DEMANGLE - "r" (r1), "r" (r3) + "r" (r1), "r" (r3) #else - "a" (env) + "a" (env) #endif - ); + ); /* Avoid `volatile function does return' warnings. */ for (;;); diff --git a/sysdeps/s390/s390-64/add_n.S b/sysdeps/s390/s390-64/add_n.S index cd71e21929..11bc60170b 100644 --- a/sysdeps/s390/s390-64/add_n.S +++ b/sysdeps/s390/s390-64/add_n.S @@ -1,6 +1,6 @@ /* Add two limb vectors of the same length > 0 and store sum in a third limb vector. - Copyright (C) 2001-2015 Free Software Foundation, Inc. + Copyright (C) 2001-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU MP Library. diff --git a/sysdeps/s390/s390-64/backtrace.c b/sysdeps/s390/s390-64/backtrace.c index 08e563e0df..5f8b7f8fff 100644 --- a/sysdeps/s390/s390-64/backtrace.c +++ b/sysdeps/s390/s390-64/backtrace.c @@ -1,5 +1,5 @@ /* Return backtrace of current program state. 64 bit S/390 version. - Copyright (C) 2001-2015 Free Software Foundation, Inc. + Copyright (C) 2001-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky <schwidefsky@de.ibm.com>. This file is part of the GNU C Library. @@ -17,7 +17,7 @@ License along with the GNU C Library; if not, see <http://www.gnu.org/licenses/>. */ -#include <bits/libc-lock.h> +#include <libc-lock.h> #include <dlfcn.h> #include <execinfo.h> #include <stddef.h> @@ -84,7 +84,7 @@ __backchain_backtrace (void **array, int size) struct layout *stack; int cnt = 0; - asm ("LGR %0,%%r15" : "=d" (stack) ); + __asm__ ("LGR %0,%%r15" : "=d" (stack) ); /* We skip the call to this function, it makes no sense to record it. */ stack = (struct layout *) stack->back_chain; while (cnt < size) @@ -125,6 +125,10 @@ int __backtrace (void **array, int size) { struct trace_arg arg = { .array = array, .size = size, .cnt = -1 }; + + if (size <= 0) + return 0; + #ifdef SHARED __libc_once_define (static, once); @@ -134,8 +138,7 @@ __backtrace (void **array, int size) return __backchain_backtrace (array, size); #endif - if (size >= 1) - unwind_backtrace (backtrace_helper, &arg); + unwind_backtrace (backtrace_helper, &arg); return arg.cnt != -1 ? arg.cnt : 0; } diff --git a/sysdeps/s390/s390-64/bcopy.S b/sysdeps/s390/s390-64/bcopy.S index cbde16d2cb..7eeeae499c 100644 --- a/sysdeps/s390/s390-64/bcopy.S +++ b/sysdeps/s390/s390-64/bcopy.S @@ -1,6 +1,6 @@ /* bcopy -- copy a block from source to destination. 64 bit S/390 version. This file is part of the GNU C Library. - Copyright (C) 2000-2015 Free Software Foundation, Inc. + Copyright (C) 2000-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/s390-64/bzero.S b/sysdeps/s390/s390-64/bzero.S index 355142b3e1..891efc2d6e 100644 --- a/sysdeps/s390/s390-64/bzero.S +++ b/sysdeps/s390/s390-64/bzero.S @@ -1,6 +1,6 @@ /* bzero -- set a block of memory to zero. 64 bit S/390 version. This file is part of the GNU C Library. - Copyright (C) 2001-2015 Free Software Foundation, Inc. + Copyright (C) 2001-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/s390-64/crti.S b/sysdeps/s390/s390-64/crti.S index 7b8bb82050..248ef83dbe 100644 --- a/sysdeps/s390/s390-64/crti.S +++ b/sysdeps/s390/s390-64/crti.S @@ -1,5 +1,5 @@ /* Special .init and .fini section support for 64 bit S/390. - Copyright (C) 2001-2015 Free Software Foundation, Inc. + Copyright (C) 2001-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU C Library. diff --git a/sysdeps/s390/s390-64/crtn.S b/sysdeps/s390/s390-64/crtn.S index 622f2e2db0..ce906acffc 100644 --- a/sysdeps/s390/s390-64/crtn.S +++ b/sysdeps/s390/s390-64/crtn.S @@ -1,5 +1,5 @@ /* Special .init and .fini section support for 64 bit S/390. - Copyright (C) 2001-2015 Free Software Foundation, Inc. + Copyright (C) 2001-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU C Library. diff --git a/sysdeps/s390/s390-64/dl-machine.h b/sysdeps/s390/s390-64/dl-machine.h index eeadbcd163..cb81aafc5d 100644 --- a/sysdeps/s390/s390-64/dl-machine.h +++ b/sysdeps/s390/s390-64/dl-machine.h @@ -1,6 +1,6 @@ /* Machine-dependent ELF dynamic relocation inline functions. 64 bit S/390 Version. - Copyright (C) 2001-2015 Free Software Foundation, Inc. + Copyright (C) 2001-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU C Library. @@ -50,8 +50,8 @@ elf_machine_dynamic (void) { register Elf64_Addr *got; - asm( " larl %0,_GLOBAL_OFFSET_TABLE_\n" - : "=&a" (got) : : "0" ); + __asm__ ( " larl %0,_GLOBAL_OFFSET_TABLE_\n" + : "=&a" (got) : : "0" ); return *got; } @@ -62,11 +62,11 @@ elf_machine_load_address (void) { Elf64_Addr addr; - asm( " larl %0,_dl_start\n" - " larl 1,_GLOBAL_OFFSET_TABLE_\n" - " lghi 2,_dl_start@GOT\n" - " slg %0,0(2,1)" - : "=&d" (addr) : : "1", "2" ); + __asm__( " larl %0,_dl_start\n" + " larl 1,_GLOBAL_OFFSET_TABLE_\n" + " lghi 2,_dl_start@GOT\n" + " slg %0,0(2,1)" + : "=&d" (addr) : : "1", "2" ); return addr; } @@ -126,7 +126,7 @@ elf_machine_runtime_setup (struct link_map *l, int lazy, int profile) The C function `_dl_start' is the real entry point; its return value is the user program's entry point. */ -#define RTLD_START asm ("\n\ +#define RTLD_START __asm__ ("\n\ .text\n\ .align 4\n\ .globl _start\n\ diff --git a/sysdeps/s390/s390-64/dl-trampoline.S b/sysdeps/s390/s390-64/dl-trampoline.S index f93b3fbc70..6919ed0138 100644 --- a/sysdeps/s390/s390-64/dl-trampoline.S +++ b/sysdeps/s390/s390-64/dl-trampoline.S @@ -1,5 +1,5 @@ /* PLT trampolines. s390x version. - Copyright (C) 2005-2015 Free Software Foundation, Inc. + Copyright (C) 2005-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/s390-64/iso-8859-1_cp037_z900.c b/sysdeps/s390/s390-64/iso-8859-1_cp037_z900.c index d020fd01db..c59f87f18d 100644 --- a/sysdeps/s390/s390-64/iso-8859-1_cp037_z900.c +++ b/sysdeps/s390/s390-64/iso-8859-1_cp037_z900.c @@ -2,7 +2,7 @@ This module uses the Z900 variant of the Translate One To One instruction. - Copyright (C) 1997-2015 Free Software Foundation, Inc. + Copyright (C) 1997-2016 Free Software Foundation, Inc. Author: Andreas Krebbel <Andreas.Krebbel@de.ibm.com> Based on the work by Ulrich Drepper <drepper@cygnus.com>, 1997. @@ -184,28 +184,28 @@ __attribute__ ((aligned (8))) = #define TROO_LOOP(TABLE) \ { \ - register const unsigned char test asm ("0") = 0; \ - register const unsigned char *pTable asm ("1") = TABLE; \ - register unsigned char *pOutput asm ("2") = outptr; \ - register uint64_t length asm ("3"); \ + register const unsigned char test __asm__ ("0") = 0; \ + register const unsigned char *pTable __asm__ ("1") = TABLE; \ + register unsigned char *pOutput __asm__ ("2") = outptr; \ + register uint64_t length __asm__ ("3"); \ const unsigned char* pInput = inptr; \ uint64_t tmp; \ \ length = (inend - inptr < outend - outptr \ ? inend - inptr : outend - outptr); \ \ - asm volatile ("0: \n\t" \ - " troo %0,%1 \n\t" \ - " jz 1f \n\t" \ - " jo 0b \n\t" \ - " llgc %3,0(%1) \n\t" \ - " la %3,0(%3,%4) \n\t" \ - " mvc 0(1,%0),0(%3) \n\t" \ - " aghi %1,1 \n\t" \ - " aghi %0,1 \n\t" \ - " aghi %2,-1 \n\t" \ - " j 0b \n\t" \ - "1: \n" \ + __asm__ volatile ("0: \n\t" \ + " troo %0,%1 \n\t" \ + " jz 1f \n\t" \ + " jo 0b \n\t" \ + " llgc %3,0(%1) \n\t" \ + " la %3,0(%3,%4) \n\t" \ + " mvc 0(1,%0),0(%3) \n\t" \ + " aghi %1,1 \n\t" \ + " aghi %0,1 \n\t" \ + " aghi %2,-1 \n\t" \ + " j 0b \n\t" \ + "1: \n" \ \ : "+a" (pOutput), "+a" (pInput), "+d" (length), "=&a" (tmp) \ : "a" (pTable), "d" (test) \ diff --git a/sysdeps/s390/s390-64/memchr.S b/sysdeps/s390/s390-64/memchr.S index 6e0c555200..8d50dcfe86 100644 --- a/sysdeps/s390/s390-64/memchr.S +++ b/sysdeps/s390/s390-64/memchr.S @@ -1,5 +1,5 @@ /* Search a character in a block of memory. 64 bit S/390 version. - Copyright (C) 2001-2015 Free Software Foundation, Inc. + Copyright (C) 2001-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU C Library. diff --git a/sysdeps/s390/s390-64/memcmp.S b/sysdeps/s390/s390-64/memcmp.S index 6767438f28..5e79d544bf 100644 --- a/sysdeps/s390/s390-64/memcmp.S +++ b/sysdeps/s390/s390-64/memcmp.S @@ -1,5 +1,5 @@ /* memcmp - compare two memory blocks. 64 bit S/390 version. - Copyright (C) 2012-2015 Free Software Foundation, Inc. + Copyright (C) 2012-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -27,36 +27,36 @@ .text #ifdef USE_MULTIARCH -ENTRY(__memcmp_z900) +ENTRY(__memcmp_default) #else ENTRY(memcmp) #endif .machine "z900" - ltgr %r4,%r4 - je .L_Z900_4 - aghi %r4,-1 - srlg %r1,%r4,8 - ltgr %r1,%r1 - jne .L_Z900_12 + ltgr %r4,%r4 + je .L_Z900_4 + aghi %r4,-1 + srlg %r1,%r4,8 + ltgr %r1,%r1 + jne .L_Z900_12 .L_Z900_3: - larl %r1,.L_Z900_15 - ex %r4,0(%r1) + larl %r1,.L_Z900_15 + ex %r4,0(%r1) .L_Z900_4: - ipm %r2 - sllg %r2,%r2,34 - srag %r2,%r2,62 - br %r14 + ipm %r2 + sllg %r2,%r2,34 + srag %r2,%r2,62 + br %r14 .L_Z900_12: - clc 0(256,%r3),0(%r2) - jne .L_Z900_4 - la %r3,256(%r3) - la %r2,256(%r2) - brctg %r1,.L_Z900_12 - j .L_Z900_3 + clc 0(256,%r3),0(%r2) + jne .L_Z900_4 + la %r3,256(%r3) + la %r2,256(%r2) + brctg %r1,.L_Z900_12 + j .L_Z900_3 .L_Z900_15: - clc 0(1,%r3),0(%r2) + clc 0(1,%r3),0(%r2) #ifdef USE_MULTIARCH -END(__memcmp_z900) +END(__memcmp_default) #else END(memcmp) libc_hidden_builtin_def (memcmp) diff --git a/sysdeps/s390/s390-64/memcpy.S b/sysdeps/s390/s390-64/memcpy.S index 3f122dcf0f..e84a3572cb 100644 --- a/sysdeps/s390/s390-64/memcpy.S +++ b/sysdeps/s390/s390-64/memcpy.S @@ -1,5 +1,5 @@ /* memcpy - copy a block from source to destination. 64 bit S/390 version. - Copyright (C) 2012-2015 Free Software Foundation, Inc. + Copyright (C) 2012-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -29,37 +29,37 @@ .text #ifdef USE_MULTIARCH -ENTRY(__memcpy_z900) +ENTRY(__memcpy_default) #else ENTRY(memcpy) #endif .machine "z900" - ltgr %r4,%r4 - je .L_Z900_4 - aghi %r4,-1 - srlg %r5,%r4,8 - ltgr %r5,%r5 - lgr %r1,%r2 - jne .L_Z900_13 + ltgr %r4,%r4 + je .L_Z900_4 + aghi %r4,-1 + srlg %r5,%r4,8 + ltgr %r5,%r5 + lgr %r1,%r2 + jne .L_Z900_13 .L_Z900_3: - larl %r5,.L_Z900_15 - ex %r4,0(%r5) + larl %r5,.L_Z900_15 + ex %r4,0(%r5) .L_Z900_4: - br %r14 + br %r14 .L_Z900_13: - chi %r5,4096 # Switch to mvcle for copies >1MB - jh __memcpy_mvcle + chi %r5,4096 # Switch to mvcle for copies >1MB + jh __memcpy_mvcle .L_Z900_12: - mvc 0(256,%r1),0(%r3) - la %r1,256(%r1) - la %r3,256(%r3) - brctg %r5,.L_Z900_12 - j .L_Z900_3 + mvc 0(256,%r1),0(%r3) + la %r1,256(%r1) + la %r3,256(%r3) + brctg %r5,.L_Z900_12 + j .L_Z900_3 .L_Z900_15: - mvc 0(1,%r1),0(%r3) + mvc 0(1,%r1),0(%r3) #ifdef USE_MULTIARCH -END(__memcpy_z900) +END(__memcpy_default) #else END(memcpy) libc_hidden_builtin_def (memcpy) diff --git a/sysdeps/s390/s390-64/memset.S b/sysdeps/s390/s390-64/memset.S index 1e307d7ec8..cab7855549 100644 --- a/sysdeps/s390/s390-64/memset.S +++ b/sysdeps/s390/s390-64/memset.S @@ -1,5 +1,5 @@ /* Set a block of memory to some byte value. 64 bit S/390 version. - Copyright (C) 2001-2015 Free Software Foundation, Inc. + Copyright (C) 2001-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU C Library. @@ -29,35 +29,35 @@ .text #ifdef USE_MULTIARCH -ENTRY(__memset_z900) +ENTRY(__memset_default) #else ENTRY(memset) #endif .machine "z900" - ltgr %r4,%r4 - je .L_Z900_4 - stc %r3,0(%r2) - cghi %r4,1 - lgr %r1,%r2 - je .L_Z900_4 - aghi %r4,-2 - srlg %r3,%r4,8 - ltgr %r3,%r3 - jne .L_Z900_14 + ltgr %r4,%r4 + je .L_Z900_4 + stc %r3,0(%r2) + cghi %r4,1 + lgr %r1,%r2 + je .L_Z900_4 + aghi %r4,-2 + srlg %r3,%r4,8 + ltgr %r3,%r3 + jne .L_Z900_14 .L_Z900_3: - larl %r3,.L_Z900_18 - ex %r4,0(%r3) + larl %r3,.L_Z900_18 + ex %r4,0(%r3) .L_Z900_4: - br %r14 + br %r14 .L_Z900_14: - mvc 1(256,%r1),0(%r1) - la %r1,256(%r1) - brctg %r3,.L_Z900_14 - j .L_Z900_3 + mvc 1(256,%r1),0(%r1) + la %r1,256(%r1) + brctg %r3,.L_Z900_14 + j .L_Z900_3 .L_Z900_18: - mvc 1(1,%r1),0(%r1) + mvc 1(1,%r1),0(%r1) #ifdef USE_MULTIARCH -END(__memset_z900) +END(__memset_default) #else END(memset) libc_hidden_builtin_def (memset) diff --git a/sysdeps/s390/s390-64/multiarch/Makefile b/sysdeps/s390/s390-64/multiarch/Makefile index 9baeecda64..91053b5364 100644 --- a/sysdeps/s390/s390-64/multiarch/Makefile +++ b/sysdeps/s390/s390-64/multiarch/Makefile @@ -1,3 +1,4 @@ ifeq ($(subdir),string) -sysdep_routines += ifunc-resolve memset memcpy memcmp +sysdep_routines += memset memset-s390x memcpy memcpy-s390x \ + memcmp memcmp-s390x endif diff --git a/sysdeps/s390/s390-64/multiarch/ifunc-resolve.c b/sysdeps/s390/s390-64/multiarch/ifunc-resolve.c deleted file mode 100644 index b303304f31..0000000000 --- a/sysdeps/s390/s390-64/multiarch/ifunc-resolve.c +++ /dev/null @@ -1,76 +0,0 @@ -/* IFUNC resolver function for CPU specific functions. - 64 bit S/390 version. - Copyright (C) 2012-2015 Free Software Foundation, Inc. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <unistd.h> -#include <dl-procinfo.h> - -#define STFLE_BITS_Z10 34 /* General instructions extension */ -#define STFLE_BITS_Z196 45 /* Distinct operands, pop ... */ - -#if IS_IN (libc) - -#define IFUNC_RESOLVE(FUNC) \ - asm (".globl " #FUNC "\n\t" \ - ".type " #FUNC ",@gnu_indirect_function\n\t" \ - ".set " #FUNC ",__resolve_" #FUNC "\n\t" \ - ".globl __GI_" #FUNC "\n\t" \ - ".set __GI_" #FUNC "," #FUNC "\n"); \ - \ - /* Make the declarations of the optimized functions hidden in order - to prevent GOT slots being generated for them. */ \ - extern void *__##FUNC##_z196 attribute_hidden; \ - extern void *__##FUNC##_z10 attribute_hidden; \ - extern void *__##FUNC##_z900 attribute_hidden; \ - \ - void *__resolve_##FUNC (unsigned long int dl_hwcap) \ - { \ - if (dl_hwcap & HWCAP_S390_STFLE) \ - { \ - /* We want just 1 double word to be returned. */ \ - register unsigned long reg0 asm("0") = 0; \ - unsigned long stfle_bits; \ - \ - asm volatile(".machine push" "\n\t" \ - ".machine \"z9-109\"" "\n\t" \ - "stfle %0" "\n\t" \ - ".machine pop" "\n" \ - : "=QS" (stfle_bits), "+d" (reg0) \ - : : "cc"); \ - \ - if ((stfle_bits & (1UL << (63 - STFLE_BITS_Z196))) != 0) \ - return &__##FUNC##_z196; \ - else if ((stfle_bits & (1UL << (63 - STFLE_BITS_Z10))) != 0) \ - return &__##FUNC##_z10; \ - else \ - return &__##FUNC##_z900; \ - } \ - else \ - return &__##FUNC##_z900; \ - } - -IFUNC_RESOLVE(memset) -IFUNC_RESOLVE(memcmp) -asm(".weak bcmp ; bcmp = memcmp"); - -/* In the static lib memcpy is needed before the reloc is resolved. */ -#ifdef SHARED -IFUNC_RESOLVE(memcpy) -#endif - -#endif diff --git a/sysdeps/s390/s390-64/multiarch/memchr.c b/sysdeps/s390/s390-64/multiarch/memchr.c new file mode 100644 index 0000000000..2281e43056 --- /dev/null +++ b/sysdeps/s390/s390-64/multiarch/memchr.c @@ -0,0 +1,21 @@ +/* Multiple versions of memchr. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* This wrapper-file is needed, because otherwise file + sysdeps/s390/s390-[32|64]/memchr.S will be used. */ +#include <sysdeps/s390/multiarch/memchr.c> diff --git a/sysdeps/s390/s390-64/multiarch/memcmp.S b/sysdeps/s390/s390-64/multiarch/memcmp-s390x.S index 049847d9cf..2a4c0ae9a6 100644 --- a/sysdeps/s390/s390-64/multiarch/memcmp.S +++ b/sysdeps/s390/s390-64/multiarch/memcmp-s390x.S @@ -1,5 +1,5 @@ /* CPU specific memcmp implementations. 64 bit S/390 version. - Copyright (C) 2012-2015 Free Software Foundation, Inc. + Copyright (C) 2012-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -31,71 +31,71 @@ ENTRY(__memcmp_z196) .machine "z196" - ltgr %r4,%r4 - je .L_Z196_4 - aghi %r4,-1 - srlg %r1,%r4,8 - ltgr %r1,%r1 - jne .L_Z196_2 + ltgr %r4,%r4 + je .L_Z196_4 + aghi %r4,-1 + srlg %r1,%r4,8 + ltgr %r1,%r1 + jne .L_Z196_2 .L_Z196_3: - exrl %r4,.L_Z196_14 + exrl %r4,.L_Z196_14 .L_Z196_4: - ipm %r2 - sllg %r2,%r2,34 - srag %r2,%r2,62 - br %r14 + ipm %r2 + sllg %r2,%r2,34 + srag %r2,%r2,62 + br %r14 .L_Z196_17: - la %r3,256(%r3) - la %r2,256(%r2) - aghi %r1,-1 - je .L_Z196_3 + la %r3,256(%r3) + la %r2,256(%r2) + aghi %r1,-1 + je .L_Z196_3 .L_Z196_2: - pfd 1,512(%r3) - pfd 1,512(%r2) - clc 0(256,%r3),0(%r2) - je .L_Z196_17 - ipm %r2 - sllg %r2,%r2,34 - srag %r2,%r2,62 - br %r14 + pfd 1,512(%r3) + pfd 1,512(%r2) + clc 0(256,%r3),0(%r2) + je .L_Z196_17 + ipm %r2 + sllg %r2,%r2,34 + srag %r2,%r2,62 + br %r14 .L_Z196_14: - clc 0(1,%r3),0(%r2) + clc 0(1,%r3),0(%r2) END(__memcmp_z196) ENTRY(__memcmp_z10) .machine "z10" - ltgr %r4,%r4 - je .L_Z10_4 - aghi %r4,-1 - srlg %r1,%r4,8 - cgijlh %r1,0,.L_Z10_12 + ltgr %r4,%r4 + je .L_Z10_4 + aghi %r4,-1 + srlg %r1,%r4,8 + cgijlh %r1,0,.L_Z10_12 .L_Z10_3: - exrl %r4,.L_Z10_15 + exrl %r4,.L_Z10_15 .L_Z10_4: - ipm %r2 - sllg %r2,%r2,34 - srag %r2,%r2,62 - br %r14 + ipm %r2 + sllg %r2,%r2,34 + srag %r2,%r2,62 + br %r14 .L_Z10_12: - pfd 1,512(%r3) - pfd 1,512(%r2) - clc 0(256,%r3),0(%r2) - jne .L_Z10_4 - la %r3,256(%r3) - la %r2,256(%r2) - brctg %r1,.L_Z10_12 - j .L_Z10_3 + pfd 1,512(%r3) + pfd 1,512(%r2) + clc 0(256,%r3),0(%r2) + jne .L_Z10_4 + la %r3,256(%r3) + la %r2,256(%r2) + brctg %r1,.L_Z10_12 + j .L_Z10_3 .L_Z10_15: - clc 0(1,%r3),0(%r2) + clc 0(1,%r3),0(%r2) END(__memcmp_z10) -#endif +#endif /* IS_IN (libc) */ #include "../memcmp.S" #if !IS_IN (libc) .globl memcmp -.set memcmp,__memcmp_z900 +.set memcmp,__memcmp_default .weak bcmp -.set bcmp,__memcmp_z900 +.set bcmp,__memcmp_default #endif diff --git a/sysdeps/s390/s390-64/multiarch/memcmp.c b/sysdeps/s390/s390-64/multiarch/memcmp.c new file mode 100644 index 0000000000..44f72dc8ca --- /dev/null +++ b/sysdeps/s390/s390-64/multiarch/memcmp.c @@ -0,0 +1,24 @@ +/* Multiple versions of memcmp. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) +# include <ifunc-resolve.h> + +s390_libc_ifunc (memcmp) +__asm__(".weak bcmp ; bcmp = memcmp"); +#endif diff --git a/sysdeps/s390/s390-64/multiarch/memcpy.S b/sysdeps/s390/s390-64/multiarch/memcpy-s390x.S index fc670c7ac4..69fa562060 100644 --- a/sysdeps/s390/s390-64/multiarch/memcpy.S +++ b/sysdeps/s390/s390-64/multiarch/memcpy-s390x.S @@ -1,5 +1,5 @@ /* CPU specific memcpy implementations. 64 bit S/390 version. - Copyright (C) 2012-2015 Free Software Foundation, Inc. + Copyright (C) 2012-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -30,65 +30,65 @@ #if defined SHARED && IS_IN (libc) ENTRY(__memcpy_z196) - .machine "z196" - ltgr %r4,%r4 - je .L_Z196_4 - aghi %r4,-1 - lgr %r1,%r2 - srlg %r5,%r4,8 - ltgr %r5,%r5 - jne .L_Z196_5 + .machine "z196" + ltgr %r4,%r4 + je .L_Z196_4 + aghi %r4,-1 + lgr %r1,%r2 + srlg %r5,%r4,8 + ltgr %r5,%r5 + jne .L_Z196_5 .L_Z196_3: - exrl %r4,.L_Z196_14 + exrl %r4,.L_Z196_14 .L_Z196_4: - br %r14 + br %r14 .L_Z196_5: - cgfi %r5,262144 # Switch to mvcle for copies >64MB - jh __memcpy_mvcle + cgfi %r5,262144 # Switch to mvcle for copies >64MB + jh __memcpy_mvcle .L_Z196_2: - pfd 1,768(%r3) - pfd 2,768(%r1) - mvc 0(256,%r1),0(%r3) - aghi %r5,-1 - la %r1,256(%r1) - la %r3,256(%r3) - jne .L_Z196_2 - j .L_Z196_3 + pfd 1,768(%r3) + pfd 2,768(%r1) + mvc 0(256,%r1),0(%r3) + aghi %r5,-1 + la %r1,256(%r1) + la %r3,256(%r3) + jne .L_Z196_2 + j .L_Z196_3 .L_Z196_14: - mvc 0(1,%r1),0(%r3) + mvc 0(1,%r1),0(%r3) END(__memcpy_z196) ENTRY(__memcpy_z10) .machine "z10" - cgije %r4,0,.L_Z10_4 - aghi %r4,-1 - lgr %r1,%r2 - srlg %r5,%r4,8 - cgijlh %r5,0,.L_Z10_13 + cgije %r4,0,.L_Z10_4 + aghi %r4,-1 + lgr %r1,%r2 + srlg %r5,%r4,8 + cgijlh %r5,0,.L_Z10_13 .L_Z10_3: - exrl %r4,.L_Z10_15 + exrl %r4,.L_Z10_15 .L_Z10_4: - br %r14 + br %r14 .L_Z10_13: - cgfi %r5,65535 # Switch to mvcle for copies >16MB - jh __memcpy_mvcle + cgfi %r5,65535 # Switch to mvcle for copies >16MB + jh __memcpy_mvcle .L_Z10_12: - pfd 1,768(%r3) - pfd 2,768(%r1) - mvc 0(256,%r1),0(%r3) - la %r1,256(%r1) - la %r3,256(%r3) - brctg %r5,.L_Z10_12 - j .L_Z10_3 + pfd 1,768(%r3) + pfd 2,768(%r1) + mvc 0(256,%r1),0(%r3) + la %r1,256(%r1) + la %r3,256(%r3) + brctg %r5,.L_Z10_12 + j .L_Z10_3 .L_Z10_15: - mvc 0(1,%r1),0(%r3) + mvc 0(1,%r1),0(%r3) END(__memcpy_z10) -#endif +#endif /* SHARED && IS_IN (libc) */ #include "../memcpy.S" #if !defined SHARED || !IS_IN (libc) .globl memcpy -.set memcpy,__memcpy_z900 +.set memcpy,__memcpy_default #endif diff --git a/sysdeps/s390/s390-64/multiarch/memcpy.c b/sysdeps/s390/s390-64/multiarch/memcpy.c new file mode 100644 index 0000000000..2a98aa0b82 --- /dev/null +++ b/sysdeps/s390/s390-64/multiarch/memcpy.c @@ -0,0 +1,24 @@ +/* Multiple versions of memcpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* In the static lib memcpy is needed before the reloc is resolved. */ +#if defined SHARED && IS_IN (libc) +# include <ifunc-resolve.h> + +s390_libc_ifunc (memcpy) +#endif diff --git a/sysdeps/s390/s390-64/multiarch/memset.S b/sysdeps/s390/s390-64/multiarch/memset-s390x.S index 3ac110a7e0..05e068279d 100644 --- a/sysdeps/s390/s390-64/multiarch/memset.S +++ b/sysdeps/s390/s390-64/multiarch/memset-s390x.S @@ -1,5 +1,5 @@ /* Set a block of memory to some byte value. 64 bit S/390 version. - Copyright (C) 2012-2015 Free Software Foundation, Inc. + Copyright (C) 2012-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -31,58 +31,58 @@ ENTRY(__memset_z196) .machine "z196" - ltgr %r4,%r4 - je .L_Z196_4 - stc %r3,0(%r2) - lgr %r1,%r2 - cghi %r4,1 - je .L_Z196_4 - aghi %r4,-2 - srlg %r5,%r4,8 - ltgr %r5,%r5 - jne .L_Z196_1 + ltgr %r4,%r4 + je .L_Z196_4 + stc %r3,0(%r2) + lgr %r1,%r2 + cghi %r4,1 + je .L_Z196_4 + aghi %r4,-2 + srlg %r5,%r4,8 + ltgr %r5,%r5 + jne .L_Z196_1 .L_Z196_3: - exrl %r4,.L_Z196_17 + exrl %r4,.L_Z196_17 .L_Z196_4: - br %r14 + br %r14 .L_Z196_1: cgfi %r5,1048576 jh __memset_mvcle # Switch to mvcle for >256MB .L_Z196_2: - pfd 2,1024(%r1) - mvc 1(256,%r1),0(%r1) - aghi %r5,-1 - la %r1,256(%r1) - jne .L_Z196_2 - j .L_Z196_3 + pfd 2,1024(%r1) + mvc 1(256,%r1),0(%r1) + aghi %r5,-1 + la %r1,256(%r1) + jne .L_Z196_2 + j .L_Z196_3 .L_Z196_17: - mvc 1(1,%r1),0(%r1) + mvc 1(1,%r1),0(%r1) END(__memset_z196) ENTRY(__memset_z10) .machine "z10" - cgije %r4,0,.L_Z10_4 - stc %r3,0(%r2) - lgr %r1,%r2 - cgije %r4,1,.L_Z10_4 - aghi %r4,-2 - srlg %r5,%r4,8 - cgijlh %r5,0,.L_Z10_15 + cgije %r4,0,.L_Z10_4 + stc %r3,0(%r2) + lgr %r1,%r2 + cgije %r4,1,.L_Z10_4 + aghi %r4,-2 + srlg %r5,%r4,8 + cgijlh %r5,0,.L_Z10_15 .L_Z10_3: - exrl %r4,.L_Z10_18 + exrl %r4,.L_Z10_18 .L_Z10_4: - br %r14 + br %r14 .L_Z10_15: cgfi %r5,163840 # Switch to mvcle for >40MB jh __memset_mvcle .L_Z10_14: - pfd 2,1024(%r1) - mvc 1(256,%r1),0(%r1) - la %r1,256(%r1) - brctg %r5,.L_Z10_14 - j .L_Z10_3 + pfd 2,1024(%r1) + mvc 1(256,%r1),0(%r1) + la %r1,256(%r1) + brctg %r5,.L_Z10_14 + j .L_Z10_3 .L_Z10_18: - mvc 1(1,%r1),0(%r1) + mvc 1(1,%r1),0(%r1) END(__memset_z10) ENTRY(__memset_mvcle) @@ -99,11 +99,11 @@ ENTRY(__memset_mvcle) br %r14 END(__memset_mvcle) -#endif +#endif /* IS_IN (libc) */ #include "../memset.S" #if !IS_IN (libc) .globl memset -.set memset,__memset_z900 +.set memset,__memset_default #endif diff --git a/sysdeps/s390/s390-64/multiarch/memset.c b/sysdeps/s390/s390-64/multiarch/memset.c new file mode 100644 index 0000000000..89b8102f2a --- /dev/null +++ b/sysdeps/s390/s390-64/multiarch/memset.c @@ -0,0 +1,23 @@ +/* Multiple versions of memset. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) +# include <ifunc-resolve.h> + +s390_libc_ifunc (memset) +#endif diff --git a/sysdeps/s390/s390-64/multiarch/strcmp.c b/sysdeps/s390/s390-64/multiarch/strcmp.c new file mode 100644 index 0000000000..b7eebc017f --- /dev/null +++ b/sysdeps/s390/s390-64/multiarch/strcmp.c @@ -0,0 +1,21 @@ +/* Multiple versions of strcmp. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* This wrapper-file is needed, because otherwise file + sysdeps/s390/s390-[32|64]/strcmp.S will be used. */ +#include <sysdeps/s390/multiarch/strcmp.c> diff --git a/sysdeps/s390/s390-64/multiarch/strcpy.c b/sysdeps/s390/s390-64/multiarch/strcpy.c new file mode 100644 index 0000000000..ae140d22b7 --- /dev/null +++ b/sysdeps/s390/s390-64/multiarch/strcpy.c @@ -0,0 +1,21 @@ +/* Multiple versions of strcpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* This wrapper-file is needed, because otherwise file + sysdeps/s390/s390-[32|64]/strcpy.S will be used. */ +#include <sysdeps/s390/multiarch/strcpy.c> diff --git a/sysdeps/s390/s390-64/multiarch/strncpy.c b/sysdeps/s390/s390-64/multiarch/strncpy.c new file mode 100644 index 0000000000..28a2af72e4 --- /dev/null +++ b/sysdeps/s390/s390-64/multiarch/strncpy.c @@ -0,0 +1,21 @@ +/* Multiple versions of strncpy. + Copyright (C) 2015-2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* This wrapper-file is needed, because otherwise file + sysdeps/s390/s390-[32|64]/strncpy.S will be used. */ +#include <sysdeps/s390/multiarch/strncpy.c> diff --git a/sysdeps/s390/s390-64/s390x-mcount.S b/sysdeps/s390/s390-64/s390x-mcount.S index dec92af012..cb67ddb7ff 100644 --- a/sysdeps/s390/s390-64/s390x-mcount.S +++ b/sysdeps/s390/s390-64/s390x-mcount.S @@ -1,5 +1,5 @@ /* 64 bit S/390-specific implemetation of profiling support. - Copyright (C) 2001-2015 Free Software Foundation, Inc. + Copyright (C) 2001-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com) This file is part of the GNU C Library. diff --git a/sysdeps/s390/s390-64/setjmp.S b/sysdeps/s390/s390-64/setjmp.S index 330a8e6c62..bbcb70db5f 100644 --- a/sysdeps/s390/s390-64/setjmp.S +++ b/sysdeps/s390/s390-64/setjmp.S @@ -1,5 +1,5 @@ /* setjmp for 64 bit S/390, ELF version. - Copyright (C) 2001-2015 Free Software Foundation, Inc. + Copyright (C) 2001-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU C Library. @@ -27,29 +27,31 @@ #if !IS_IN (rtld) # if defined SHARED && SHLIB_COMPAT (libc, GLIBC_2_19, GLIBC_2_20) /* we need a unique name in case of symbol versioning. */ +# define setjmp __v1setjmp +# define _setjmp __v1_setjmp # define __sigsetjmp __v1__sigsetjmp + +# undef libc_hidden_def +# define libc_hidden_def(name) strong_alias(_setjmp, __GI__setjmp) # endif /* if defined SHARED && SHLIB_COMPAT (libc, GLIBC_2_19, GLIBC_2_20) */ #endif /* !IS_IN (rtld) */ - /* We include the BSD entry points here as well but we make - them weak. */ + /* We include the BSD entry points here as well. */ ENTRY (setjmp) - .weak C_SYMBOL_NAME (setjmp) - lghi %r3,1 /* Second argument of one. */ - j .Linternal_sigsetjmp /* Branch relativ to __sigsetjmp. */ + lghi %r3,1 /* Second argument of one. */ + j .Linternal_sigsetjmp /* Branch relativ to __sigsetjmp. */ END (setjmp) /* Binary compatibility entry point. */ ENTRY(_setjmp) - .weak C_SYMBOL_NAME (_setjmp) - slgr %r3,%r3 /* Second argument of zero. */ - j .Linternal_sigsetjmp /* Branch relativ to __sigsetjmp. */ + slgr %r3,%r3 /* Second argument of zero. */ + j .Linternal_sigsetjmp /* Branch relativ to __sigsetjmp. */ END (_setjmp) libc_hidden_def (_setjmp) ENTRY(__setjmp) - slgr %r3,%r3 /* Second argument of zero. */ - j .Linternal_sigsetjmp /* Branch relativ to __sigsetjmp. */ + slgr %r3,%r3 /* Second argument of zero. */ + j .Linternal_sigsetjmp /* Branch relativ to __sigsetjmp. */ END (__setjmp) ENTRY(__sigsetjmp) @@ -65,7 +67,7 @@ ENTRY(__sigsetjmp) PTR_MANGLE2 (%r5, %r1) stmg %r4,%r5,64(%r2) #else - stmg %r6,%r15,0(%r2) /* Store registers in jmp_buf. */ + stmg %r6,%r15,0(%r2) /* Store registers in jmp_buf. */ #endif std %f8,80(%r2) std %f9,88(%r2) @@ -80,7 +82,7 @@ ENTRY(__sigsetjmp) lghi %r2,0 br %r14 #elif defined PIC - jg __sigjmp_save@PLT /* Branch to PLT of __sigsetjmp. */ + jg __sigjmp_save@PLT /* Branch to PLT of __sigsetjmp. */ #else jg __sigjmp_save #endif @@ -91,15 +93,15 @@ END (__sigsetjmp) /* In glibc release 2.19 new versions of setjmp-functions were introduced, but were reverted before 2.20. Thus both versions are the same function. */ +# undef setjmp +# undef _setjmp # undef __sigsetjmp -weak_alias (setjmp, __v1setjmp); -weak_alias (setjmp, __v2setjmp); +strong_alias (__v1setjmp, __v2setjmp); versioned_symbol (libc, __v1setjmp, setjmp, GLIBC_2_0); compat_symbol (libc, __v2setjmp, setjmp, GLIBC_2_19); -weak_alias (_setjmp, __v1_setjmp); -weak_alias (_setjmp, __v2_setjmp); +strong_alias (__v1_setjmp, __v2_setjmp); versioned_symbol (libc, __v1_setjmp, _setjmp, GLIBC_2_0); compat_symbol (libc, __v2_setjmp, _setjmp, GLIBC_2_19); diff --git a/sysdeps/s390/s390-64/stackguard-macros.h b/sysdeps/s390/s390-64/stackguard-macros.h index c8270fbe79..2c97d3824f 100644 --- a/sysdeps/s390/s390-64/stackguard-macros.h +++ b/sysdeps/s390/s390-64/stackguard-macros.h @@ -1,18 +1,18 @@ #include <stdint.h> #define STACK_CHK_GUARD \ - ({ uintptr_t x; asm ("ear %0,%%a0; sllg %0,%0,32; ear %0,%%a1; lg %0,0x28(%0)" : "=a" (x)); x; }) + ({ uintptr_t x; __asm__ ("ear %0,%%a0; sllg %0,%0,32; ear %0,%%a1; lg %0,0x28(%0)" : "=a" (x)); x; }) /* On s390/s390x there is no unique pointer guard, instead we use the same value as the stack guard. */ -#define POINTER_CHK_GUARD \ - ({ \ - uintptr_t x; \ - asm ("ear %0,%%a0;" \ - "sllg %0,%0,32;" \ - "ear %0,%%a1;" \ - "lg %0,%1(%0)" \ - : "=a" (x) \ - : "i" (offsetof (tcbhead_t, stack_guard))); \ - x; \ - }) +#define POINTER_CHK_GUARD \ + ({ \ + uintptr_t x; \ + __asm__ ("ear %0,%%a0;" \ + "sllg %0,%0,32;" \ + "ear %0,%%a1;" \ + "lg %0,%1(%0)" \ + : "=a" (x) \ + : "i" (offsetof (tcbhead_t, stack_guard))); \ + x; \ + }) diff --git a/sysdeps/s390/s390-64/start.S b/sysdeps/s390/s390-64/start.S index 48c11b13e5..e261460dd5 100644 --- a/sysdeps/s390/s390-64/start.S +++ b/sysdeps/s390/s390-64/start.S @@ -1,5 +1,5 @@ /* Startup code compliant to the 64 bit S/390 ELF ABI. - Copyright (C) 2001-2015 Free Software Foundation, Inc. + Copyright (C) 2001-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU C Library. diff --git a/sysdeps/s390/s390-64/strcmp.S b/sysdeps/s390/s390-64/strcmp.S index 7b4be304ec..245b54cc9d 100644 --- a/sysdeps/s390/s390-64/strcmp.S +++ b/sysdeps/s390/s390-64/strcmp.S @@ -1,6 +1,6 @@ /* strcmp - compare two string. 64 bit S/390 version. This file is part of the GNU C Library. - Copyright (C) 2001-2015 Free Software Foundation, Inc. + Copyright (C) 2001-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/s390-64/strcpy.S b/sysdeps/s390/s390-64/strcpy.S index 8ad0aeefd6..9864e98b24 100644 --- a/sysdeps/s390/s390-64/strcpy.S +++ b/sysdeps/s390/s390-64/strcpy.S @@ -1,5 +1,5 @@ /* strcpy - copy a string from source to destination. 64 bit S/390 version. - Copyright (C) 2001-2015 Free Software Foundation, Inc. + Copyright (C) 2001-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU C Library. diff --git a/sysdeps/s390/s390-64/strncpy.S b/sysdeps/s390/s390-64/strncpy.S index 2d433078ec..56c8a526ae 100644 --- a/sysdeps/s390/s390-64/strncpy.S +++ b/sysdeps/s390/s390-64/strncpy.S @@ -1,6 +1,6 @@ /* strncpy - copy at most n characters from a string from source to destination. 64 bit S/390 version - Copyright (C) 2001-2015 Free Software Foundation, Inc. + Copyright (C) 2001-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU C Library. diff --git a/sysdeps/s390/s390-64/sub_n.S b/sysdeps/s390/s390-64/sub_n.S index 5a0ac3021f..7318836db9 100644 --- a/sysdeps/s390/s390-64/sub_n.S +++ b/sysdeps/s390/s390-64/sub_n.S @@ -1,6 +1,6 @@ /* __mpn_sub_n -- Add two limb vectors of the same length > 0 and store sum in a third limb vector. 64 bit S/390 version. - Copyright (C) 2001-2015 Free Software Foundation, Inc. + Copyright (C) 2001-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU MP Library. diff --git a/sysdeps/s390/s390-64/sysdep.h b/sysdeps/s390/s390-64/sysdep.h index c70ef7e2b1..7fac89da51 100644 --- a/sysdeps/s390/s390-64/sysdep.h +++ b/sysdeps/s390/s390-64/sysdep.h @@ -1,5 +1,5 @@ /* Assembler macros for 64 bit S/390. - Copyright (C) 2001-2015 Free Software Foundation, Inc. + Copyright (C) 2001-2016 Free Software Foundation, Inc. Contributed by Martin Schwidefsky (schwidefsky@de.ibm.com). This file is part of the GNU C Library. diff --git a/sysdeps/s390/s390-64/tls-macros.h b/sysdeps/s390/s390-64/tls-macros.h index 3c59436341..d70ea6ce0c 100644 --- a/sysdeps/s390/s390-64/tls-macros.h +++ b/sysdeps/s390/s390-64/tls-macros.h @@ -1,88 +1,88 @@ #define TLS_LE(x) \ ({ unsigned long __offset; \ - asm ("bras %0,1f\n" \ - "0:\t.quad " #x "@ntpoff\n" \ - "1:\tlg %0,0(%0)" \ - : "=a" (__offset) : : "cc" ); \ + __asm__ ("bras %0,1f\n" \ + "0:\t.quad " #x "@ntpoff\n" \ + "1:\tlg %0,0(%0)" \ + : "=a" (__offset) : : "cc" ); \ (int *) (__builtin_thread_pointer() + __offset); }) #ifdef PIC # define TLS_IE(x) \ ({ unsigned long __offset, __got; \ - asm ("bras %0,0f\n\t" \ - ".quad " #x "@gotntpoff\n" \ - "0:\tlarl %1,_GLOBAL_OFFSET_TABLE_\n\t" \ - "lg %0,0(%0)\n\t" \ - "lg %0,0(%0,%1):tls_load:" #x "\n" \ - : "=&a" (__offset), "=&a" (__got) : : "cc" ); \ + __asm__ ("bras %0,0f\n\t" \ + ".quad " #x "@gotntpoff\n" \ + "0:\tlarl %1,_GLOBAL_OFFSET_TABLE_\n\t" \ + "lg %0,0(%0)\n\t" \ + "lg %0,0(%0,%1):tls_load:" #x "\n" \ + : "=&a" (__offset), "=&a" (__got) : : "cc" ); \ (int *) (__builtin_thread_pointer() + __offset); }) #else # define TLS_IE(x) \ ({ unsigned long __offset; \ - asm ("bras %0,1f\n" \ - "0:\t.quad " #x "@indntpoff\n" \ - "1:\t lg %0,0(%0)\n\t" \ - "lg %0,0(%0):tls_load:" #x \ - : "=&a" (__offset) : : "cc" ); \ + __asm__ ("bras %0,1f\n" \ + "0:\t.quad " #x "@indntpoff\n" \ + "1:\t lg %0,0(%0)\n\t" \ + "lg %0,0(%0):tls_load:" #x \ + : "=&a" (__offset) : : "cc" ); \ (int *) (__builtin_thread_pointer() + __offset); }) #endif #ifdef PIC # define TLS_LD(x) \ ({ unsigned long __offset, __save12; \ - asm ("bras %0,1f\n" \ - "0:\t.quad " #x "@tlsldm\n\t" \ - ".quad " #x "@dtpoff\n" \ - "1:\tlgr %1,%%r12\n\t" \ - "larl %%r12,_GLOBAL_OFFSET_TABLE_\n\t" \ - "lg %%r2,0(%0)\n\t" \ - "brasl %%r14,__tls_get_offset@plt:tls_ldcall:" #x "\n\t" \ - "lg %0,8(%0)\n\t" \ - "algr %0,%%r2\n\t" \ - "lgr %%r12,%1" \ - : "=&a" (__offset), "=&a" (__save12) \ - : : "cc", "0", "1", "2", "3", "4", "5", "14" ); \ + __asm__ ("bras %0,1f\n" \ + "0:\t.quad " #x "@tlsldm\n\t" \ + ".quad " #x "@dtpoff\n" \ + "1:\tlgr %1,%%r12\n\t" \ + "larl %%r12,_GLOBAL_OFFSET_TABLE_\n\t" \ + "lg %%r2,0(%0)\n\t" \ + "brasl %%r14,__tls_get_offset@plt:tls_ldcall:" #x "\n\t" \ + "lg %0,8(%0)\n\t" \ + "algr %0,%%r2\n\t" \ + "lgr %%r12,%1" \ + : "=&a" (__offset), "=&a" (__save12) \ + : : "cc", "0", "1", "2", "3", "4", "5", "14" ); \ (int *) (__builtin_thread_pointer() + __offset); }) #else # define TLS_LD(x) \ ({ unsigned long __offset; \ - asm ("bras %0,1f\n" \ - "0:\t.quad " #x "@tlsldm\n\t" \ - ".quad " #x "@dtpoff\n" \ - "1:\tlarl %%r12,_GLOBAL_OFFSET_TABLE_\n\t" \ - "lg %%r2,0(%0)\n\t" \ - "brasl %%r14,__tls_get_offset@plt:tls_ldcall:" #x "\n\t" \ - "lg %0,8(%0)\n\t" \ - "algr %0,%%r2" \ - : "=&a" (__offset) \ - : : "cc", "0", "1", "2", "3", "4", "5", "12", "14" ); \ + __asm__ ("bras %0,1f\n" \ + "0:\t.quad " #x "@tlsldm\n\t" \ + ".quad " #x "@dtpoff\n" \ + "1:\tlarl %%r12,_GLOBAL_OFFSET_TABLE_\n\t" \ + "lg %%r2,0(%0)\n\t" \ + "brasl %%r14,__tls_get_offset@plt:tls_ldcall:" #x "\n\t" \ + "lg %0,8(%0)\n\t" \ + "algr %0,%%r2" \ + : "=&a" (__offset) \ + : : "cc", "0", "1", "2", "3", "4", "5", "12", "14" ); \ (int *) (__builtin_thread_pointer() + __offset); }) #endif #ifdef PIC # define TLS_GD(x) \ ({ unsigned long __offset, __save12; \ - asm ("bras %0,1f\n" \ - "0:\t.quad " #x "@tlsgd\n" \ - "1:\tlgr %1,%%r12\n\t" \ - "larl %%r12,_GLOBAL_OFFSET_TABLE_\n\t" \ - "lg %%r2,0(%0)\n\t" \ - "brasl %%r14,__tls_get_offset@plt:tls_gdcall:" #x "\n\t" \ - "lgr %0,%%r2\n\t" \ - "lgr %%r12,%1" \ - : "=&a" (__offset), "=&a" (__save12) \ - : : "cc", "0", "1", "2", "3", "4", "5", "14" ); \ + __asm__ ("bras %0,1f\n" \ + "0:\t.quad " #x "@tlsgd\n" \ + "1:\tlgr %1,%%r12\n\t" \ + "larl %%r12,_GLOBAL_OFFSET_TABLE_\n\t" \ + "lg %%r2,0(%0)\n\t" \ + "brasl %%r14,__tls_get_offset@plt:tls_gdcall:" #x "\n\t" \ + "lgr %0,%%r2\n\t" \ + "lgr %%r12,%1" \ + : "=&a" (__offset), "=&a" (__save12) \ + : : "cc", "0", "1", "2", "3", "4", "5", "14" ); \ (int *) (__builtin_thread_pointer() + __offset); }) #else # define TLS_GD(x) \ ({ unsigned long __offset; \ - asm ("bras %0,1f\n" \ - "0:\t.quad " #x "@tlsgd\n" \ - "1:\tlarl %%r12,_GLOBAL_OFFSET_TABLE_\n\t" \ - "lg %%r2,0(%0)\n\t" \ - "brasl %%r14,__tls_get_offset@plt:tls_gdcall:" #x "\n\t" \ - "lgr %0,%%r2" \ - : "=&a" (__offset) \ - : : "cc", "0", "1", "2", "3", "4", "5", "12", "14" ); \ + __asm__ ("bras %0,1f\n" \ + "0:\t.quad " #x "@tlsgd\n" \ + "1:\tlarl %%r12,_GLOBAL_OFFSET_TABLE_\n\t" \ + "lg %%r2,0(%0)\n\t" \ + "brasl %%r14,__tls_get_offset@plt:tls_gdcall:" #x "\n\t" \ + "lgr %0,%%r2" \ + : "=&a" (__offset) \ + : : "cc", "0", "1", "2", "3", "4", "5", "12", "14" ); \ (int *) (__builtin_thread_pointer() + __offset); }) #endif diff --git a/sysdeps/s390/s390-64/tst-audit.h b/sysdeps/s390/s390-64/tst-audit.h index 2c5dbe1d0b..3283e95037 100644 --- a/sysdeps/s390/s390-64/tst-audit.h +++ b/sysdeps/s390/s390-64/tst-audit.h @@ -1,6 +1,6 @@ /* Definitions for testing PLT entry/exit auditing. S/390 64-bit version. - Copyright (C) 2012-2015 Free Software Foundation, Inc. + Copyright (C) 2012-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. diff --git a/sysdeps/s390/s390-64/utf16-utf32-z9.c b/sysdeps/s390/s390-64/utf16-utf32-z9.c index f887c34e61..a3863ee244 100644 --- a/sysdeps/s390/s390-64/utf16-utf32-z9.c +++ b/sysdeps/s390/s390-64/utf16-utf32-z9.c @@ -2,7 +2,7 @@ This module uses the Z9-109 variants of the Convert Unicode instructions. - Copyright (C) 1997-2015 Free Software Foundation, Inc. + Copyright (C) 1997-2016 Free Software Foundation, Inc. Author: Andreas Krebbel <Andreas.Krebbel@de.ibm.com> Based on the work by Ulrich Drepper <drepper@cygnus.com>, 1997. @@ -163,22 +163,22 @@ gconv_end (struct __gconv_step *data) directions. */ #define HARDWARE_CONVERT(INSTRUCTION) \ { \ - register const unsigned char* pInput asm ("8") = inptr; \ - register unsigned long long inlen asm ("9") = inend - inptr; \ - register unsigned char* pOutput asm ("10") = outptr; \ - register unsigned long long outlen asm("11") = outend - outptr; \ + register const unsigned char* pInput __asm__ ("8") = inptr; \ + register unsigned long long inlen __asm__ ("9") = inend - inptr; \ + register unsigned char* pOutput __asm__ ("10") = outptr; \ + register unsigned long long outlen __asm__("11") = outend - outptr; \ uint64_t cc = 0; \ \ - asm volatile (".machine push \n\t" \ - ".machine \"z9-109\" \n\t" \ - "0: " INSTRUCTION " \n\t" \ - ".machine pop \n\t" \ - " jo 0b \n\t" \ - " ipm %2 \n" \ - : "+a" (pOutput), "+a" (pInput), "+d" (cc), \ - "+d" (outlen), "+d" (inlen) \ - : \ - : "cc", "memory"); \ + __asm__ volatile (".machine push \n\t" \ + ".machine \"z9-109\" \n\t" \ + "0: " INSTRUCTION " \n\t" \ + ".machine pop \n\t" \ + " jo 0b \n\t" \ + " ipm %2 \n" \ + : "+a" (pOutput), "+a" (pInput), "+d" (cc), \ + "+d" (outlen), "+d" (inlen) \ + : \ + : "cc", "memory"); \ \ inptr = pInput; \ outptr = pOutput; \ diff --git a/sysdeps/s390/s390-64/utf8-utf16-z9.c b/sysdeps/s390/s390-64/utf8-utf16-z9.c index 1425cb116a..4148ed796b 100644 --- a/sysdeps/s390/s390-64/utf8-utf16-z9.c +++ b/sysdeps/s390/s390-64/utf8-utf16-z9.c @@ -2,7 +2,7 @@ This module uses the Z9-109 variants of the Convert Unicode instructions. - Copyright (C) 1997-2015 Free Software Foundation, Inc. + Copyright (C) 1997-2016 Free Software Foundation, Inc. Author: Andreas Krebbel <Andreas.Krebbel@de.ibm.com> Based on the work by Ulrich Drepper <drepper@cygnus.com>, 1997. @@ -145,22 +145,22 @@ gconv_end (struct __gconv_step *data) directions. */ #define HARDWARE_CONVERT(INSTRUCTION) \ { \ - register const unsigned char* pInput asm ("8") = inptr; \ - register unsigned long long inlen asm ("9") = inend - inptr; \ - register unsigned char* pOutput asm ("10") = outptr; \ - register unsigned long long outlen asm("11") = outend - outptr; \ + register const unsigned char* pInput __asm__ ("8") = inptr; \ + register unsigned long long inlen __asm__ ("9") = inend - inptr; \ + register unsigned char* pOutput __asm__ ("10") = outptr; \ + register unsigned long long outlen __asm__("11") = outend - outptr; \ uint64_t cc = 0; \ \ - asm volatile (".machine push \n\t" \ - ".machine \"z9-109\" \n\t" \ - "0: " INSTRUCTION " \n\t" \ - ".machine pop \n\t" \ - " jo 0b \n\t" \ - " ipm %2 \n" \ - : "+a" (pOutput), "+a" (pInput), "+d" (cc), \ - "+d" (outlen), "+d" (inlen) \ - : \ - : "cc", "memory"); \ + __asm__ volatile (".machine push \n\t" \ + ".machine \"z9-109\" \n\t" \ + "0: " INSTRUCTION " \n\t" \ + ".machine pop \n\t" \ + " jo 0b \n\t" \ + " ipm %2 \n" \ + : "+a" (pOutput), "+a" (pInput), "+d" (cc), \ + "+d" (outlen), "+d" (inlen) \ + : \ + : "cc", "memory"); \ \ inptr = pInput; \ outptr = pOutput; \ @@ -183,6 +183,7 @@ gconv_end (struct __gconv_step *data) #define MIN_NEEDED_INPUT MIN_NEEDED_FROM #define MAX_NEEDED_INPUT MAX_NEEDED_FROM #define MIN_NEEDED_OUTPUT MIN_NEEDED_TO +#define MAX_NEEDED_OUTPUT MAX_NEEDED_TO #define LOOPFCT FROM_LOOP /* The software implementation is based on the code in gconv_simple.c. */ #define BODY \ @@ -340,6 +341,7 @@ gconv_end (struct __gconv_step *data) /* Conversion from UTF-16 to UTF-8. */ #define MIN_NEEDED_INPUT MIN_NEEDED_TO +#define MAX_NEEDED_INPUT MAX_NEEDED_TO #define MIN_NEEDED_OUTPUT MIN_NEEDED_FROM #define MAX_NEEDED_OUTPUT MAX_NEEDED_FROM #define LOOPFCT TO_LOOP diff --git a/sysdeps/s390/s390-64/utf8-utf32-z9.c b/sysdeps/s390/s390-64/utf8-utf32-z9.c index 9a74448285..defd47d251 100644 --- a/sysdeps/s390/s390-64/utf8-utf32-z9.c +++ b/sysdeps/s390/s390-64/utf8-utf32-z9.c @@ -2,7 +2,7 @@ This module uses the Z9-109 variants of the Convert Unicode instructions. - Copyright (C) 1997-2015 Free Software Foundation, Inc. + Copyright (C) 1997-2016 Free Software Foundation, Inc. Author: Andreas Krebbel <Andreas.Krebbel@de.ibm.com> Based on the work by Ulrich Drepper <drepper@cygnus.com>, 1997. @@ -149,22 +149,22 @@ gconv_end (struct __gconv_step *data) directions. */ #define HARDWARE_CONVERT(INSTRUCTION) \ { \ - register const unsigned char* pInput asm ("8") = inptr; \ - register unsigned long long inlen asm ("9") = inend - inptr; \ - register unsigned char* pOutput asm ("10") = outptr; \ - register unsigned long long outlen asm("11") = outend - outptr; \ + register const unsigned char* pInput __asm__ ("8") = inptr; \ + register unsigned long long inlen __asm__ ("9") = inend - inptr; \ + register unsigned char* pOutput __asm__ ("10") = outptr; \ + register unsigned long long outlen __asm__("11") = outend - outptr; \ uint64_t cc = 0; \ \ - asm volatile (".machine push \n\t" \ - ".machine \"z9-109\" \n\t" \ - "0: " INSTRUCTION " \n\t" \ - ".machine pop \n\t" \ - " jo 0b \n\t" \ - " ipm %2 \n" \ - : "+a" (pOutput), "+a" (pInput), "+d" (cc), \ - "+d" (outlen), "+d" (inlen) \ - : \ - : "cc", "memory"); \ + __asm__ volatile (".machine push \n\t" \ + ".machine \"z9-109\" \n\t" \ + "0: " INSTRUCTION " \n\t" \ + ".machine pop \n\t" \ + " jo 0b \n\t" \ + " ipm %2 \n" \ + : "+a" (pOutput), "+a" (pInput), "+d" (cc), \ + "+d" (outlen), "+d" (inlen) \ + : \ + : "cc", "memory"); \ \ inptr = pInput; \ outptr = pOutput; \ diff --git a/sysdeps/s390/sotruss-lib.c b/sysdeps/s390/sotruss-lib.c index be790ddd22..8c53bc5b79 100644 --- a/sysdeps/s390/sotruss-lib.c +++ b/sysdeps/s390/sotruss-lib.c @@ -1,5 +1,5 @@ /* Override generic sotruss-lib.c to define actual functions for s390. - Copyright (C) 2012-2015 Free Software Foundation, Inc. + Copyright (C) 2012-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. diff --git a/sysdeps/s390/stackinfo.h b/sysdeps/s390/stackinfo.h index 5eb4190987..0d4b70abd7 100644 --- a/sysdeps/s390/stackinfo.h +++ b/sysdeps/s390/stackinfo.h @@ -1,4 +1,4 @@ -/* Copyright (C) 2000-2015 Free Software Foundation, Inc. +/* Copyright (C) 2000-2016 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or diff --git a/sysdeps/s390/string_private.h b/sysdeps/s390/string_private.h new file mode 100644 index 0000000000..9e11eee3dc --- /dev/null +++ b/sysdeps/s390/string_private.h @@ -0,0 +1,20 @@ +/* Define _STRING_ARCH_unaligned. S/390 version. + Copyright (C) 2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* The s390 processors can access unaligned multi-byte variables. */ +#define _STRING_ARCH_unaligned 1 |