summaryrefslogtreecommitdiff
path: root/arch/x86/boot/compressed
AgeCommit message (Collapse)Author
2020-09-07x86/boot/compressed/64: Add IDT InfrastructureJoerg Roedel
Add code needed to setup an IDT in the early pre-decompression boot-code. The IDT is loaded first in startup_64, which is after EfiExitBootServices() has been called, and later reloaded when the kernel image has been relocated to the end of the decompression area. This allows to setup different IDT handlers before and after the relocation. Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Borislav Petkov <bp@suse.de> Link: https://lkml.kernel.org/r/20200907131613.12703-14-joro@8bytes.org
2020-09-07x86/boot/compressed/64: Disable red-zone usageJoerg Roedel
The x86-64 ABI defines a red-zone on the stack: The 128-byte area beyond the location pointed to by %rsp is considered to be reserved and shall not be modified by signal or interrupt handlers. Therefore, functions may use this area for temporary data that is not needed across function calls. In particular, leaf functions may use this area for their entire stack frame, rather than adjusting the stack pointer in the prologue and epilogue. This area is known as the red zone. This is not compatible with exception handling, because the IRET frame written by the hardware at the stack pointer and the functions to handle the exception will overwrite the temporary variables of the interrupted function, causing undefined behavior. So disable red-zones for the pre-decompression boot code. Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Borislav Petkov <bp@suse.de> Reviewed-by: Kees Cook <keescook@chromium.org> Link: https://lkml.kernel.org/r/20200907131613.12703-13-joro@8bytes.org
2020-09-07Merge 'x86/kaslr' to pick up dependent bitsBorislav Petkov
Signed-off-by: Borislav Petkov <bp@suse.de>
2020-09-03x86/boot/compressed: Warn on orphan section placementKees Cook
We don't want to depend on the linker's orphan section placement heuristics as these can vary between linkers, and may change between versions. All sections need to be explicitly handled in the linker script. Now that all sections are explicitly handled, enable orphan section warnings. Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Ingo Molnar <mingo@kernel.org> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Link: https://lore.kernel.org/r/20200902025347.2504702-6-keescook@chromium.org
2020-09-01x86/boot/compressed: Add missing debugging sections to outputKees Cook
Include the missing DWARF and STABS sections in the compressed image, when they are present. Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200821194310.3089815-29-keescook@chromium.org
2020-09-01x86/boot/compressed: Remove, discard, or assert for unwanted sectionsKees Cook
In preparation for warning on orphan sections, stop the linker from generating the .eh_frame* sections, discard unwanted non-zero-sized generated sections, and enforce other expected-to-be-zero-sized sections (since discarding them might hide problems with them suddenly gaining unexpected entries). Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200821194310.3089815-28-keescook@chromium.org
2020-09-01x86/boot/compressed: Reorganize zero-size section assertsKees Cook
For readability, move the zero-sized sections to the end after DISCARDS. Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200821194310.3089815-27-keescook@chromium.org
2020-09-01vmlinux.lds.h: Split ELF_DETAILS from STABS_DEBUGKees Cook
The .comment section doesn't belong in STABS_DEBUG. Split it out into a new macro named ELF_DETAILS. This will gain other non-debug sections that need to be accounted for when linking with --orphan-handling=warn. Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Ingo Molnar <mingo@kernel.org> Cc: linux-arch@vger.kernel.org Link: https://lore.kernel.org/r/20200821194310.3089815-5-keescook@chromium.org
2020-08-23treewide: Use fallthrough pseudo-keywordGustavo A. R. Silva
Replace the existing /* fall through */ comments and its variants with the new pseudo-keyword macro fallthrough[1]. Also, remove unnecessary fall-through markings when it is the case. [1] https://www.kernel.org/doc/html/v5.7/process/deprecated.html?highlight=fallthrough#implicit-switch-case-fall-through Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
2020-08-19x86/boot/compressed: Use builtin mem functions for decompressorArvind Sankar
Since commits c041b5ad8640 ("x86, boot: Create a separate string.h file to provide standard string functions") fb4cac573ef6 ("x86, boot: Move memcmp() into string.h and string.c") the decompressor stub has been using the compiler's builtin memcpy, memset and memcmp functions, _except_ where it would likely have the largest impact, in the decompression code itself. Remove the #undef's of memcpy and memset in misc.c so that the decompressor code also uses the compiler builtins. The rationale given in the comment doesn't really apply: just because some functions use the out-of-line version is no reason to not use the builtin version in the rest. Replace the comment with an explanation of why memzero and memmove are being #define'd. Drop the suggestion to #undef in boot/string.h as well: the out-of-line versions are not really optimized versions, they're generic code that's good enough for the preboot environment. The compiler will likely generate better code for constant-size memcpy/memset/memcmp if it is allowed to. Most decompressors' performance is unchanged, with the exception of LZ4 and 64-bit ZSTD. Before After ARCH LZ4 73ms 10ms 32 LZ4 120ms 10ms 64 ZSTD 90ms 74ms 64 Measurements on QEMU on 2.2GHz Broadwell Xeon, using defconfig kernels. Decompressor code size has small differences, with the largest being that 64-bit ZSTD decreases just over 2k. The largest code size increase was on 64-bit XZ, of about 400 bytes. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Suggested-by: Nick Terrell <nickrterrell@gmail.com> Tested-by: Nick Terrell <nickrterrell@gmail.com> Acked-by: Kees Cook <keescook@chromium.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-08-14x86/boot: Check that there are no run-time relocationsArvind Sankar
Add a linker script check that there are no run-time relocations, and remove the old one that tries to check via looking for specially-named sections in the object files. Drop the tests for -fPIE compiler option and -pie linker option, as they are available in all supported gcc and binutils versions (as well as clang and lld). Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Ingo Molnar <mingo@kernel.org> Tested-by: Nick Desaulniers <ndesaulniers@google.com> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> Reviewed-by: Kees Cook <keescook@chromium.org> Reviewed-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Fangrui Song <maskray@google.com> Reviewed-by: Sedat Dilek <sedat.dilek@gmail.com> Link: https://lore.kernel.org/r/20200731230820.1742553-8-keescook@chromium.org
2020-08-14x86/boot: Remove run-time relocations from head_{32,64}.SArvind Sankar
The BFD linker generates run-time relocations for z_input_len and z_output_len, even though they are absolute symbols. This is fixed for binutils-2.35 [1]. Work around this for earlier versions by defining two variables input_len and output_len in addition to the symbols, and use them via position-independent references. This eliminates the last two run-time relocations in the head code and allows us to drop the -z noreloc-overflow flag to the linker. Move the -pie and --no-dynamic-linker LDFLAGS to LDFLAGS_vmlinux instead of KBUILD_LDFLAGS. There shouldn't be anything else getting linked, but this is the more logical location for these flags, and modversions might call the linker if an EXPORT_SYMBOL is left over accidentally in one of the decompressors. [1] https://sourceware.org/bugzilla/show_bug.cgi?id=25754 Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Ingo Molnar <mingo@kernel.org> Tested-by: Nick Desaulniers <ndesaulniers@google.com> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> Reviewed-by: Kees Cook <keescook@chromium.org> Reviewed-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Fangrui Song <maskray@google.com> Link: https://lore.kernel.org/r/20200731230820.1742553-7-keescook@chromium.org
2020-08-14x86/boot: Remove run-time relocations from .head.text codeArvind Sankar
The assembly code in head_{32,64}.S, while meant to be position-independent, generates run-time relocations because it uses instructions such as: leal gdt(%edx), %eax which make the assembler and linker think that the code is using %edx as an index into gdt, and hence gdt needs to be relocated to its run-time address. On 32-bit, with lld Dmitry Golovin reports that this results in a link-time error with default options (i.e. unless -z notext is explicitly passed): LD arch/x86/boot/compressed/vmlinux ld.lld: error: can't create dynamic relocation R_386_32 against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output With the BFD linker, this generates a warning during the build, if --warn-shared-textrel is enabled, which at least Gentoo enables by default: LD arch/x86/boot/compressed/vmlinux ld: arch/x86/boot/compressed/head_32.o: warning: relocation in read-only section `.head.text' ld: warning: creating a DT_TEXTREL in object On 64-bit, it is not possible to link the kernel as -pie with lld, and it is only possible with a BFD linker that supports -z noreloc-overflow, i.e. versions >2.26. This is because these instructions cannot really be relocated: the displacement field is only 32-bits wide, and thus cannot be relocated for a 64-bit load address. The -z noreloc-overflow option simply overrides the linker error, and results in R_X86_64_RELATIVE relocations that apply a 64-bit relocation to a 32-bit field anyway. This happens to work because nothing will process these run-time relocations. Start fixing this by removing relocations from .head.text: - On 32-bit, use a base register that holds the address of the GOT and reference symbol addresses using @GOTOFF, i.e. leal gdt@GOTOFF(%edx), %eax - On 64-bit, most of the code can (and already does) use %rip-relative addressing, however the .code32 bits can't, and the 64-bit code also needs to reference symbol addresses as they will be after moving the compressed kernel to the end of the decompression buffer. For these cases, reference the symbols as an offset to startup_32 to avoid creating relocations, i.e.: leal (gdt-startup_32)(%bp), %eax This only works in .head.text as the subtraction cannot be represented as a PC-relative relocation unless startup_32 is in the same section as the code. Move efi32_pe_entry into .head.text so that it can use the same method to avoid relocations. Reported-by: Dmitry Golovin <dima@golovin.in> Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Ingo Molnar <mingo@kernel.org> Tested-by: Nick Desaulniers <ndesaulniers@google.com> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> Reviewed-by: Kees Cook <keescook@chromium.org> Reviewed-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Fangrui Song <maskray@google.com> Link: https://lore.kernel.org/r/20200731230820.1742553-6-keescook@chromium.org
2020-08-14x86/boot/compressed: Get rid of GOT fixup codeArd Biesheuvel
In a previous patch, we have eliminated GOT entries from the decompressor binary and added an assertion that the .got section is empty. This means that the GOT fixup routines that exist in both the 32-bit and 64-bit startup routines have become dead code, and can be removed. While at it, drop the KEEP() from the linker script, as it has no effect on the contents of output sections that are created by the linker itself. Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Ingo Molnar <mingo@kernel.org> Tested-by: Nick Desaulniers <ndesaulniers@google.com> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> Reviewed-by: Kees Cook <keescook@chromium.org> Acked-by: Arvind Sankar <nivedita@alum.mit.edu> Link: https://lore.kernel.org/r/20200731230820.1742553-4-keescook@chromium.org
2020-08-14x86/boot/compressed: Force hidden visibility for all symbol referencesArd Biesheuvel
Eliminate all GOT entries in the decompressor binary, by forcing hidden visibility for all symbol references, which informs the compiler that such references will be resolved at link time without the need for allocating GOT entries. To ensure that no GOT entries will creep back in, add an assertion to the decompressor linker script that will fire if the .got section has a non-zero size. [Arvind: move hidden.h to include/linux instead of making a copy] Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Ingo Molnar <mingo@kernel.org> Tested-by: Nick Desaulniers <ndesaulniers@google.com> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> Reviewed-by: Kees Cook <keescook@chromium.org> Acked-by: Arvind Sankar <nivedita@alum.mit.edu> Link: https://lore.kernel.org/r/20200731230820.1742553-3-keescook@chromium.org
2020-08-14x86/boot/compressed: Move .got.plt entries out of the .got sectionArd Biesheuvel
The .got.plt section contains the part of the GOT which is used by PLT entries, and which gets updated lazily by the dynamic loader when function calls are dispatched through those PLT entries. On fully linked binaries such as the kernel proper or the decompressor, this never happens, and so in practice, the .got.plt section consists only of the first 3 magic entries that are meant to point at the _DYNAMIC section and at the fixup routine in the loader. However, since we don't use a dynamic loader, those entries are never populated or used. This means that treating those entries like ordinary GOT entries, and updating their values based on the actual placement of the executable in memory is completely pointless, and we can just ignore the .got.plt section entirely, provided that it has no additional entries beyond the first 3 ones. So add an assertion in the linker script to ensure that this assumption holds, and move the contents out of the [_got, _egot) memory range that is modified by the GOT fixup routines. While at it, drop the KEEP(), since it has no effect on the contents of output sections that are created by the linker itself. Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Ingo Molnar <mingo@kernel.org> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> Tested-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Acked-by: Arvind Sankar <nivedita@alum.mit.edu> Link: https://lore.kernel.org/r/20200731230820.1742553-2-keescook@chromium.org
2020-08-09Merge tag 'kbuild-v5.9' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild Pull Kbuild updates from Masahiro Yamada: - run the checker (e.g. sparse) after the compiler - remove unneeded cc-option tests for old compiler flags - fix tar-pkg to install dtbs - introduce ccflags-remove-y and asflags-remove-y syntax - allow to trace functions in sub-directories of lib/ - introduce hostprogs-always-y and userprogs-always-y syntax - various Makefile cleanups * tag 'kbuild-v5.9' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: kbuild: stop filtering out $(GCC_PLUGINS_CFLAGS) from cc-option base kbuild: include scripts/Makefile.* only when relevant CONFIG is enabled kbuild: introduce hostprogs-always-y and userprogs-always-y kbuild: sort hostprogs before passing it to ifneq kbuild: move host .so build rules to scripts/gcc-plugins/Makefile kbuild: Replace HTTP links with HTTPS ones kbuild: trace functions in subdirectories of lib/ kbuild: introduce ccflags-remove-y and asflags-remove-y kbuild: do not export LDFLAGS_vmlinux kbuild: always create directories of targets powerpc/boot: add DTB to 'targets' kbuild: buildtar: add dtbs support kbuild: remove cc-option test of -ffreestanding kbuild: remove cc-option test of -fno-stack-protector Revert "kbuild: Create directory for target DTB" kbuild: run the checker after the compiler
2020-08-06x86/kaslr: Replace strlen() with strnlen()Arvind Sankar
strnlen is safer in case the command line is not NUL-terminated. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200803011534.730645-2-nivedita@alum.mit.edu
2020-07-31x86: Add support for ZSTD compressed kernelNick Terrell
- Add support for zstd compressed kernel - Define __DISABLE_EXPORTS in Makefile - Remove __DISABLE_EXPORTS definition from kaslr.c - Bump the heap size for zstd. - Update the documentation. Integrates the ZSTD decompression code to the x86 pre-boot code. Zstandard requires slightly more memory during the kernel decompression on x86 (192 KB vs 64 KB), and the memory usage is independent of the window size. __DISABLE_EXPORTS is now defined in the Makefile, which covers both the existing use in kaslr.c, and the use needed by the zstd decompressor in misc.c. This patch has been boot tested with both a zstd and gzip compressed kernel on i386 and x86_64 using buildroot and QEMU. Additionally, this has been tested in production on x86_64 devices. We saw a 2 second boot time reduction by switching kernel compression from xz to zstd. Signed-off-by: Nick Terrell <terrelln@fb.com> Signed-off-by: Ingo Molnar <mingo@kernel.org> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> Reviewed-by: Kees Cook <keescook@chromium.org> Link: https://lore.kernel.org/r/20200730190841.2071656-7-nickrterrell@gmail.com
2020-07-31x86/kaslr: Add a check that the random address is in rangeArvind Sankar
Check in find_random_phys_addr() that the chosen address is inside the range that was required. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200728225722.67457-22-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Make local variables 64-bitArvind Sankar
Change the type of local variables/fields that store mem_vector addresses to u64 to make it less likely that 32-bit overflow will cause issues on 32-bit. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200728225722.67457-21-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Replace 'unsigned long long' with 'u64'Arvind Sankar
No functional change. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200728225722.67457-20-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Make minimum/image_size 'unsigned long'Arvind Sankar
Change type of minimum/image_size arguments in process_mem_region to 'unsigned long'. These actually can never be above 4G (even on x86_64), and they're 'unsigned long' in every other function except this one. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200728225722.67457-19-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Small cleanup of find_random_phys_addr()Arvind Sankar
Just a trivial rearrangement to do all the processing together, and only have one call to slots_fetch_random() in the source. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200728225722.67457-18-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Drop unnecessary alignment in find_random_virt_addr()Arvind Sankar
Drop unnecessary alignment of image_size to CONFIG_PHYSICAL_ALIGN in find_random_virt_addr, it cannot change the result: the largest valid slot is the largest n that satisfies minimum + n * CONFIG_PHYSICAL_ALIGN + image_size <= KERNEL_IMAGE_SIZE (since minimum is already aligned) and so n is equal to (KERNEL_IMAGE_SIZE - minimum - image_size) / CONFIG_PHYSICAL_ALIGN even if image_size is not aligned to CONFIG_PHYSICAL_ALIGN. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200728225722.67457-17-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Drop redundant check in store_slot_info()Arvind Sankar
Drop unnecessary check that number of slots is not zero in store_slot_info, it's guaranteed to be at least 1 by the calculation. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200728225722.67457-16-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Make the type of number of slots/slot areas consistentArvind Sankar
The number of slots can be 'unsigned int', since on 64-bit, the maximum amount of memory is 2^52, the minimum alignment is 2^21, so the slot number cannot be greater than 2^31. But in case future processors have more than 52 physical address bits, make it 'unsigned long'. The slot areas are limited by MAX_SLOT_AREA, currently 100. It is indexed by an int, but the number of areas is stored as 'unsigned long'. Change both to 'unsigned int' for consistency. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200728225722.67457-15-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Drop test for command-line parameters before parsingArvind Sankar
This check doesn't save anything. In the case when none of the parameters are present, each strstr will scan args twice (once to find the length and then for searching), six scans in total. Just going ahead and parsing the arguments only requires three scans: strlen, memcpy, and parsing. This will be the first malloc, so free will actually free up the memory, so the check doesn't save heap space either. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200728225722.67457-14-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Simplify process_gb_huge_pages()Arvind Sankar
Replace the loop to determine the number of 1Gb pages with arithmetic. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200728225722.67457-13-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Short-circuit gb_huge_pages on x86-32Arvind Sankar
32-bit does not have GB pages, so don't bother checking for them. Using the IS_ENABLED() macro allows the compiler to completely remove the gb_huge_pages code. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200728225722.67457-12-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Fix off-by-one error in process_gb_huge_pages()Arvind Sankar
If the remaining size of the region is exactly 1Gb, there is still one hugepage that can be reserved. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200728225722.67457-11-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Drop some redundant checks from __process_mem_region()Arvind Sankar
Clip the start and end of the region to minimum and mem_limit prior to the loop. region.start can only increase during the loop, so raising it to minimum before the loop is enough. A region that becomes empty due to this will get checked in the first iteration of the loop. Drop the check for overlap extending beyond the end of the region. This will get checked in the next loop iteration anyway. Rename end to region_end for symmetry with region.start. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200728225722.67457-10-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Drop redundant variable in __process_mem_region()Arvind Sankar
region.size can be trimmed to store the portion of the region before the overlap, instead of a separate mem_vector variable. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200728225722.67457-9-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Eliminate 'start_orig' local variable from __process_mem_region()Arvind Sankar
Set the region.size within the loop, which removes the need for start_orig. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200728225722.67457-8-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Drop redundant cur_entry from __process_mem_region()Arvind Sankar
cur_entry is only used as cur_entry.start + cur_entry.size, which is always equal to end. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200728225722.67457-7-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Fix off-by-one error in __process_mem_region()Arvind Sankar
In case of an overlap, the beginning of the region should be used even if it is exactly image_size, not just strictly larger. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20200728225722.67457-6-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Initialize mem_limit to the real maximum addressArvind Sankar
On 64-bit, the kernel must be placed below MAXMEM (64TiB with 4-level paging or 4PiB with 5-level paging). This is currently not enforced by KASLR, which thus implicitly relies on physical memory being limited to less than 64TiB. On 32-bit, the limit is KERNEL_IMAGE_SIZE (512MiB). This is enforced by special checks in __process_mem_region(). Initialize mem_limit to the maximum (depending on architecture), instead of ULLONG_MAX, and make sure the command-line arguments can only decrease it. This makes the enforcement explicit on 64-bit, and eliminates the 32-bit specific checks to keep the kernel below 512M. Check upfront to make sure the minimum address is below the limit before doing any work. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Acked-by: Kees Cook <keescook@chromium.org> Link: https://lore.kernel.org/r/20200727230801.3468620-5-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Fix process_efi_entries commentArvind Sankar
Since commit: 0982adc74673 ("x86/boot/KASLR: Work around firmware bugs by excluding EFI_BOOT_SERVICES_* and EFI_LOADER_* from KASLR's choice") process_efi_entries() will return true if we have an EFI memmap, not just if it contained EFI_MEMORY_MORE_RELIABLE regions. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Reviewed-by: Kees Cook <keescook@chromium.org> Link: https://lore.kernel.org/r/20200727230801.3468620-4-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Remove bogus warning and unnecessary gotoArvind Sankar
Drop the warning on seeing "--" in handle_mem_options(). This will trigger whenever one of the memory options is present in the command line together with "--", but there's no problem if that is the case. Replace goto with break. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Reviewed-by: Kees Cook <keescook@chromium.org> Link: https://lore.kernel.org/r/20200727230801.3468620-3-nivedita@alum.mit.edu
2020-07-31x86/kaslr: Make command line handling saferArvind Sankar
Handle the possibility that the command line is NULL. Replace open-coded strlen with a function call. Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Ingo Molnar <mingo@kernel.org> Reviewed-by: Kees Cook <keescook@chromium.org> Link: https://lore.kernel.org/r/20200727230801.3468620-2-nivedita@alum.mit.edu
2020-07-19Merge tag 'x86-urgent-2020-07-19' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip into master Pull x86 fixes from Thomas Gleixner: "A pile of fixes for x86: - Fix the I/O bitmap invalidation on XEN PV, which was overlooked in the recent ioperm/iopl rework. This caused the TSS and XEN's I/O bitmap to get out of sync. - Use the proper vectors for HYPERV. - Make disabling of stack protector for the entry code work with GCC builds which enable stack protector by default. Removing the option is not sufficient, it needs an explicit -fno-stack-protector to shut it off. - Mark check_user_regs() noinstr as it is called from noinstr code. The missing annotation causes it to be placed in the text section which makes it instrumentable. - Add the missing interrupt disable in exc_alignment_check() - Fixup a XEN_PV build dependency in the 32bit entry code - A few fixes to make the Clang integrated assembler happy - Move EFI stub build to the right place for out of tree builds - Make prepare_exit_to_usermode() static. It's not longer called from ASM code" * tag 'x86-urgent-2020-07-19' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: x86/boot: Don't add the EFI stub to targets x86/entry: Actually disable stack protector x86/ioperm: Fix io bitmap invalidation on Xen PV x86: math-emu: Fix up 'cmp' insn for clang ias x86/entry: Fix vectors to IDTENTRY_SYSVEC for CONFIG_HYPERV x86/entry: Add compatibility with IAS x86/entry/common: Make prepare_exit_to_usermode() static x86/entry: Mark check_user_regs() noinstr x86/traps: Disable interrupts in exc_aligment_check() x86/entry/32: Fix XEN_PV build dependency
2020-07-19x86/boot: Don't add the EFI stub to targetsArvind Sankar
vmlinux-objs-y is added to targets, which currently means that the EFI stub gets added to the targets as well. It shouldn't be added since it is built elsewhere. This confuses Makefile.build which interprets the EFI stub as a target $(obj)/$(objtree)/drivers/firmware/efi/libstub/lib.a and will create drivers/firmware/efi/libstub/ underneath arch/x86/boot/compressed, to hold this supposed target, if building out-of-tree. [0] Fix this by pulling the stub out of vmlinux-objs-y into efi-obj-y. [0] See scripts/Makefile.build near the end: # Create directories for object files if they do not exist Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Ard Biesheuvel <ardb@kernel.org> Link: https://lkml.kernel.org/r/20200715032631.1562882-1-nivedita@alum.mit.edu
2020-07-07kbuild: remove cc-option test of -ffreestandingMasahiro Yamada
Some Makefiles already pass -ffreestanding unconditionally. For example, arch/arm64/lib/Makefile, arch/x86/purgatory/Makefile. No problem report so far about hard-coding this option. So, we can assume all supported compilers know -ffreestanding. I confirmed GCC 4.8 and Clang manuals document this option. Get rid of cc-option from -ffreestanding. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Acked-by: Ard Biesheuvel <ardb@kernel.org>
2020-07-07kbuild: remove cc-option test of -fno-stack-protectorMasahiro Yamada
Some Makefiles already pass -fno-stack-protector unconditionally. For example, arch/arm64/kernel/vdso/Makefile, arch/x86/xen/Makefile. No problem report so far about hard-coding this option. So, we can assume all supported compilers know -fno-stack-protector. GCC 4.8 and Clang support this option (https://godbolt.org/z/_HDGzN) Get rid of cc-option from -fno-stack-protector. Remove CONFIG_CC_HAS_STACKPROTECTOR_NONE, which is always 'y'. Note: arch/mips/vdso/Makefile adds -fno-stack-protector twice, first unconditionally, and second conditionally. I removed the second one. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Kees Cook <keescook@chromium.org> Acked-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
2020-06-17efi/x86: Setup stack correctly for efi_pe_entryArvind Sankar
Commit 17054f492dfd ("efi/x86: Implement mixed mode boot without the handover protocol") introduced a new entry point for the EFI stub to be booted in mixed mode on 32-bit firmware. When entered via efi32_pe_entry, control is first transferred to startup_32 to setup for the switch to long mode, and then the EFI stub proper is entered via efi_pe_entry. efi_pe_entry is an MS ABI function, and the ABI requires 32 bytes of shadow stack space to be allocated by the caller, as well as the stack being aligned to 8 mod 16 on entry. Allocate 40 bytes on the stack before switching to 64-bit mode when calling efi_pe_entry to account for this. For robustness, explicitly align boot_stack_end to 16 bytes. It is currently implicitly aligned since .bss is cacheline-size aligned, head_64.o is the first object file with a .bss section, and the heap and boot sizes are aligned. Fixes: 17054f492dfd ("efi/x86: Implement mixed mode boot without the handover protocol") Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Link: https://lore.kernel.org/r/20200617131957.2507632-1-nivedita@alum.mit.edu Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
2020-06-11Rebase locking/kcsan to locking/urgentThomas Gleixner
Merge the state of the locking kcsan branch before the read/write_once() and the atomics modifications got merged. Squash the fallout of the rebase on top of the read/write once and atomic fallback work into the merge. The history of the original branch is preserved in tag locking-kcsan-2020-06-02. Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-06-09mm: reorder includes after introduction of linux/pgtable.hMike Rapoport
The replacement of <asm/pgrable.h> with <linux/pgtable.h> made the include of the latter in the middle of asm includes. Fix this up with the aid of the below script and manual adjustments here and there. import sys import re if len(sys.argv) is not 3: print "USAGE: %s <file> <header>" % (sys.argv[0]) sys.exit(1) hdr_to_move="#include <linux/%s>" % sys.argv[2] moved = False in_hdrs = False with open(sys.argv[1], "r") as f: lines = f.readlines() for _line in lines: line = _line.rstrip(' ') if line == hdr_to_move: continue if line.startswith("#include <linux/"): in_hdrs = True elif not moved and in_hdrs: moved = True print hdr_to_move print line Signed-off-by: Mike Rapoport <rppt@linux.ibm.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Cain <bcain@codeaurora.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Chris Zankel <chris@zankel.net> Cc: "David S. Miller" <davem@davemloft.net> Cc: Geert Uytterhoeven <geert@linux-m68k.org> Cc: Greentime Hu <green.hu@gmail.com> Cc: Greg Ungerer <gerg@linux-m68k.org> Cc: Guan Xuetao <gxt@pku.edu.cn> Cc: Guo Ren <guoren@kernel.org> Cc: Heiko Carstens <heiko.carstens@de.ibm.com> Cc: Helge Deller <deller@gmx.de> Cc: Ingo Molnar <mingo@redhat.com> Cc: Ley Foon Tan <ley.foon.tan@intel.com> Cc: Mark Salter <msalter@redhat.com> Cc: Matthew Wilcox <willy@infradead.org> Cc: Matt Turner <mattst88@gmail.com> Cc: Max Filippov <jcmvbkbc@gmail.com> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Michal Simek <monstr@monstr.eu> Cc: Nick Hu <nickhu@andestech.com> Cc: Paul Walmsley <paul.walmsley@sifive.com> Cc: Richard Weinberger <richard@nod.at> Cc: Rich Felker <dalias@libc.org> Cc: Russell King <linux@armlinux.org.uk> Cc: Stafford Horne <shorne@gmail.com> Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Tony Luck <tony.luck@intel.com> Cc: Vincent Chen <deanbo422@gmail.com> Cc: Vineet Gupta <vgupta@synopsys.com> Cc: Will Deacon <will@kernel.org> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Link: http://lkml.kernel.org/r/20200514170327.31389-4-rppt@kernel.org Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-06-09mm: introduce include/linux/pgtable.hMike Rapoport
The include/linux/pgtable.h is going to be the home of generic page table manipulation functions. Start with moving asm-generic/pgtable.h to include/linux/pgtable.h and make the latter include asm/pgtable.h. Signed-off-by: Mike Rapoport <rppt@linux.ibm.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Cain <bcain@codeaurora.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Chris Zankel <chris@zankel.net> Cc: "David S. Miller" <davem@davemloft.net> Cc: Geert Uytterhoeven <geert@linux-m68k.org> Cc: Greentime Hu <green.hu@gmail.com> Cc: Greg Ungerer <gerg@linux-m68k.org> Cc: Guan Xuetao <gxt@pku.edu.cn> Cc: Guo Ren <guoren@kernel.org> Cc: Heiko Carstens <heiko.carstens@de.ibm.com> Cc: Helge Deller <deller@gmx.de> Cc: Ingo Molnar <mingo@redhat.com> Cc: Ley Foon Tan <ley.foon.tan@intel.com> Cc: Mark Salter <msalter@redhat.com> Cc: Matthew Wilcox <willy@infradead.org> Cc: Matt Turner <mattst88@gmail.com> Cc: Max Filippov <jcmvbkbc@gmail.com> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Michal Simek <monstr@monstr.eu> Cc: Nick Hu <nickhu@andestech.com> Cc: Paul Walmsley <paul.walmsley@sifive.com> Cc: Richard Weinberger <richard@nod.at> Cc: Rich Felker <dalias@libc.org> Cc: Russell King <linux@armlinux.org.uk> Cc: Stafford Horne <shorne@gmail.com> Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Tony Luck <tony.luck@intel.com> Cc: Vincent Chen <deanbo422@gmail.com> Cc: Vineet Gupta <vgupta@synopsys.com> Cc: Will Deacon <will@kernel.org> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Link: http://lkml.kernel.org/r/20200514170327.31389-3-rppt@kernel.org Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-06-01Merge tag 'x86-build-2020-06-01' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip Pull x86 build updates from Ingo Molnar: "Misc dependency fixes, plus a documentation update about memory protection keys support" * tag 'x86-build-2020-06-01' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: x86/Kconfig: Update config and kernel doc for MPK feature on AMD x86/boot: Discard .discard.unreachable for arch/x86/boot/compressed/vmlinux x86/boot/build: Add phony targets in arch/x86/boot/Makefile to PHONY x86/boot/build: Make 'make bzlilo' not depend on vmlinux or $(obj)/bzImage x86/boot/build: Add cpustr.h to targets and remove clean-files
2020-06-01Merge tag 'x86-boot-2020-06-01' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip Pull x86 boot updates from Ingo Molnar: "Misc updates: - Add the initrdmem= boot option to specify an initrd embedded in RAM (flash most likely) - Sanitize the CS value earlier during boot, which also fixes SEV-ES - Various fixes and smaller cleanups" * tag 'x86-boot-2020-06-01' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: x86/boot: Correct relocation destination on old linkers x86/boot/compressed/64: Switch to __KERNEL_CS after GDT is loaded x86/boot: Fix -Wint-to-pointer-cast build warning x86/boot: Add kstrtoul() from lib/ x86/tboot: Mark tboot static x86/setup: Add an initrdmem= option to specify initrd physical address
generated by cgit v1.2.3 (git 2.39.1) at 2025-08-03 22:35:56 +0000