1. 18 Jul, 2021 1 commit
    • Lecopzer Chen's avatar
      Kbuild: lto: fix module versionings mismatch in GNU make 3.X · 1d11053d
      Lecopzer Chen authored
      When building modules(CONFIG_...=m), I found some of module versions
      are incorrect and set to 0.
      This can be found in build log for first clean build which shows
      
      WARNING: EXPORT symbol "XXXX" [drivers/XXX/XXX.ko] version generation failed,
      symbol will not be versioned.
      
      But in second build(incremental build), the WARNING disappeared and the
      module version becomes valid CRC and make someone who want to change
      modules without updating kernel image can't insert their modules.
      
      The problematic code is
      +	$(foreach n, $(filter-out FORCE,$^),				\
      +		$(if $(wildcard $(n).symversions),			\
      +			; cat $(n).symversions >> $@.symversions))
      
      For example:
        rm -f fs/notify/built-in.a.symversions    ; rm -f fs/notify/built-in.a; \
      llvm-ar cDPrST fs/notify/built-in.a fs/notify/fsnotify.o \
      fs/notify/notification.o fs/notify/group.o ...
      
      `foreach n` shows nothing to `cat` into $(n).symversions because
      `if $(wildcard $(n).symversions)` return nothing, but actually
      they do exist during this line was executed.
      
      -rw-r--r-- 1 root root 168580 Jun 13 19:10 fs/notify/fsnotify.o
      -rw-r--r-- 1 root root    111 Jun 13 19:10 fs/notify/fsnotify.o.symversions
      
      The reason is the $(n).symversions are generated at runtime, but
      Makefile wildcard function expends and checks the file exist or not
      during parsing the Makefile.
      
      Thus fix this by use `test` shell command to check the file
      existence in runtime.
      
      Rebase from both:
      1. [https://lore.kernel.org/lkml/20210616080252.32046-1-lecopzer.chen@mediatek.com/]
      2. [https://lore.kernel.org/lkml/20210702032943.7865-1-lecopzer.chen@mediatek.com/]
      
      Fixes: 38e89184
      
       ("kbuild: lto: fix module versioning")
      Co-developed-by: default avatarSami Tolvanen <samitolvanen@google.com>
      Signed-off-by: default avatarLecopzer Chen <lecopzer.chen@mediatek.com>
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      1d11053d
  2. 24 May, 2021 1 commit
    • Masahiro Yamada's avatar
      kbuild: remove libelf checks from top Makefile · 0d989ac2
      Masahiro Yamada authored
      I do not see a good reason why only the libelf development package must
      be so carefully checked.
      
      Kbuild generally does not check host tools or libraries.
      
      For example, x86_64 defconfig fails to build with no libssl development
      package installed.
      
      scripts/extract-cert.c:21:10: fatal error: openssl/bio.h: No such file or directory
         21 | #include <openssl/bio.h>
            |          ^~~~~~~~~~~~~~~
      
      To solve the build error, you need to install libssl-dev or openssl-devel
      package, depending on your distribution.
      
      'apt-file search', 'dnf provides', etc. is your frined to find a proper
      package to install.
      
      This commit removes all the libelf checks from the top Makefile.
      
      If libelf is missing, objtool will fail to build in a similar pattern:
      
      .../linux/tools/objtool/include/objtool/elf.h:10:10: fatal error: gelf.h: No such file or directory
         10 | #include <gelf.h>
      
      You need to install libelf-dev, libelf-devel, or elfutils-libelf-devel
      to proceed.
      
      Another remarkable change is, CONFIG_STACK_VALIDATION (without
      CONFIG_UNWINDER_ORC) previously continued to build with a warning,
      but now it will treat missing libelf as an error.
      
      This is just a one-time installation, so it should not hurt to break
      a build and make a user install the package.
      
      BTW, the traditional way to handle such checks is autotool, but according
      to [1], I do not expect the kernel build would have similar scripting
      like './configure' does.
      
      [1]: https://lore.kernel.org/lkml/CA+55aFzr2HTZVOuzpHYDwmtRJLsVzE-yqg2DHpHi_9ePsYp5ug@mail.gmail.com/
      
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      Acked-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      0d989ac2
  3. 19 May, 2021 1 commit
    • Josh Poimboeuf's avatar
      kbuild: Fix objtool dependency for 'OBJECT_FILES_NON_STANDARD_<obj> := n' · 8852c552
      Josh Poimboeuf authored
      "OBJECT_FILES_NON_STANDARD_vma.o := n" has a dependency bug.  When
      objtool source is updated, the affected object doesn't get re-analyzed
      by objtool.
      
      Peter's new variable-sized jump label feature relies on objtool
      rewriting the object file.  Otherwise the system can fail to boot.  That
      effectively upgrades this minor dependency issue to a major bug.
      
      The problem is that variables in prerequisites are expanded early,
      during the read-in phase.  The '$(objtool_dep)' variable indirectly uses
      '$@', which isn't yet available when the target prerequisites are
      evaluated.
      
      Use '.SECONDEXPANSION:' which causes '$(objtool_dep)' to be expanded in
      a later phase, after the target-specific '$@' variable has been defined.
      
      Fixes: b9ab5ebb ("objtool: Add CONFIG_STACK_VALIDATION option")
      Fixes: ab325704
      
       ("jump_label, x86: Allow short NOPs")
      Reported-by: default avatarMatthew Wilcox <willy@infradead.org>
      Signed-off-by: default avatarJosh Poimboeuf <jpoimboe@redhat.com>
      8852c552
  4. 05 May, 2021 1 commit
    • Masahiro Yamada's avatar
      kbuild: refactor fdtoverlay rule · bcf0c664
      Masahiro Yamada authored
      
      
      Rename overlay-y to multi-dtb-y, which is a consistent name with
      multi-obj-y. Also, use multi-search to avoid code duplication.
      
      Introduce real-dtb-y, which is a consistent name with real-obj-y,
      to contain primitive blobs compiled from *.dts. This is used to
      calculate the list of *.dt.yaml files.
      
      Set -@ to base DTB without using $(eval ).
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      bcf0c664
  5. 24 Apr, 2021 3 commits
    • Alexey Dobriyan's avatar
      kbuild: redo fake deps at include/config/*.h · 0e0345b7
      Alexey Dobriyan authored
      
      
      Make include/config/foo/bar.h fake deps files generation simpler.
      
      * delete .h suffix
      	those aren't header files, shorten filenames,
      
      * delete tolower()
      	Linux filesystems can deal with both upper and lowercase
      	filenames very well,
      
      * put everything in 1 directory
      	Presumably 'mkdir -p' split is from dark times when filesystems
      	handled huge directories badly, disks were round adding to
      	seek times.
      
      	x86_64 allmodconfig lists 12364 files in include/config.
      
      	../obj/include/config/
      	├── 104_QUAD_8
      	├── 60XX_WDT
      	├── 64BIT
      		...
      	├── ZSWAP_DEFAULT_ON
      	├── ZSWAP_ZPOOL_DEFAULT
      	└── ZSWAP_ZPOOL_DEFAULT_ZBUD
      
      	0 directories, 12364 files
      Signed-off-by: default avatarAlexey Dobriyan <adobriyan@gmail.com>
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      0e0345b7
    • Masahiro Yamada's avatar
      kbuild: rename multi-used-* to multi-obj-* · a6601e01
      Masahiro Yamada authored
      
      
      I think multi-obj-* is clearer, and more consistent with real-obj-*.
      
      Rename as follows:
      
        multi-used-y  ->  multi-obj-y
        multi-used-m  ->  multi-obj-m
        multi-used    ->  multi-obj-ym
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      a6601e01
    • Masahiro Yamada's avatar
      kbuild: split cc-option and friends to scripts/Makefile.compiler · 57fd251c
      Masahiro Yamada authored
      
      
      scripts/Kbuild.include is included everywhere, but macros such as
      cc-option are needed by build targets only.
      
      For example, when 'make clean' traverses the tree, it does not need
      to evaluate $(call cc-option,).
      
      Split cc-option, ld-option, etc. to scripts/Makefile.compiler, which
      is only included from the top Makefile and scripts/Makefile.build.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      57fd251c
  6. 15 Mar, 2021 1 commit
    • Masahiro Yamada's avatar
      kbuild: prefix $(srctree)/ to some included Makefiles · 3204a7fb
      Masahiro Yamada authored
      
      
      VPATH is used in Kbuild to make pattern rules search for prerequisites
      in both $(objtree) and $(srctree). Some of *.c, *.S files are not real
      sources, but generated by tools such as flex, bison, perl.
      
      In contrast, I doubt the benefit of --include-dir=$(abs_srctree) because
      it is always clear which Makefiles are real sources, and which are not.
      
      So, my hope is to add $(srctree)/ prefix to all check-in Makefiles,
      then remove --include-dir=$(abs_srctree) flag in the future.
      
      I am touching only some Kbuild core parts for now. Treewide fixes will
      be needed to achieve this goal.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      3204a7fb
  7. 24 Feb, 2021 1 commit
  8. 23 Feb, 2021 2 commits
  9. 14 Jan, 2021 2 commits
  10. 08 Jan, 2021 1 commit
  11. 06 Dec, 2020 1 commit
    • Masahiro Yamada's avatar
      kbuild: avoid split lines in .mod files · 7d32358b
      Masahiro Yamada authored
      "xargs echo" is not a safe way to remove line breaks because the input
      may exceed the command line limit and xargs may break it up into
      multiple invocations of echo. This should never happen because
      scripts/gen_autoksyms.sh expects all undefined symbols are placed in
      the second line of .mod files.
      
      One possible way is to replace "xargs echo" with
      "sed ':x;N;$!bx;s/\n/ /g'" or something, but I rewrote the code by
      using awk because it is more readable.
      
      This issue was reported by Sami Tolvanen; in his Clang LTO patch set,
      $(multi-used-m) is no longer an ELF object, but a thin archive that
      contains LLVM bitcode files. llvm-nm prints out symbols for each
      archive member separately, which results a lot of dupications, in some
      places, beyond the system-defined limit.
      
      This problem must be fixed irrespective of LTO, and we must ensure
      zero possibility of having this issue.
      
      Link: https://lkml.org/lkml/2020/12/1/1658
      
      Reported-by: default avatarSami Tolvanen <samitolvanen@google.com>
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      Reviewed-by: default avatarSami Tolvanen <samitolvanen@google.com>
      7d32358b
  12. 20 Oct, 2020 1 commit
  13. 09 Aug, 2020 3 commits
    • Masahiro Yamada's avatar
      kbuild: sort hostprogs before passing it to ifneq · 85569d19
      Masahiro Yamada authored
      
      
      The conditional:
      
        ifneq ($(hostprogs),)
      
      ... is evaluated to true if $(hostprogs) does not contain any word but
      whitespace characters.
      
        ifneq ($(strip $(hostprogs)),)
      
      ... is a safe way to avoid interpreting whitespace as a non-empty value,
      but I'd rather want to use the side-effect of $(sort ...) to do the
      equivalent.
      
      $(sort ...) is used in scripts/Makefile.host in order to drop duplication
      in $(hostprogs). It is also useful to strip excessive spaces.
      
      Move $(sort ...) before evaluating the ifneq.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      85569d19
    • Masahiro Yamada's avatar
      kbuild: move host .so build rules to scripts/gcc-plugins/Makefile · 42640b13
      Masahiro Yamada authored
      The host shared library rules are currently implemented in
      scripts/Makefile.host, but actually GCC-plugin is the only user of
      them. (The VDSO .so files are built for the target by different
      build rules) Hence, they do not need to be treewide available.
      
      Move all the relevant build rules to scripts/gcc-plugins/Makefile.
      
      I also optimized the build steps so *.so is directly built from .c
      because every upstream plugin is compiled from a single source file.
      
      I am still keeping the multi-file plugin support, which Kees Cook
      mentioned might be needed by out-of-tree plugins.
      (https://lkml.org/lkml/2019/1/11/1107
      
      )
      
      If the plugin, foo.so, is compiled from two files foo.c and foo2.c,
      then you can do like follows:
      
        foo-objs := foo.o foo2.o
      
      Single-file plugins do not need the *-objs notation.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      Acked-by: default avatarKees Cook <keescook@chromium.org>
      42640b13
    • Masahiro Yamada's avatar
      kbuild: always create directories of targets · cc8a51ca
      Masahiro Yamada authored
      
      
      Currently, the directories of objects are automatically created
      only for O= builds.
      
      It should not hurt to cater to this for in-tree builds too.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      cc8a51ca
  14. 07 Jul, 2020 1 commit
  15. 03 Jun, 2020 1 commit
  16. 01 Jun, 2020 1 commit
  17. 25 May, 2020 5 commits
  18. 17 May, 2020 1 commit
    • Masahiro Yamada's avatar
      kbuild: add infrastructure to build userspace programs · 7f3a59db
      Masahiro Yamada authored
      Kbuild supports the infrastructure to build host programs, but there
      was no support to build userspace programs for the target architecture
      (i.e. the same architecture as the kernel).
      
      Sam Ravnborg worked on this in 2014 (https://lkml.org/lkml/2014/7/13/154),
      but it was not merged. One problem at that time was, there was no good way
      to know whether $(CC) can link standalone programs. In fact, pre-built
      kernel.org toolchains [1] are often used for building the kernel, but they
      do not provide libc.
      
      Now, we can handle this cleanly because the compiler capability is
      evaluated at the Kconfig time. If $(CC) cannot link standalone programs,
      the relevant options are hidden by 'depends on CC_CAN_LINK'.
      
      The implementation just mimics scripts/Makefile.host
      
      The userspace programs are compiled with the same flags as the host
      programs. In addition, it uses -m32 or -m64 if it is found in
      $(KBUILD_CFLAGS).
      
      This new syntax has two usecases.
      
      - Sample programs
      
        Several userspace programs under samples/ include UAPI headers
        installed in usr/include. Most of them were previously built for
        the host architecture just to use the 'hostprogs' syntax.
      
        However, 'make headers' always works for the target architecture.
        This caused the arch mismatch in cross-compiling. To fix this
        distortion, sample code should be built for the target architecture.
      
      - Bpfilter
      
        net/bpfilter/Makefile compiles bpfilter_umh as the user mode helper,
        and embeds it into the kernel. Currently, it overrides HOSTCC with
        CC to use the 'hostprogs' syntax. This hack should go away.
      
      [1]: https://mirrors.edge.kernel.org/pub/tools/crosstool/
      
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      Acked-by: default avatarSam Ravnborg <sam@ravnborg.org>
      7f3a59db
  19. 08 Apr, 2020 2 commits
    • Masahiro Yamada's avatar
      kbuild: link lib-y objects to vmlinux forcibly when CONFIG_MODULES=y · 7273ad2b
      Masahiro Yamada authored
      Kbuild supports not only obj-y but also lib-y to list objects linked to
      vmlinux.
      
      The difference between them is that all the objects from obj-y are
      forcibly linked to vmlinux, whereas the objects from lib-y are linked
      as needed; if there is no user of a lib-y object, it is not linked.
      
      lib-y is intended to list utility functions that may be called from all
      over the place (and may be unused at all), but it is a problem for
      EXPORT_SYMBOL(). Even if there is no call-site in the vmlinux, we need
      to keep exported symbols for the use from loadable modules.
      
      Commit 7f2084fa ("[kbuild] handle exports in lib-y objects reliably")
      worked around it by linking a dummy object, lib-ksyms.o, which contains
      references to all the symbols exported from lib.a in that directory.
      It uses the linker script command, EXTERN. Unfortunately, the meaning of
      EXTERN of ld.lld is different from that of ld.bfd. Therefore, this does
      not work with LD=ld.lld (CBL issue #515).
      
      Anyway, the build rule of lib-ksyms.o is somewhat tricky. So, I want to
      get rid of it.
      
      At first, I was thinking of accumulating lib-y objects into obj-y
      (or even replacing lib-y with obj-y entirely), but the lib-y syntax
      is used beyond the ordinary use in lib/ and arch/*/lib/.
      
      Examples:
      
       - drivers/firmware/efi/libstub/Makefile builds lib.a, which is linked
         into vmlinux in the own way (arm64), or linked to the decompressor
         (arm, x86).
      
       - arch/alpha/lib/Makefile builds lib.a which is linked not only to
         vmlinux, but also to bootloaders in arch/alpha/boot/Makefile.
      
       - arch/xtensa/boot/lib/Makefile builds lib.a for use from
         arch/xtensa/boot/boot-redboot/Makefile.
      
      One more thing, adding everything to obj-y would increase the vmlinux
      size of allnoconfig (or tinyconfig).
      
      For less impact, I tweaked the destination of lib.a at the top Makefile;
      when CONFIG_MODULES=y, lib.a goes to KBUILD_VMLINUX_OBJS, which is
      forcibly linked to vmlinux, otherwise lib.a goes to KBUILD_VMLINUX_LIBS
      as before.
      
      The size impact for normal usecases is quite small since at lease one
      symbol in every lib-y object is eventually called by someone. In case
      you are intrested, here are the figures.
      
      x86_64_defconfig:
      
         text	   data	    bss	    dec	    hex	filename
      19566602 5422072 1589328 26578002 1958c52 vmlinux.before
      19566932 5422104 1589328 26578364 1958dbc vmlinux.after
      
      The case with the biggest impact is allnoconfig + CONFIG_MODULES=y.
      
      ARCH=x86 allnoconfig + CONFIG_MODULES=y:
      
         text	   data	    bss	    dec	    hex	filename
      1175162	 254740	1220608	2650510	 28718e	vmlinux.before
      1177974	 254836	1220608	2653418	 287cea	vmlinux.after
      
      Hopefully this is still not a big deal. The per-file trimming with the
      static library is not so effective after all.
      
      If fine-grained optimization is desired, some architectures support
      CONFIG_LD_DEAD_CODE_DATA_ELIMINATION, which trims dead code per-symbol
      basis. When LTO is supported in mainline, even better optimization will
      be possible.
      
      Link: https://github.com/ClangBuiltLinux/linux/issues/515
      
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      Reported-by: default avatarkbuild test robot <lkp@intel.com>
      Reviewed-by: default avatarNick Desaulniers <ndesaulniers@google.com>
      7273ad2b
    • Masahiro Yamada's avatar
      gcc-plugins: drop support for GCC <= 4.7 · 77342a02
      Masahiro Yamada authored
      Nobody was opposed to raising minimum GCC version to 4.8 [1]
      So, we will drop GCC <= 4.7 support sooner or later.
      
      We always use C++ compiler for building plugins for GCC >= 4.8.
      
      This commit drops the plugin support for GCC <= 4.7 a bit earlier,
      which allows us to dump lots of code.
      
      [1] https://lkml.org/lkml/2020/1/23/545
      
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      Acked-by: default avatarKees Cook <keescook@chromium.org>
      77342a02
  20. 03 Feb, 2020 1 commit
    • Masahiro Yamada's avatar
      kbuild: rename hostprogs-y/always to hostprogs/always-y · 5f2fb52f
      Masahiro Yamada authored
      
      
      In old days, the "host-progs" syntax was used for specifying host
      programs. It was renamed to the current "hostprogs-y" in 2004.
      
      It is typically useful in scripts/Makefile because it allows Kbuild to
      selectively compile host programs based on the kernel configuration.
      
      This commit renames like follows:
      
        always       ->  always-y
        hostprogs-y  ->  hostprogs
      
      So, scripts/Makefile will look like this:
      
        always-$(CONFIG_BUILD_BIN2C) += ...
        always-$(CONFIG_KALLSYMS)    += ...
            ...
        hostprogs := $(always-y) $(always-m)
      
      I think this makes more sense because a host program is always a host
      program, irrespective of the kernel configuration. We want to specify
      which ones to compile by CONFIG options, so always-y will be handier.
      
      The "always", "hostprogs-y", "hostprogs-m" will be kept for backward
      compatibility for a while.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      5f2fb52f
  21. 06 Jan, 2020 2 commits
  22. 14 Nov, 2019 1 commit
    • Masahiro Yamada's avatar
      kbuild: remove header compile test · fcbb8461
      Masahiro Yamada authored
      There are both positive and negative options about this feature.
      At first, I thought it was a good idea, but actually Linus stated a
      negative opinion (https://lkml.org/lkml/2019/9/29/227
      
      ). I admit it
      is ugly and annoying.
      
      The baseline I'd like to keep is the compile-test of uapi headers.
      (Otherwise, kernel developers have no way to ensure the correctness
      of the exported headers.)
      
      I will maintain a small build rule in usr/include/Makefile.
      Remove the other header test functionality.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      fcbb8461
  23. 11 Nov, 2019 2 commits
  24. 01 Oct, 2019 1 commit
  25. 06 Sep, 2019 1 commit
  26. 21 Aug, 2019 2 commits