1. 04 Jan, 2021 1 commit
  2. 24 Dec, 2020 1 commit
    • zlib: Add support for building with Chromium's zlib implementation · 83265b3e
      This change builds libgit2 using Chromium's zlib implementation by
      invoking cmake with `-DUSE_BUNDLED_ZLIB=ON -DUSE_CHROMIUM_ZLIB=ON`,
      which is ~10% faster than the bundled zlib for the core::zstream suite.
      
      This version of zlib has some optimizations:
      
      a) Decompression (Intel+ARM): inflate_fast, adler32, crc32, etc.
      b) Compression (Intel): fill_window, longest_match, hash function, etc.
      
      Due to the introduction of SIMD optimizations, and to get the maximum
      performance out of this fork of zlib, this requires an x86_64 processor
      with SSE4.2 and CLMUL (anything Westmere or later, ~2010). The Chromium
      zlib implementation also supports ARM with NEON, but it has not been
      enabled in this patch.
      
      Performance
      ===========
      
      TL;DR: Running just `./libgit2_clar -score::zstream` 100 times in a loop
      took 0:56.30 before and 0:50.67 after (~10% reduction!).
      
      The bundled and system zlib implementations on an Ubuntu Focal system
      perform relatively similar (the bundled one is marginally better due to
      the compiler being able to inline some functions), so only the bundled
      and Chromium zlibs were compared.
      
      For a more balanced comparison (to ensure that nothing regressed
      overall), `libgit2_clar` under `perf` was also run, and the zlib-related
      functions were compared.
      
      Bundled
      -------
      
      ```shell
      cmake \
        -DUSE_BUNDLED_ZLIB=ON \
        -DUSE_CHROMIUM_ZLIB=OFF \
        -DCMAKE_BUILD_TYPE="RelWithDebInfo" \
        -DCMAKE_C_FLAGS="-fPIC -fno-omit-frame-pointer" \
        -GNinja \
        ..
      ninja
      perf record --call-graph=dwarf ./libgit2_clar
      perf report --children
      ```
      
      ```
      Samples: 87K of event 'cycles', Event count (approx.): 75923450603
        Children      Self  Command       Shared Objec  Symbol
      +    4.14%     0.01%  libgit2_clar  libgit2_clar  [.] git_zstream_get_output_chunk
      +    2.91%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_get_output
      +    0.69%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_get_output (inlined)
           0.17%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_init
           0.02%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_reset
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_eos
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_done
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_free (inlined)
      
      Samples: 87K of event 'cycles', Event count (approx.): 75923450603
        Children      Self  Command       Shared Objec  Symbol
      +    3.12%     0.01%  libgit2_clar  libgit2_clar  [.] deflate
      +    2.65%     1.48%  libgit2_clar  libgit2_clar  [.] deflate_slow
      +    1.60%     0.55%  libgit2_clar  libgit2_clar  [.] inflate
      +    0.53%     0.00%  libgit2_clar  libgit2_clar  [.] write_deflate
           0.49%     0.36%  libgit2_clar  libgit2_clar  [.] inflate_fast
           0.46%     0.02%  libgit2_clar  libgit2_clar  [.] deflate_fast
           0.19%     0.19%  libgit2_clar  libgit2_clar  [.] inflate_table
           0.16%     0.01%  libgit2_clar  libgit2_clar  [.] inflateInit_
           0.15%     0.00%  libgit2_clar  libgit2_clar  [.] inflateInit2_ (inlined)
           0.10%     0.00%  libgit2_clar  libgit2_clar  [.] deflateInit_
           0.10%     0.00%  libgit2_clar  libgit2_clar  [.] deflateInit2_
           0.03%     0.00%  libgit2_clar  libgit2_clar  [.] deflateReset (inlined)
           0.02%     0.00%  libgit2_clar  libgit2_clar  [.] deflateReset
           0.02%     0.00%  libgit2_clar  libgit2_clar  [.] inflateEnd
           0.02%     0.00%  libgit2_clar  libgit2_clar  [.] deflateEnd
           0.01%     0.00%  libgit2_clar  libgit2_clar  [.] deflateResetKeep
           0.01%     0.01%  libgit2_clar  libgit2_clar  [.] inflateReset2
           0.01%     0.00%  libgit2_clar  libgit2_clar  [.] deflateReset (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] inflateStateCheck (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] inflateReset (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] inflateStateCheck (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] deflateStateCheck (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] inflateResetKeep (inlined)
      ```
      
      Chromium
      --------
      
      ```shell
      cmake \
        -DUSE_BUNDLED_ZLIB=ON \
        -DUSE_CHROMIUM_ZLIB=ON \
        -DCMAKE_BUILD_TYPE="RelWithDebInfo" \
        -DCMAKE_C_FLAGS="-fPIC -fno-omit-frame-pointer" \
        -GNinja \
        ..
      ninja
      perf record --call-graph=dwarf ./libgit2_clar
      perf report --children
      ```
      
      ```
      Samples: 97K of event 'cycles', Event count (approx.): 80862210917
        Children      Self  Command       Shared Objec  Symbol
      +    3.31%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_get_output_chunk
      +    2.27%     0.01%  libgit2_clar  libgit2_clar  [.] git_zstream_get_output
      +    0.55%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_get_output (inlined)
           0.18%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_init
           0.02%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_reset
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_free (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_done
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_free
      
      Samples: 97K of event 'cycles', Event count (approx.): 80862210917
        Children      Self  Command       Shared Objec  Symbol
      +    2.55%     0.01%  libgit2_clar  libgit2_clar  [.] deflate
      +    2.25%     1.41%  libgit2_clar  libgit2_clar  [.] deflate_slow
      +    1.10%     0.52%  libgit2_clar  libgit2_clar  [.] inflate
           0.36%     0.00%  libgit2_clar  libgit2_clar  [.] write_deflate
           0.30%     0.03%  libgit2_clar  libgit2_clar  [.] deflate_fast
           0.28%     0.15%  libgit2_clar  libgit2_clar  [.] inflate_fast_chunk_
           0.19%     0.19%  libgit2_clar  libgit2_clar  [.] inflate_table
           0.17%     0.01%  libgit2_clar  libgit2_clar  [.] inflateInit_
           0.16%     0.00%  libgit2_clar  libgit2_clar  [.] inflateInit2_ (inlined)
           0.15%     0.00%  libgit2_clar  libgit2_clar  [.] deflateInit_
           0.15%     0.00%  libgit2_clar  libgit2_clar  [.] deflateInit2_
           0.11%     0.01%  libgit2_clar  libgit2_clar  [.] adler32_z
           0.09%     0.09%  libgit2_clar  libgit2_clar  [.] adler32_simd_
           0.05%     0.00%  libgit2_clar  libgit2_clar  [.] deflateReset (inlined)
           0.05%     0.00%  libgit2_clar  libgit2_clar  [.] deflate_read_buf
           0.03%     0.00%  libgit2_clar  libgit2_clar  [.] inflateEnd
           0.02%     0.00%  libgit2_clar  libgit2_clar  [.] deflateReset
           0.01%     0.00%  libgit2_clar  libgit2_clar  [.] deflateEnd
           0.01%     0.01%  libgit2_clar  libgit2_clar  [.] inflateReset2
           0.01%     0.00%  libgit2_clar  libgit2_clar  [.] inflateReset (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] adler32
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] inflateResetKeep (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] deflateResetKeep
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] inflateStateCheck (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] inflateStateCheck (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] inflateStateCheck (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] deflateStateCheck (inlined)
      ```
      lhchavez committed
  3. 12 Jul, 2020 3 commits
    • cmake: use target-specific compile definitions · 4218403e
      We set up some compile definitions as part of our src/CMakeLists.txt.
      While the definitions are global, we really only need them as part of
      the git2internal target which compiles all the objects. Let's thus use
      `target_compile_definitions` instead of `add_definitions`.
      Patrick Steinhardt committed
    • cmake: use git2internal target to populate sources · 53911edd
      Modern CMake is usually target-driven in that a target is first defined
      and then the likes of `target_sources`, `target_include_directories`
      etc. are used to further populate the target. We still use old-style
      CMake, where we first set up a set of variables and then populate the
      target in a single call.
      
      Let's migrate to modern CMake usage by starting to populate the sources
      of our git2internal target piece-by-piece. While this is a small step,
      it allows us to convert to target-based build instructions
      piece-by-piece.
      Patrick Steinhardt committed
    • cmake: specify project version · 19eb1e4b
      We currently do not set up a project version within CMake, meaning that
      it can't be use by other projects including libgit2 as a sub-project and
      also not by other tools like IDEs.
      
      This commit changes this to always set up a project version, but instead
      of extracting it from the "version.h" header we now set it up directly.
      This is mostly to avoid mis-use of the previous `LIBGIT2_VERSION`
      variables, as we should now always use the `libgit2_VERSION` ones that
      are set up by CMake if one provides the "VERSION" keyword to the
      `project()` call. While this is one more moving target we need to adjust
      on releases, this commit also adjusts our release script to verify that
      the project version was incremented as expected.
      Patrick Steinhardt committed
  4. 01 Jul, 2020 2 commits
  5. 09 Jun, 2020 1 commit
    • cmake: enable warnings for missing function declarations · 03c4f86c
      Over time, we have accumulated quite a lot of functions with missing
      prototypes, missing `static` keywords or which were completely unused.
      It's easy to miss these mistakes, but luckily GCC and Clang both have
      the `-Wmissing-declarations` warning. Enabling this will cause them to
      emit warnings for every not-static function that doesn't have a previous
      declaration. This is a very sane thing to enable, and with the preceding
      commits all these new warnings have been fixed.
      
      So let's always enable this warning so we won't introduce new instances
      of them.
      Patrick Steinhardt committed
  6. 15 May, 2020 1 commit
    • cmake: Sort source files for reproducible builds · b85eefb4
      We currently use `FILE(GLOB ...)` in most places to find source and
      header files. This is problematic in that the order of files returned
      depends on the operating system's directory iteration order and may thus
      not be deterministic. As a result, we link object files in unspecified
      order, which may cause the linker to emit different code across runs.
      
      Fix this issue by sorting all code used as input to the libgit2 library
      to improve the reliability of reproducible builds.
      Patrick Steinhardt committed
  7. 14 Mar, 2020 1 commit
    • cmake: use install directories provided via GNUInstallDirs · 87fc539f
      We currently hand-code logic to configure where to install our artifacts
      via the `LIB_INSTALL_DIR`, `INCLUDE_INSTALL_DIR` and `BIN_INSTALL_DIR`
      variables. This is reinventing the wheel, as CMake already provide a way
      to do that via `CMAKE_INSTALL_<DIR>` paths, e.g. `CMAKE_INSTALL_LIB`.
      This requires users of libgit2 to know about the discrepancy and will
      require special hacks for any build systems that handle these variables
      in an automated way. One such example is Gentoo Linux, which sets up
      these paths in both the cmake and cmake-utils eclass.
      
      So let's stop doing that: the GNUInstallDirs module handles it in a
      better way for us, especially so as the actual values are dependent on
      CMAKE_INSTALL_PREFIX. This commit removes our own set of variables and
      instead refers users to use the standard ones.
      
      As a second benefit, this commit also fixes our pkgconfig generation to
      use the GNUInstallDirs module. We had a bug there where we ignored the
      CMAKE_INSTALL_PREFIX when configuring the libdir and includedir keys, so
      if libdir was set to "lib64", then libdir would be an invalid path. With
      GNUInstallDirs, we can now use `CMAKE_INSTALL_FULL_LIBDIR`, which
      handles the prefix for us.
      Patrick Steinhardt committed
  8. 13 Mar, 2020 1 commit
  9. 03 Mar, 2020 1 commit
  10. 24 Feb, 2020 1 commit
    • cmake: fix ENABLE_TRACE parameter being too strict · d8e71cb2
      In order to check whether tracing support should be turned on, we check
      whether ENABLE_TRACE equals "ON". This is being much too strict, as
      CMake will also treat "on", "true", "yes" and others as true-ish, but
      passing them will disable tracing support now.
      
      Fix the issue by simply removing the STREQUAL, which will cause CMake to
      do the right thing automatically.
      Patrick Steinhardt committed
  11. 11 Feb, 2020 1 commit
    • cmake: consolidate Valgrind option · 877054f3
      OpenSSL doesn't initialize bytes on purpose in order to generate
      additional entropy. Valgrind isn't too happy about that though, causing
      it to generate warninings about various issues regarding use of
      uninitialized bytes.
      
      We traditionally had some infrastructure to silence these errors in our
      OpenSSL stream implementation, where we invoke the Valgrind macro
      `VALGRIND_MAKE_MEMDEFINED` in various callbacks that we provide to
      OpenSSL. Naturally, we only include these instructions if a preprocessor
      define "VALGRIND" is set, and that in turn is only set if passing
      "-DVALGRIND" to CMake. We do that in our usual Azure pipelines, but we
      in fact forgot to do this in our nightly build. As a result, we get a
      slew of warnings for these nightly builds, but not for our normal
      builds.
      
      To fix this, we could just add "-DVALGRIND" to our nightly builds. But
      starting with commit d827b11b (tests: execute leak checker via CTest
      directly, 2019-06-28), we do have a secondary variable that directs
      whether we want to use memory sanitizers for our builds. As such, every
      user wishing to use Valgrind for our tests needs to pass both options
      "VALGRIND" and "USE_LEAK_CHECKER", which is cumbersome and error prone,
      as can be seen by our own builds.
      
      Instead, let's consolidate this into a single option, removing the old
      "-DVALGRIND" one. Instead, let's just add the preprocessor directive if
      USE_LEAK_CHECKER equals "valgrind" and remove "-DVALGRIND" from our own
      pipelines.
      Patrick Steinhardt committed
  12. 24 Nov, 2019 1 commit
    • valgrind: add valgrind hints in OpenSSL · cb77423f
      Provide usage hints to valgrind.  We trust the data coming back from
      OpenSSL to have been properly initialized.  (And if it has not, it's an
      OpenSSL bug, not a libgit2 bug.)
      
      We previously took the `VALGRIND` option to CMake as a hint to disable
      mmap.  Remove that; it's broken.  Now use it to pass on the `VALGRIND`
      definition so that sources can provide valgrind hints.
      Edward Thomson committed
  13. 13 Oct, 2019 1 commit
  14. 10 Oct, 2019 1 commit
    • cmake: update minimum CMake version to v3.5.1 · ebabb88f
      Back in commit cf9f3452 (cmake: bump minimum version to 2.8.11,
      2017-09-06), we have bumped the minimum CMake version to require at
      least v2.8.11. The main hold-backs back then were distributions like
      RHEL/CentOS as well as Ubuntu Trusty, which caused us to not target a
      more modern version. Nowadays, Ubuntu Trusty has been EOL'd and CentOS 6
      has CMake v3.6.1 available via the EPEL6 repository, and thus it seems
      fair to upgrade to a more recent version.
      
      Going through repology [1], one can see that all supported mainstream
      distributions do in fact have CMake 3 available. Going through the list,
      the minimum version that is supported by all mainstream distros is in
      fact v3.5.1:
      
      	- CentOS 6 via EPEL6: 3.6.1
      	- Debian Oldstable: 3.7.2
      	- Fedora 26: 3.8.2
      	- OpenMandriva 3.x: 3.5.1
      	- Slackware 14.2: 3.5.2
      	- Ubuntu 16.04: 3.5.1
      
      Consequentally, let's upgrade CMake to the minimum version of 3.5.1 and
      remove all the version CMake checks that aren't required anymore.
      
      [1]: https://repology.org/project/cmake/versions
      Patrick Steinhardt committed
  15. 14 Sep, 2019 2 commits
  16. 17 Aug, 2019 1 commit
  17. 24 Jun, 2019 1 commit
    • win32: support upgrading warnings to errors (/WX) · cc9e47c9
      For MSVC, support warnings as errors by providing the /WX compiler
      flags.  (/WX is the moral equivalent of -Werror.)
      
      Disable warnings as errors ass part of xdiff, since it contains
      warnings.  But as a component of git itself, we want to avoid skew and
      keep our implementation as similar as possible to theirs.  We'll work
      with upstream to fix these issues, but in the meantime, simply let those
      continue to warn.
      Edward Thomson committed
  18. 14 Jun, 2019 1 commit
    • cmake: Modulize our TLS & hash detection · 94fc83b6
      The interactions between `USE_HTTPS` and `SHA1_BACKEND` have been
      streamlined. Previously we would have accepted not quite working
      configurations (like, `-DUSE_HTTPS=OFF -DSHA1_BACKEND=OpenSSL`) and, as
      the OpenSSL detection only ran with `USE_HTTPS`, the link would fail.
      
      The detection was moved to a new `USE_SHA1`, modeled after `USE_HTTPS`,
      which takes the values "CollisionDetection/Backend/Generic", to better
      match how the "hashing backend" is selected, the default (ON) being
      "CollisionDetection".
      
      Note that, as `SHA1_BACKEND` is still used internally, you might need to
      check what customization you're using it for.
      Etienne Samson committed
  19. 13 Jun, 2019 1 commit
    • http-parser: use our bundled http-parser by default · fb529a01
      Our bundled http-parser includes bugfixes, therefore we should prefer
      our http-parser until such time as we can identify that the system
      http-parser has these bugfixes (using a version check).
      
      Since these bugs are - at present - minor, retain the ability for users
      to force that they want to use the system http-parser anyway.  This does
      change the cmake specification so that people _must_ opt-in to the new
      behavior knowingly.
      Edward Thomson committed
  20. 10 Jun, 2019 1 commit
  21. 21 May, 2019 5 commits
  22. 19 May, 2019 5 commits
  23. 02 May, 2019 2 commits
    • cmake: fix include ordering issues with bundled deps · ee3d71fb
      When linking against bundled libraries, we include their header
      directories by using "-isystem". The reason for that is that we
      want to handle our vendored library headers specially, most
      importantly to ignore warnings generated by including them. By
      using "-isystem", though, we screw up the order of searched
      include directories by moving those bundled dependencies towards
      the end of the lookup order. Like this, chances are high that any
      other specified include directory contains a file that collides
      with the actual desired include file.
      
      Fix this by not treating the bundled dependencies' include
      directories as system includes. This will move them to the front
      of the lookup order and thus cause them to override
      system-provided headers. While this may cause the compiler to
      generate additional warnings when processing bundled headers,
      this is a tradeoff we should make regardless to fix builds on
      systems hitting this issue.
      Patrick Steinhardt committed
    • cmake: correctly detect if system provides `regcomp` · 13cb9f7a
      We assume that if we are on Win32, Amiga OS, Solaris or SunOS,
      that the regcomp(3P) function cannot be provided by the system.
      Thus we will in these cases always include our own, bundled regex
      sources to make a regcomp implementation available. This test is
      obviously very fragile, and we have seen it fail on MSYS2/MinGW
      systems, which do in fact provide the regcomp symbol. The effect
      is that during compilation, we will use the "regex.h" header
      provided by MinGW, but use symbols provided by ourselves. This
      in fact may cause subtle memory layout issues, as the structure
      made available via MinGW doesn't match what our bundled code
      expects.
      
      There's one more problem with our regex detection: on the listed
      platforms, we will incorrectly include the bundled regex code
      even in case where the system provides regcomp_l(3), but it will
      never be used for anything.
      
      Fix the issue by improving our regcomp detection code. Instead of
      relying on a fragile listing of platforms, we can just use
      `CHECK_FUNCTION_EXISTS` instead. This will not in fact avoid the
      header-ordering problem. But we can assume that as soon as a
      system-provided "regex.h" header is provided, that
      `CHECK_FUNCTION_EXISTS` will now correctly find the desired
      symbol and thus not include our bundled regex code.
      Patrick Steinhardt committed
  24. 21 Feb, 2019 1 commit
  25. 28 Nov, 2018 1 commit
  26. 20 Oct, 2018 1 commit
    • cmake: increase WIN32_WINNT to Vista · b8bdffb5
      Increase the WIN32_WINNT level to 0x0600, which enables support for new
      APIs from Windows 6.0 (Vista).  We had previously set this to 0x0501,
      which was Windows XP.  Although we removed XP support many years ago,
      there was no need to update this level previously.  We're doing so now
      explicitly so that we can get support for the `CreateSymbolicLink` API.
      Edward Thomson committed
  27. 24 Aug, 2018 1 commit