1. 07 Jan, 2021 2 commits
  2. 05 Jan, 2021 2 commits
  3. 04 Jan, 2021 1 commit
  4. 30 Dec, 2020 1 commit
  5. 27 Dec, 2020 1 commit
  6. 24 Dec, 2020 1 commit
    • zlib: Add support for building with Chromium's zlib implementation · 83265b3e
      This change builds libgit2 using Chromium's zlib implementation by
      invoking cmake with `-DUSE_BUNDLED_ZLIB=ON -DUSE_CHROMIUM_ZLIB=ON`,
      which is ~10% faster than the bundled zlib for the core::zstream suite.
      
      This version of zlib has some optimizations:
      
      a) Decompression (Intel+ARM): inflate_fast, adler32, crc32, etc.
      b) Compression (Intel): fill_window, longest_match, hash function, etc.
      
      Due to the introduction of SIMD optimizations, and to get the maximum
      performance out of this fork of zlib, this requires an x86_64 processor
      with SSE4.2 and CLMUL (anything Westmere or later, ~2010). The Chromium
      zlib implementation also supports ARM with NEON, but it has not been
      enabled in this patch.
      
      Performance
      ===========
      
      TL;DR: Running just `./libgit2_clar -score::zstream` 100 times in a loop
      took 0:56.30 before and 0:50.67 after (~10% reduction!).
      
      The bundled and system zlib implementations on an Ubuntu Focal system
      perform relatively similar (the bundled one is marginally better due to
      the compiler being able to inline some functions), so only the bundled
      and Chromium zlibs were compared.
      
      For a more balanced comparison (to ensure that nothing regressed
      overall), `libgit2_clar` under `perf` was also run, and the zlib-related
      functions were compared.
      
      Bundled
      -------
      
      ```shell
      cmake \
        -DUSE_BUNDLED_ZLIB=ON \
        -DUSE_CHROMIUM_ZLIB=OFF \
        -DCMAKE_BUILD_TYPE="RelWithDebInfo" \
        -DCMAKE_C_FLAGS="-fPIC -fno-omit-frame-pointer" \
        -GNinja \
        ..
      ninja
      perf record --call-graph=dwarf ./libgit2_clar
      perf report --children
      ```
      
      ```
      Samples: 87K of event 'cycles', Event count (approx.): 75923450603
        Children      Self  Command       Shared Objec  Symbol
      +    4.14%     0.01%  libgit2_clar  libgit2_clar  [.] git_zstream_get_output_chunk
      +    2.91%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_get_output
      +    0.69%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_get_output (inlined)
           0.17%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_init
           0.02%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_reset
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_eos
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_done
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_free (inlined)
      
      Samples: 87K of event 'cycles', Event count (approx.): 75923450603
        Children      Self  Command       Shared Objec  Symbol
      +    3.12%     0.01%  libgit2_clar  libgit2_clar  [.] deflate
      +    2.65%     1.48%  libgit2_clar  libgit2_clar  [.] deflate_slow
      +    1.60%     0.55%  libgit2_clar  libgit2_clar  [.] inflate
      +    0.53%     0.00%  libgit2_clar  libgit2_clar  [.] write_deflate
           0.49%     0.36%  libgit2_clar  libgit2_clar  [.] inflate_fast
           0.46%     0.02%  libgit2_clar  libgit2_clar  [.] deflate_fast
           0.19%     0.19%  libgit2_clar  libgit2_clar  [.] inflate_table
           0.16%     0.01%  libgit2_clar  libgit2_clar  [.] inflateInit_
           0.15%     0.00%  libgit2_clar  libgit2_clar  [.] inflateInit2_ (inlined)
           0.10%     0.00%  libgit2_clar  libgit2_clar  [.] deflateInit_
           0.10%     0.00%  libgit2_clar  libgit2_clar  [.] deflateInit2_
           0.03%     0.00%  libgit2_clar  libgit2_clar  [.] deflateReset (inlined)
           0.02%     0.00%  libgit2_clar  libgit2_clar  [.] deflateReset
           0.02%     0.00%  libgit2_clar  libgit2_clar  [.] inflateEnd
           0.02%     0.00%  libgit2_clar  libgit2_clar  [.] deflateEnd
           0.01%     0.00%  libgit2_clar  libgit2_clar  [.] deflateResetKeep
           0.01%     0.01%  libgit2_clar  libgit2_clar  [.] inflateReset2
           0.01%     0.00%  libgit2_clar  libgit2_clar  [.] deflateReset (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] inflateStateCheck (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] inflateReset (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] inflateStateCheck (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] deflateStateCheck (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] inflateResetKeep (inlined)
      ```
      
      Chromium
      --------
      
      ```shell
      cmake \
        -DUSE_BUNDLED_ZLIB=ON \
        -DUSE_CHROMIUM_ZLIB=ON \
        -DCMAKE_BUILD_TYPE="RelWithDebInfo" \
        -DCMAKE_C_FLAGS="-fPIC -fno-omit-frame-pointer" \
        -GNinja \
        ..
      ninja
      perf record --call-graph=dwarf ./libgit2_clar
      perf report --children
      ```
      
      ```
      Samples: 97K of event 'cycles', Event count (approx.): 80862210917
        Children      Self  Command       Shared Objec  Symbol
      +    3.31%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_get_output_chunk
      +    2.27%     0.01%  libgit2_clar  libgit2_clar  [.] git_zstream_get_output
      +    0.55%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_get_output (inlined)
           0.18%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_init
           0.02%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_reset
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_free (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_done
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] git_zstream_free
      
      Samples: 97K of event 'cycles', Event count (approx.): 80862210917
        Children      Self  Command       Shared Objec  Symbol
      +    2.55%     0.01%  libgit2_clar  libgit2_clar  [.] deflate
      +    2.25%     1.41%  libgit2_clar  libgit2_clar  [.] deflate_slow
      +    1.10%     0.52%  libgit2_clar  libgit2_clar  [.] inflate
           0.36%     0.00%  libgit2_clar  libgit2_clar  [.] write_deflate
           0.30%     0.03%  libgit2_clar  libgit2_clar  [.] deflate_fast
           0.28%     0.15%  libgit2_clar  libgit2_clar  [.] inflate_fast_chunk_
           0.19%     0.19%  libgit2_clar  libgit2_clar  [.] inflate_table
           0.17%     0.01%  libgit2_clar  libgit2_clar  [.] inflateInit_
           0.16%     0.00%  libgit2_clar  libgit2_clar  [.] inflateInit2_ (inlined)
           0.15%     0.00%  libgit2_clar  libgit2_clar  [.] deflateInit_
           0.15%     0.00%  libgit2_clar  libgit2_clar  [.] deflateInit2_
           0.11%     0.01%  libgit2_clar  libgit2_clar  [.] adler32_z
           0.09%     0.09%  libgit2_clar  libgit2_clar  [.] adler32_simd_
           0.05%     0.00%  libgit2_clar  libgit2_clar  [.] deflateReset (inlined)
           0.05%     0.00%  libgit2_clar  libgit2_clar  [.] deflate_read_buf
           0.03%     0.00%  libgit2_clar  libgit2_clar  [.] inflateEnd
           0.02%     0.00%  libgit2_clar  libgit2_clar  [.] deflateReset
           0.01%     0.00%  libgit2_clar  libgit2_clar  [.] deflateEnd
           0.01%     0.01%  libgit2_clar  libgit2_clar  [.] inflateReset2
           0.01%     0.00%  libgit2_clar  libgit2_clar  [.] inflateReset (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] adler32
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] inflateResetKeep (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] deflateResetKeep
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] inflateStateCheck (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] inflateStateCheck (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] inflateStateCheck (inlined)
           0.00%     0.00%  libgit2_clar  libgit2_clar  [.] deflateStateCheck (inlined)
      ```
      lhchavez committed
  7. 23 Dec, 2020 4 commits
  8. 22 Dec, 2020 2 commits
  9. 20 Dec, 2020 2 commits
  10. 19 Dec, 2020 2 commits
  11. 18 Dec, 2020 1 commit
  12. 15 Dec, 2020 1 commit
  13. 13 Dec, 2020 1 commit
    • Make git__strntol64() ~70%* faster · e99e833f
      This change uses compiler intrinsics to detect overflows instead of
      using divisions to detect potential overflow. This makes the code faster
      and makes it easier to read as a bonus side-effect!
      
      Some of the things this quickens:
      
      * Config parsing.
      * Tree parsing.
      * Smart protocol negotiation.
      
      \* Measured by running `libgit2_clar` with `-fno-optimize-sibling-calls
      -fno-omit-frame-pointer` under `perf(1)`:
      
      ```shell
      $ perf diff --symbols=git__strntol64 --compute=ratio \
        --percentage=absolute baseline.data perf.data
      \# Event 'cycles'
      \#
      \# Baseline           Ratio  Shared Object
      \# ........  ..............  .............
      \#
           0.25%        0.321836  libgit2_clar
      ```
      lhchavez committed
  14. 11 Dec, 2020 3 commits
  15. 09 Dec, 2020 2 commits
    • alloc: set up an allocator that fails before library init · 08f28ff5
      We require the library to be initialized with git_libgit2_init before it
      is functional.  However, if a user tries to uses the library without
      doing so - as they might when getting started with the library for the
      first time - we will likely crash.
      
      This commit introduces some guard rails - now instead of having _no_
      allocator by default, we'll have an allocator that always fails, and
      never tries to set an error message (since the thread-local state is
      set up by git_libgit2_init).  We've modified the error retrieval
      function to (try to) ensure that the library has been initialized before
      getting the thread-local error message.
      
      (Unfortunately, we cannot determine if the thread local storage has
      actually been configured, this does require initialization by
      git_libgit2_init.  But a naive attempt should be good enough for most
      cases.)
      Edward Thomson committed
    • libgit2: provide init_count of the library · 6c51014d
      A function to provide the initialization count of the library; this is
      subject to race conditions but is useful for a naive determination as to
      whether the library has been initialized or not.
      Edward Thomson committed
  16. 08 Dec, 2020 3 commits
  17. 06 Dec, 2020 5 commits
  18. 05 Dec, 2020 2 commits
  19. 29 Nov, 2020 1 commit
    • Make the pack and mwindow implementations data-race-free · 322c15ee
      This change fixes a packfile heap corruption that can happen when
      interacting with multiple packfiles concurrently across multiple
      threads. This is exacerbated by setting a lower mwindow open file limit.
      
      This change:
      
      * Renames most of the internal methods in pack.c to clearly indicate
        that they expect to be called with a certain lock held, making
        reasoning about the state of locks a bit easier.
      * Splits the `git_pack_file` lock in two: the one in `git_pack_file`
        only protects the `index_map`. The protection to `git_mwindow_file` is
        now in that struct.
      * Explicitly checks for freshness of the `git_pack_file` in
        `git_packfile_unpack_header`: this allows the mwindow implementation
        to close files whenever there is enough cache pressure, and
        `git_packfile_unpack_header` will reopen the packfile if needed.
      * After a call to `p_munmap()`, the `data` and `len` fields are poisoned
        with `NULL` to make use-after-frees more evident and crash rather than
        being open to the possibility of heap corruption.
      * Adds a test case to prevent this from regressing in the future.
      
      Fixes: #5591
      lhchavez committed
  20. 28 Nov, 2020 1 commit
  21. 27 Nov, 2020 2 commits
    • Fix the non-debug build in Groovy Gorilla (gcc 10.2) · 5db304f0
      The release build has been failing with the following error:
      
      ```shell
      FAILED: src/CMakeFiles/git2internal.dir/hash.c.o
      /usr/bin/cc -DHAVE_QSORT_R_GNU -DSHA1DC_CUSTOM_INCLUDE_SHA1_C=\"common.h\" -DSHA1DC_CUSTOM_INCLUDE_UBC_CHECK_C=\"common.h\" -DSHA1DC_NO_STANDARD_INCLUDES=1 -D_FILE_OFFSET_BITS=64 -Isrc -I../src -I../include -I../deps/pcre -I../deps/http-parser -D_GNU_SOURCE -fsanitize=thread -fno-optimize-sibling-calls -fno-omit-frame-pointer -Werror -Wall -Wextra -fvisibility=
      hidden -fPIC -Wno-documentation-deprecated-sync -Wno-missing-field-initializers -Wstrict-aliasing -Wstrict-prototypes -Wdeclaration-after-statement -Wshift-count-overflow -Wunused-const-variable -Wunused-function -Wint-conversion -Wformat -Wformat-security -Wmissing-declarations -g -D_DEBUG -O0   -std=gnu90 -MD -MT src/CMakeFiles/git2internal.dir/hash.c.o -MF s
      rc/CMakeFiles/git2internal.dir/hash.c.o.d -o src/CMakeFiles/git2internal.dir/hash.c.o   -c ../src/hash.c
      ../src/hash.c: In function ‘git_hash_init’:
      ../src/hash.c:47:1: error: control reaches end of non-void function [-Werror=return-type]
         47 | }
            | ^
      ../src/hash.c: In function ‘git_hash_update’:
      ../src/hash.c:58:1: error: control reaches end of non-void function [-Werror=return-type]
         58 | }
            | ^
      ../src/hash.c: In function ‘git_hash_final’:
      ../src/hash.c:68:1: error: control reaches end of non-void function [-Werror=return-type]
         68 | }
            | ^
      ../src/hash.c: At top level:
      cc1: note: unrecognized command-line option ‘-Wno-documentation-deprecated-sync’ may have been intended to silence earlier diagnostics
      cc1: all warnings being treated as errors
      [11/533] Building C object src/CMakeFiles/git2internal.dir/odb_pack.c.o
      ninja: build stopped: subcommand failed.
      ```
      
      The compiler _should_ be able to figure out that there is no way to reach the
      end of the non-void function since `GIT_ASSERT(0)` expands to either `assert()`
      or an unconditional `return -1;` (after doing constant folding and stuff,
      depending on the debug level). But it's not doing so at the moment, so let's
      help it.
      lhchavez committed
    • Also add the raw hostkey to `git_cert_hostkey` · 29fe5f61
      `git_cert_x509` has the raw encoded certificate. Let's do the same for
      the SSH certificate for symmetry.
      lhchavez committed