1. 11 Jun, 2018 3 commits
  2. 09 Jun, 2018 7 commits
  3. 07 Jun, 2018 11 commits
    • Merge pull request #4576 from pks-t/pks/memory-allocator · 0ef3242e
      Custom memory allocators
      Patrick Steinhardt committed
    • settings: allow swapping out memory allocator · 74b7ddbf
      Tie in the newly created infrastructure for swapping out memory
      allocators into our settings code. A user can now simply use the new
      option "GIT_OPT_SET_ALLOCATOR" with `git_libgit2_opts`, passing in an
      already initialized allocator structure as vararg.
      Patrick Steinhardt committed
    • alloc: make memory allocators use function pointers · 9865cd16
      Currently, our memory allocators are being redirected to the correct
      implementation at compile time by simply using macros. In order to make
      them swappable at runtime, this commit reshuffles that by instead making
      use of a global "git_allocator" structure, whose pointers are set up to
      reference the allocator functions. Like this, it becomes easy to swap
      out allocators by simply setting these function pointers.
      
      In order to initialize a "git_allocator", our provided allocators
      "stdalloc" and "crtdbg" both provide an init function. This is being
      called to initialize a passed in allocator struct and set up its members
      correctly.
      
      No support is yet included to enable users of libgit2 to switch out the
      memory allocator at a global level.
      Patrick Steinhardt committed
    • stdalloc: extend allocators by file and line · 08b318c0
      Our desired architecture would make allocators completely pluggable,
      such that users of libgit2 can swap out memory allocators at runtime.
      While making e.g. debugging easier by not having to do a separate build,
      this feature can also help maintainers of bindings for libgit2 by tying
      the memory allocations into the other language's memory system.
      
      In order to do so, though, we first need to make our two different
      pre-existing allocators "stdalloc" and "crtdbg" have the same function
      signatures, as the "crtdbg" allocators all have an additional file and
      line argument. This is required to build correct stack traces for
      debugging memory allocations. As that feature may also be interesting to
      authors of other applications for debugging libgit2, we now simply add
      these arguments to our standard allocators.
      
      Obviously, this may come with a performance penalty. During some simple
      benchmarks no real impact could be measured though in contrast to a
      simple pluggable allocator. The following table summarizes the
      benchmarks. There were three different builds with our current standard
      allocator ("standard"), with pluggable authenticators accessed via
      function pointers ("pluggable") and for pluggable authenticators with
      file and line being added ("fileline"). Furthermore, there were three
      scenarios for 100.000.000 allocations of 100B ("small alloc"),
      100.000.000 allocations of 100KB ("medium alloc"), and 1.000.000
      allocations of 100MB. All results are best of 10 runs.
      
      |------------|-------------------|-------------------|-------------------|
      | build/test | small alloc       | medium alloc      | big alloc         |
      |------------|-------------------|-------------------|-------------------|
      | standard   | 4539779566, +0.0% | 5912927186, +0.0% | 5166935308, +0.0% |
      |------------|-------------------|-------------------|-------------------|
      | pluggable  | 4611074505, +1.5% | 5979185308, +1.1% | 5388776352, +4.2% |
      |------------|-------------------|-------------------|-------------------|
      | fileline   | 4588338192, +1.1% | 6004951910, +1.5% | 4942528135, -4.4% |
      |------------|-------------------|-------------------|-------------------|
      
      As can be seen, there is a performance overhead for pluggable
      allocators. Furthermore, it can also be seen that there is some big
      variance between runs, especially in the "big alloc" scenario. This is
      probably being caused by nondeterministic behaviour in the kernel for
      dynamic allocations. Still, it can be observed that there should be no
      real difference between the "pluggable" and "fileline" allocators.
      Patrick Steinhardt committed
    • util: extract allocators into its own "alloc.h" header · d2e996fa
      Our "util.h" header is a grabbag of various different functions, where
      many don't have a clear group they belong to. Our set of allocator
      functions though can be clearly singled out as a single group of
      functions that always belongs together. Furthermore, we will need to
      implement additional functions relating to our allocators subsystem when
      moving to pluggable allocators. Thus, we should just move these
      functions into their own "alloc" module.
      Patrick Steinhardt committed
    • util: extract `stdalloc` allocator into its own module · c47f7155
      Right now, the standard allocator is being declared as part of the
      "util.h" header as a set of inline functions. As with the crtdbg
      allocator functions, these inline functions make it hard to convert to
      function pointers for our allocators.
      
      Create a new "stdalloc" module containing our standard allocations
      functions to split these out. Convert the existing allocators to macros
      which make use of the stdalloc functions.
      Patrick Steinhardt committed
    • win32: crtdbg: provide independent `free` function · 496b0df2
      Currently, the `git__free` function is being defined in a single place,
      only, disregarding whether we use our standard allocators or the crtdbg
      allocators. This makes it a bit harder to convert our code base to use
      pluggable allocators, and furthermore makes the border between our two
      allocators a bit more blurry.
      
      Implement a separate `git__crtdbg__free` function for the crtdbg
      allocator in order to completely separate both allocator
      implementations.
      Patrick Steinhardt committed
    • win32: crtdbg: internalize implementation of allocators · aab8f87b
      The crtdbg allocators are currently being implemented as inline
      functions as part of the "w32_crtdbg_stacktrace.h" header. As we are
      moving towards pluggable allocators with the help of function pointers,
      though, we cannot make use of inlining anymore. Instead, we can only
      have a single implementation of these allocating functions.
      
      Move all implementations of the crtdbg allocators into
      "w32_crtdbg_stacktrace.c".
      Patrick Steinhardt committed
    • Merge pull request #4655 from glaubitz/alignment · 422cd59b
      index: Fix alignment issues in write_disk_entry()
      Patrick Steinhardt committed
    • Merge pull request #4558 from tiennou/travis/war-on-leaks · 534b70af
      travis: war on leaks
      Patrick Steinhardt committed
  4. 06 Jun, 2018 19 commits
    • Fix stash save bug with fast path index check · 5a7d454b
      If the index contains stat data for a modified file, and the file is
      not racily dirty, and there exists an untracked working tree directory
      alphabetically after that file, and there are no other changes to the
      repo, then git_stash_save would fail. It would confuse the untracked
      working tree directory for the modified file, because they have the
      same sha: zero.  The wt directory has a sha of zero because it's a
      directory, and the file would have a zero sha because we wouldn't read
      the file -- we would just know that it doesn't match the index.  To
      fix this confusion, we simply check mode as well as SHA.
      David Turner committed
    • Merge pull request #4665 from neithernut/fix-refdb-glob · 20306d36
      refdb_fs: fix regression: failure when globbing for non-existant references
      Patrick Steinhardt committed
    • Merge pull request #4673 from pks-t/pks/submodule-dupes-simplify-test · 991bf691
      tests: submodule: do not rely on config iteration order
      Patrick Steinhardt committed
    • travis: let cmake perform the build & install step · 1f4ada2a
      The goal is to let cmake manage the parallelism
      Etienne Samson committed
    • valgrind: silence invalid free in libc atexit handler · 234443e3
      ==17851== Invalid free() / delete / delete[] / realloc()
      ==17851==    at 0x4C2BDEC: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
      ==17851==    by 0x60BBE2B: __libc_freeres (in /lib/x86_64-linux-gnu/libc-2.19.so)
      ==17851==    by 0x4A256BC: _vgnU_freeres (in /usr/lib/valgrind/vgpreload_core-amd64-linux.so)
      ==17851==    by 0x5F8F16A: __run_exit_handlers (exit.c:97)
      ==17851==    by 0x5F8F1F4: exit (exit.c:104)
      ==17851==    by 0x5F74F4B: (below main) (libc-start.c:321)
      ==17851==  Address 0x63153c0 is 0 bytes inside data symbol "noai6ai_cached"
      Etienne Samson committed
    • valgrind: silence libssh2 leaking something from gcrypt · dd75885a
      ==2957== 912 bytes in 19 blocks are still reachable in loss record 323 of 369
      ==2957==    at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
      ==2957==    by 0x675B120: ??? (in /lib/x86_64-linux-gnu/libgcrypt.so.11.8.2)
      ==2957==    by 0x675BDF8: ??? (in /lib/x86_64-linux-gnu/libgcrypt.so.11.8.2)
      ==2957==    by 0x675FE0D: ??? (in /lib/x86_64-linux-gnu/libgcrypt.so.11.8.2)
      ==2957==    by 0x6761DC4: ??? (in /lib/x86_64-linux-gnu/libgcrypt.so.11.8.2)
      ==2957==    by 0x676477E: ??? (in /lib/x86_64-linux-gnu/libgcrypt.so.11.8.2)
      ==2957==    by 0x675B071: ??? (in /lib/x86_64-linux-gnu/libgcrypt.so.11.8.2)
      ==2957==    by 0x675B544: ??? (in /lib/x86_64-linux-gnu/libgcrypt.so.11.8.2)
      ==2957==    by 0x675914B: gcry_control (in /lib/x86_64-linux-gnu/libgcrypt.so.11.8.2)
      ==2957==    by 0x5D30EC9: libssh2_init (in /usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1)
      ==2957==    by 0x66BCCD: git_transport_ssh_global_init (ssh.c:910)
      ==2957==    by 0x616443: init_common (global.c:65)
      Etienne Samson committed
    • valgrind: silence curl_global_init leaks · c0c9e9ee
      ==18109== 664 bytes in 1 blocks are still reachable in loss record 279 of 339
      ==18109==    at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
      ==18109==    by 0x675B120: ??? (in /lib/x86_64-linux-gnu/libgcrypt.so.11.8.2)
      ==18109==    by 0x675C13C: ??? (in /lib/x86_64-linux-gnu/libgcrypt.so.11.8.2)
      ==18109==    by 0x675C296: ??? (in /lib/x86_64-linux-gnu/libgcrypt.so.11.8.2)
      ==18109==    by 0x679BD14: ??? (in /lib/x86_64-linux-gnu/libgcrypt.so.11.8.2)
      ==18109==    by 0x679CC64: ??? (in /lib/x86_64-linux-gnu/libgcrypt.so.11.8.2)
      ==18109==    by 0x6A64946: ??? (in /usr/lib/x86_64-linux-gnu/libgnutls.so.26.22.6)
      ==18109==    by 0x6A116E8: ??? (in /usr/lib/x86_64-linux-gnu/libgnutls.so.26.22.6)
      ==18109==    by 0x6A01114: gnutls_global_init (in /usr/lib/x86_64-linux-gnu/libgnutls.so.26.22.6)
      ==18109==    by 0x52A6C78: ??? (in /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.3.0)
      ==18109==    by 0x5285ADC: curl_global_init (in /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.3.0)
      ==18109==    by 0x663524: git_curl_stream_global_init (curl.c:44)
      Etienne Samson committed
    • tests: submodule: do not rely on config iteration order · 8178c70f
      The test submodule::lookup::duplicated_path, which tries to verify that
      we detect submodules with duplicated paths, currently relies on the
      gitmodules file of "submod2_target". While this file has two gitmodules
      with the same path, one of these gitmodules has an empty name and thus
      does not pass `git_submodule_name_is_valid`. Because of this, the test
      is in fact dependent on the iteration order in which we process the
      submodules. In fact the "valid" submodule comes first, the "invalid"
      submodule will cause the desired error. In fact the "invalid" submodule
      comes first, it will be skipped due to its name being invalid, and we
      will not see the desired error. While this works on the master branch
      just right due to the refactoring of our config code, where iteration
      order is now deterministic, this breaks on all older maintenance
      branches.
      
      Fix the issue by simply using `cl_git_rewritefile` to rewrite the
      gitmodules file. This greatly simplifies the test and also makes the
      intentions of it much clearer.
      Patrick Steinhardt committed
    • Merge pull request #4641 from pks-t/pks/submodule-names-memleak · 54990d75
      Detect duplicated submodules for the same path
      Patrick Steinhardt committed
    • ignore: remove now-useless check for LEADINGDIR · d22fd81c
      When checking whether a rule negates another rule, we were checking
      whether a rule had the `GIT_ATTR_FNMATCH_LEADINGDIR` flag set and, if
      so, added a "/*" to its end before passing it to `fnmatch`. Our code now
      sets `GIT_ATTR_FNMATCH_NOLEADINGDIR`, thus the `LEADINGDIR` flag shall
      never be set. Furthermore, due to the `NOLEADINGDIR` flag, trailing
      globs do not get consumed by our ignore parser anymore.
      
      Clean up code by just dropping this now useless logic.
      Patrick Steinhardt committed
    • ignore: fix negative leading directory rules unignoring subdirectory files · 20b4c175
      When computing whether a file is ignored, we simply search for the first
      matching rule and return whether it is a positive ignore rule (the file
      is really ignored) or whether it is a negative ignore rule (the file is
      being unignored). Each rule has a set of flags which are being passed to
      `fnmatch`, depending on what kind of rule it is. E.g. in case it is a
      negative ignore we add a flag `GIT_ATTR_FNMATCH_NEGATIVE`, in case it
      contains a glob we set the `GIT_ATTR_FNMATCH_HASGLOB` flag.
      
      One of these flags is the `GIT_ATTR_FNMATCH_LEADINGDIR` flag, which is
      always set in case the pattern has a trailing "/*" or in case the
      pattern is negative. The flag causes the `fnmatch` function to return a
      match in case a string is a leading directory of another, e.g. "dir/"
      matches "dir/foo/bar.c". In case of negative patterns, this is wrong in
      certain cases.
      
      Take the following simple example of a gitignore:
      
          dir/
          !dir/
      
      The `LEADINGDIR` flag causes "!dir/" to match "dir/foo/bar.c", and we
      correctly unignore the directory. But take this example:
      
          *.test
          !dir/*
      
      We expect everything in "dir/" to be unignored, but e.g. a file in a
      subdirectory of dir should be ignored, as the "*" does not cross
      directory hierarchies. With `LEADINGDIR`, though, we would just see that
      "dir/" matches and return that the file is unignored, even if it is
      contained in a subdirectory. Instead, we want to ignore leading
      directories here and check "*.test". Afterwards, we have to iterate up
      to the parent directory and do the same checks.
      
      To fix the issue, disallow matching against leading directories in
      gitignore files. This can be trivially done by just adding the
      `GIT_ATTR_FNMATCH_NOLEADINGDIR` to the spec passed to
      `git_attr_fnmatch__parse`. Due to a bug in that function, though, this
      flag is being ignored for negative patterns, which is fixed in this
      commit, as well. As a last fix, we need to ignore rules that are
      supposed to match a directory when our path itself is a file.
      
      All together, these changes fix the described error case.
      Patrick Steinhardt committed