1. 18 Sep, 2020 1 commit
  2. 09 Sep, 2020 4 commits
  3. 01 Jun, 2020 1 commit
  4. 07 Feb, 2020 1 commit
  5. 28 Nov, 2019 1 commit
  6. 06 Nov, 2019 1 commit
  7. 05 Nov, 2019 1 commit
    • config_entries: micro-optimize storage of multivars · b7dcea04
      Multivars are configuration entries that have many values for the same
      name; we can thus micro-optimize this case by just retaining the name of
      the first configuration entry and freeing all the others, letting them
      point to the string of the first entry.
      
      The attached test case is an extreme example that demonstrates this. It
      contains a section name that is approximately 500kB in size with 20.000
      entries "a=b". Without the optimization, this would require at least
      20000*500kB bytes, which is around 10GB. With this patch, it only
      requires 500kB+20000*1B=20500kB.
      
      The obvious culprit here is the section header, which we repeatedly
      include in each of the configuration entry's names. This makes it very
      easier for an adversary to provide a small configuration file that
      disproportionally blows up in memory during processing and is thus a
      feasible way for a denial-of-service attack. Unfortunately, we cannot
      fix the root cause by e.g. having a separate "section" field that may
      easily be deduplicated due to the `git_config_entry` structure being
      part of our public API. So this micro-optimization is the best we can do
      for now.
      Patrick Steinhardt committed
  8. 01 Nov, 2019 1 commit
  9. 01 Aug, 2019 4 commits
  10. 21 Jul, 2019 1 commit
    • config_file: refresh when creating an iterator · 2766b92d
      When creating a new iterator for a config file backend, then we should
      always make sure that we're up to date by calling `config_refresh`.
      Otherwise, we might not notice when another process has modified the
      configuration file and thus will represent outdated values.
      
      Add two tests to config::stress that verify that we get up-to-date
      values when reading configuration entries via `git_config_iterator`.
      Patrick Steinhardt committed
  11. 20 Jul, 2019 1 commit
  12. 11 Jul, 2019 1 commit
    • config_file: implement stat cache to avoid repeated rehashing · d7f58eab
      To decide whether a config file has changed, we always hash its
      complete contents. This is unnecessarily expensive, as
      well-behaved filesystems will always update stat information for
      files which have changed. So before computing the hash, we should
      first check whether the stat info has actually changed for either
      the configuration file or any of its includes. This avoids having
      to re-read the configuration file and its includes every time
      when we check whether it's been modified.
      
      Tracing the for-each-ref example previous to this commit, one can
      see that we repeatedly re-open both the repo configuration as
      well as the global configuration:
      
      	$ strace lg2 for-each-ref |& grep config
      	access("/home/pks/.gitconfig", F_OK)    = -1 ENOENT (No such file or directory)
      	access("/home/pks/.config/git/config", F_OK) = 0
      	access("/etc/gitconfig", F_OK)          = -1 ENOENT (No such file or directory)
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	access("/tmp/repo/.git/config", F_OK)   = 0
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/home/pks/.gitconfig", 0x7ffd15c05290) = -1 ENOENT (No such file or directory)
      	access("/home/pks/.gitconfig", F_OK)    = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	access("/home/pks/.config/git/config", F_OK) = 0
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/home/pks/.gitconfig", 0x7ffd15c051f0) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/home/pks/.gitconfig", 0x7ffd15c05090) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/home/pks/.gitconfig", 0x7ffd15c05090) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/home/pks/.gitconfig", 0x7ffd15c05090) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      
      With the change, we only do stats for those files and open them a
      single time, only:
      
      	$ strace lg2 for-each-ref |& grep config
      	access("/home/pks/.gitconfig", F_OK)    = -1 ENOENT (No such file or directory)
      	access("/home/pks/.config/git/config", F_OK) = 0
      	access("/etc/gitconfig", F_OK)          = -1 ENOENT (No such file or directory)
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	access("/tmp/repo/.git/config", F_OK)   = 0
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/home/pks/.gitconfig", 0x7ffe70540d20) = -1 ENOENT (No such file or directory)
      	access("/home/pks/.gitconfig", F_OK)    = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	access("/home/pks/.config/git/config", F_OK) = 0
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	stat("/home/pks/.gitconfig", 0x7ffe70540ca0) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.gitconfig", 0x7ffe70540c80) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	stat("/home/pks/.gitconfig", 0x7ffe70540b40) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.gitconfig", 0x7ffe70540b20) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	stat("/home/pks/.gitconfig", 0x7ffe70540b40) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.gitconfig", 0x7ffe70540b20) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	stat("/home/pks/.gitconfig", 0x7ffe70540b40) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.gitconfig", 0x7ffe70540b20) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      
      The following benchmark has been performed with and without the
      stat cache in a best-of-ten run:
      
      ```
      
      int lg2_repro(git_repository *repo, int argc, char **argv)
      {
      	git_config *cfg;
      	int32_t dummy;
      	int i;
      
      	UNUSED(argc);
      	UNUSED(argv);
      
      	check_lg2(git_repository_config(&cfg, repo),
      			"Could not obtain config", NULL);
      
      	for (i = 1; i < 100000; ++i)
      		git_config_get_int32(&dummy, cfg, "foo.bar");
      
      	git_config_free(cfg);
      	return 0;
      }
      ```
      
      Without stat cache:
      
      	$ time lg2 repro
      	real    0m1.528s
      	user    0m0.568s
      	sys     0m0.944s
      
      With stat cache:
      
      	$ time lg2 repro
      	real    0m0.526s
      	user    0m0.268s
      	sys     0m0.258s
      
      This benchmark shows a nearly three-fold performance improvement.
      
      This change requires that we check our configuration stress tests
      as we're now in fact becoming more racy. If somebody is writing a
      configuration file at nearly the same time (there is a window of
      100ns on Windows-based systems), then it might be that we realize
      that this file has actually changed and thus may not re-read it.
      This will only happen if either an external process is rewriting
      the configuration file or if the same process has multiple
      `git_config` structures pointing to the same time, where one of
      both is being used to write and the other one is used to read
      values.
      Patrick Steinhardt committed
  13. 15 Jun, 2019 2 commits
    • config_file: use `wildmatch` to evaluate conditionals · 5811e3ba
      We currently use `p_fnmatch` to compute whether a given "gitdir:"
      or "gitdir/i:" conditional matches the current configuration file
      path. As git.git has moved to use `wildmatch` instead of
      `p_fnmatch` throughout its complete codebase, we evaluate
      conditionals inconsistently with git.git in some special cases.
      
      Convert `p_fnmatch` to use `wildmatch`. The `FNM_LEADINGDIR` flag
      cannot be translated to `wildmatch`, but in fact git.git doesn't
      use it here either. And in fact, dropping it while we go
      increases compatibility with git.git.
      Patrick Steinhardt committed
    • config_file: do not include trailing '/' for "gitdir" conditionals · cf1a114b
      When evaluating "gitdir:" and "gitdir/i:" conditionals, we
      currently compare the given pattern with the value of
      `git_repository_path`. Thing is though that `git_repository_path`
      returns the gitdir path with trailing '/', while we actually need
      to match against the gitdir without it.
      
      Fix this issue by stripping the trailing '/' previous to
      matching. Add various tests to ensure we get this right.
      Patrick Steinhardt committed
  14. 22 May, 2019 1 commit
    • config: validate quoted section value · 23c5699e
      When we reach a whitespace after a section name, we assume that what
      will follow will be a quoted subsection name.  Pass the current position
      of the line being parsed to the subsection parser, so that it can
      validate that subsequent characters are additional whitespace or a
      single quote.
      
      Previously we would begin parsing after the section name, looking for
      the first quotation mark.  This allows invalid characters to embed
      themselves between the end of the section name and the first quotation
      mark, eg `[section foo "subsection"]`, which is illegal.
      Edward Thomson committed
  15. 30 Apr, 2019 1 commit
  16. 29 Mar, 2019 3 commits
  17. 22 Jan, 2019 1 commit
  18. 28 Nov, 2018 1 commit
    • config: fix adding files if their parent directory is a file · 43cbe6b7
      When we try to add a configuration file with `git_config_add_file_ondisk`, we
      treat nonexisting files as empty. We do this by performing a stat call, ignoring
      ENOENT errors. This works just fine in case the file or any of its parents
      simply does not exist, but there is also the case where any of the parent
      directories is not a directory, but a file. So e.g. trying to add a
      configuration file "/dev/null/.gitconfig" will fail, as `errno` will be ENOTDIR
      instead of ENOENT.
      
      Catch ENOTDIR in addition to ENOENT to fix the issue. Add a test that verifies
      we are able to add configuration files with such an invalid path file just fine.
      Patrick Steinhardt committed
  19. 15 Oct, 2018 1 commit
  20. 09 Oct, 2018 1 commit
  21. 05 Oct, 2018 2 commits
    • config_file: properly ignore includes without "path" value · d06d4220
      In case a configuration includes a key "include.path=" without any
      value, the generated configuration entry will have its value set to
      `NULL`. This is unexpected by the logic handling includes, and as soon
      as we try to calculate the included path we will unconditionally
      dereference that `NULL` pointer and thus segfault.
      
      Fix the issue by returning early in both `parse_include` and
      `parse_conditional_include` in case where the `file` argument is `NULL`.
      Add a test to avoid future regression.
      
      The issue has been found by the oss-fuzz project, issue 10810.
      Patrick Steinhardt committed
    • tests: always unlink created config files · bf662f7c
      While our tests in config::include create a plethora of configuration
      files, most of them do not get removed at the end of each test. This can
      cause weird interactions with tests that are being run at a later stage
      if these later tests try to create files or directories with the same
      name as any of the created configuration files.
      
      Fix the issue by unlinking all created files at the end of these tests.
      Patrick Steinhardt committed
  22. 28 Sep, 2018 2 commits
    • config: introduce new read-only in-memory backend · 2be39cef
      Now that we have abstracted away how to store and retrieve config
      entries, it became trivial to implement a new in-memory backend by
      making use of this. And thus we do so.
      
      This commit implements a new read-only in-memory backend that can parse
      a chunk of memory into a `git_config_backend` structure.
      Patrick Steinhardt committed
    • config: rename "config_file.h" to "config_backend.h" · b944e137
      The header "config_file.h" has a list of inline-functions to access the
      contents of a config backend without directly messing with the struct's
      function pointers. While all these functions are called
      "git_config_file_*", they are in fact completely backend-agnostic and
      don't care whether it is a file or not. Rename all the function to
      instead be backend-agnostic versions called "git_config_backend_*" and
      rename the header to match.
      Patrick Steinhardt committed
  23. 14 Aug, 2018 2 commits
  24. 13 Jul, 2018 1 commit
    • treewide: remove use of C++ style comments · 9994cd3f
      C++ style comment ("//") are not specified by the ISO C90 standard and
      thus do not conform to it. While libgit2 aims to conform to C90, we did
      not enforce it until now, which is why quite a lot of these
      non-conforming comments have snuck into our codebase. Do a tree-wide
      conversion of all C++ style comments to the supported C style comments
      to allow us enforcing strict C90 compliance in a later commit.
      Patrick Steinhardt committed
  25. 10 Jun, 2018 1 commit
  26. 26 Mar, 2018 1 commit
  27. 08 Feb, 2018 2 commits
    • config_parse: fix reading files with BOM · 2eea5f1c
      The function `skip_bom` is being used to detect and skip BOM marks
      previously to parsing a configuration file. To do so, it simply uses
      `git_buf_text_detect_bom`. But since the refactoring to use the parser
      interface in commit 9e66590b (config_parse: use common parser
      interface, 2017-07-21), the BOM detection was actually broken.
      
      The issue stems from a misunderstanding of `git_buf_text_detect_bom`. It
      was assumed that its third parameter limits the length of the character
      sequence that is to be analyzed, while in fact it was an offset at which
      we want to detect the BOM. Fix the parameter to be `0` instead of the
      buffer length, as we always want to check the beginning of the
      configuration file.
      Patrick Steinhardt committed
    • config_parse: handle empty lines with CRLF · 848153f3
      Currently, the configuration parser will fail reading empty lines with
      just an CRLF-style line ending. Special-case the '\r' character in order
      to handle it the same as Unix-style line endings. Add tests to spot this
      regression in the future.
      Patrick Steinhardt committed