1. 28 Nov, 2019 1 commit
  2. 05 Nov, 2019 1 commit
    • config_entries: micro-optimize storage of multivars · b7dcea04
      Multivars are configuration entries that have many values for the same
      name; we can thus micro-optimize this case by just retaining the name of
      the first configuration entry and freeing all the others, letting them
      point to the string of the first entry.
      
      The attached test case is an extreme example that demonstrates this. It
      contains a section name that is approximately 500kB in size with 20.000
      entries "a=b". Without the optimization, this would require at least
      20000*500kB bytes, which is around 10GB. With this patch, it only
      requires 500kB+20000*1B=20500kB.
      
      The obvious culprit here is the section header, which we repeatedly
      include in each of the configuration entry's names. This makes it very
      easier for an adversary to provide a small configuration file that
      disproportionally blows up in memory during processing and is thus a
      feasible way for a denial-of-service attack. Unfortunately, we cannot
      fix the root cause by e.g. having a separate "section" field that may
      easily be deduplicated due to the `git_config_entry` structure being
      part of our public API. So this micro-optimization is the best we can do
      for now.
      Patrick Steinhardt committed
  3. 21 Jul, 2019 1 commit
    • config_file: refresh when creating an iterator · 2766b92d
      When creating a new iterator for a config file backend, then we should
      always make sure that we're up to date by calling `config_refresh`.
      Otherwise, we might not notice when another process has modified the
      configuration file and thus will represent outdated values.
      
      Add two tests to config::stress that verify that we get up-to-date
      values when reading configuration entries via `git_config_iterator`.
      Patrick Steinhardt committed
  4. 20 Jul, 2019 1 commit
  5. 11 Jul, 2019 1 commit
    • config_file: implement stat cache to avoid repeated rehashing · d7f58eab
      To decide whether a config file has changed, we always hash its
      complete contents. This is unnecessarily expensive, as
      well-behaved filesystems will always update stat information for
      files which have changed. So before computing the hash, we should
      first check whether the stat info has actually changed for either
      the configuration file or any of its includes. This avoids having
      to re-read the configuration file and its includes every time
      when we check whether it's been modified.
      
      Tracing the for-each-ref example previous to this commit, one can
      see that we repeatedly re-open both the repo configuration as
      well as the global configuration:
      
      	$ strace lg2 for-each-ref |& grep config
      	access("/home/pks/.gitconfig", F_OK)    = -1 ENOENT (No such file or directory)
      	access("/home/pks/.config/git/config", F_OK) = 0
      	access("/etc/gitconfig", F_OK)          = -1 ENOENT (No such file or directory)
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	access("/tmp/repo/.git/config", F_OK)   = 0
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/home/pks/.gitconfig", 0x7ffd15c05290) = -1 ENOENT (No such file or directory)
      	access("/home/pks/.gitconfig", F_OK)    = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	access("/home/pks/.config/git/config", F_OK) = 0
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/home/pks/.gitconfig", 0x7ffd15c051f0) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/home/pks/.gitconfig", 0x7ffd15c05090) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/home/pks/.gitconfig", 0x7ffd15c05090) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/home/pks/.gitconfig", 0x7ffd15c05090) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      
      With the change, we only do stats for those files and open them a
      single time, only:
      
      	$ strace lg2 for-each-ref |& grep config
      	access("/home/pks/.gitconfig", F_OK)    = -1 ENOENT (No such file or directory)
      	access("/home/pks/.config/git/config", F_OK) = 0
      	access("/etc/gitconfig", F_OK)          = -1 ENOENT (No such file or directory)
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	access("/tmp/repo/.git/config", F_OK)   = 0
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/home/pks/.gitconfig", 0x7ffe70540d20) = -1 ENOENT (No such file or directory)
      	access("/home/pks/.gitconfig", F_OK)    = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	access("/home/pks/.config/git/config", F_OK) = 0
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	stat("/home/pks/.gitconfig", 0x7ffe70540ca0) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.gitconfig", 0x7ffe70540c80) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	stat("/home/pks/.gitconfig", 0x7ffe70540b40) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.gitconfig", 0x7ffe70540b20) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	stat("/home/pks/.gitconfig", 0x7ffe70540b40) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.gitconfig", 0x7ffe70540b20) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	stat("/home/pks/.gitconfig", 0x7ffe70540b40) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.gitconfig", 0x7ffe70540b20) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      
      The following benchmark has been performed with and without the
      stat cache in a best-of-ten run:
      
      ```
      
      int lg2_repro(git_repository *repo, int argc, char **argv)
      {
      	git_config *cfg;
      	int32_t dummy;
      	int i;
      
      	UNUSED(argc);
      	UNUSED(argv);
      
      	check_lg2(git_repository_config(&cfg, repo),
      			"Could not obtain config", NULL);
      
      	for (i = 1; i < 100000; ++i)
      		git_config_get_int32(&dummy, cfg, "foo.bar");
      
      	git_config_free(cfg);
      	return 0;
      }
      ```
      
      Without stat cache:
      
      	$ time lg2 repro
      	real    0m1.528s
      	user    0m0.568s
      	sys     0m0.944s
      
      With stat cache:
      
      	$ time lg2 repro
      	real    0m0.526s
      	user    0m0.268s
      	sys     0m0.258s
      
      This benchmark shows a nearly three-fold performance improvement.
      
      This change requires that we check our configuration stress tests
      as we're now in fact becoming more racy. If somebody is writing a
      configuration file at nearly the same time (there is a window of
      100ns on Windows-based systems), then it might be that we realize
      that this file has actually changed and thus may not re-read it.
      This will only happen if either an external process is rewriting
      the configuration file or if the same process has multiple
      `git_config` structures pointing to the same time, where one of
      both is being used to write and the other one is used to read
      values.
      Patrick Steinhardt committed
  6. 10 Jun, 2018 1 commit
  7. 24 Nov, 2015 1 commit
  8. 30 Oct, 2015 1 commit
  9. 03 Mar, 2015 1 commit
    • config: borrow refcounted references · 9a97f49e
      This changes the get_entry() method to return a refcounted version of
      the config entry, which you have to free when you're done.
      
      This allows us to avoid freeing the memory in which the entry is stored
      on a refresh, which may happen at any time for a live config.
      
      For this reason, get_string() has been forbidden on live configs and a
      new function get_string_buf() has been added, which stores the string in
      a git_buf which the user then owns.
      
      The functions which parse the string value takea advantage of the
      borrowing to parse safely and then release the entry.
      Carlos Martín Nieto committed
  10. 04 Oct, 2014 1 commit
  11. 09 Aug, 2014 1 commit
    • config: a multiline var can start immediately · 9dac1f95
      In the check for multiline, we traverse the backslashes from the end
      backwards and int the end assert that we haven't gone past the beginning
      of the line. We make sure of this in the loop condition, but we also
      check in the return value.
      
      However, for certain configurations, a line in a multiline variable
      might be empty to aid formatting. In that case, 'end' == 'start', since
      we ended up looking at the first char which made it a multiline.
      
      There is no need for the (end > start) check in the return, since the
      loop guarantees we won't go further back than the first char in the
      line, and we do accept the first char to be the final backslash.
      
      This fixes #2483.
      Carlos Martín Nieto committed
  12. 14 Nov, 2013 1 commit
  13. 05 Nov, 2013 1 commit
  14. 25 Jan, 2013 2 commits
  15. 23 Oct, 2012 1 commit
    • Add config level support in the config API · a1abe66a
      Added `struct git_config_entry`: a git_config_entry contains the key, the value, and the config file level from which a config element was found.
      Added `git_config_open_level`: build a single-level focused config object from a multi-level one.
      
      We are now storing `git_config_entry`s in the khash of the config_file
      yorah committed
  16. 21 Jul, 2012 2 commits
  17. 17 May, 2012 1 commit
  18. 16 Apr, 2012 1 commit
  19. 05 Mar, 2012 1 commit
  20. 25 Jan, 2012 1 commit
  21. 17 Jan, 2012 1 commit
    • Move path related functions from fileops to path · 1744fafe
      This takes all of the functions that look up simple data about
      paths (such as `git_futils_isdir`) and moves them over to path.h
      (becoming `git_path_isdir`).  This leaves fileops.h just with
      functions that actually manipulate the filesystem or look at
      the file contents in some way.
      
      As part of this, the dir.h header which is really just for win32
      support was moved into win32 (with some minor changes).
      Russell Belfer committed
  22. 26 Nov, 2011 1 commit
    • repository: Change ownership semantics · 9462c471
      The ownership semantics have been changed all over the library to be
      consistent. There are no more "borrowed" or duplicated references.
      
      Main changes:
      
      	- `git_repository_open2` and `3` have been dropped.
      
      	- Added setters and getters to hotswap all the repository owned
      	objects:
      
      		`git_repository_index`
      		`git_repository_set_index`
      		`git_repository_odb`
      		`git_repository_set_odb`
      		`git_repository_config`
      		`git_repository_set_config`
      		`git_repository_workdir`
      		`git_repository_set_workdir`
      
      	Now working directories/index files/ODBs and so on can be
      	hot-swapped after creating a repository and between operations.
      
      	- All these objects now have proper ownership semantics with
      	refcounting: they all require freeing after they are no longer
      	needed (the repository always keeps its internal reference).
      
      	- Repository open and initialization has been updated to keep in
      	mind the configuration files. Bare repositories are now always
      	detected, and a default config file is created on init.
      
      	- All the tests affected by these changes have been dropped from the
      	old test suite and ported to the new one.
      Vicent Marti committed
  23. 22 Nov, 2011 1 commit
  24. 18 Nov, 2011 1 commit