1. 11 Jul, 2019 10 commits
    • patch_parse: handle missing newline indicator in old file · 3f855fe8
      When either the old or new file contents have no newline at the end of
      the file, then git-diff(1) will print out a "\ No newline at end of
      file" indicator. While we do correctly handle this in the case where the
      new file has this indcator, we fail to parse patches where the old file
      is missing a newline at EOF.
      
      Fix this bug by handling and missing newline indicators in the old file.
      Add tests to verify that we can parse such files.
      Patrick Steinhardt committed
    • diff: ignore EOFNL for computing patch IDs · 001d76e1
      The patch ID is supposed to be mostly context-insignificant and
      thus only includes added or deleted lines. As such, we shouldn't honor
      end-of-file-without-newline markers in diffs.
      
      Ignore such lines to fix how we compute the patch ID for such diffs.
      Patrick Steinhardt committed
    • config_file: avoid re-reading files on write · 2ba7020f
      When we rewrite the configuration file due to any of its values
      being modified, we call `config_refresh` to update the in-memory
      representation of our config file backend. This is needlessly
      wasteful though, as `config_refresh` will always open the on-disk
      representation to reads the file contents while we already know
      the complete file contents at this point in time as we have just
      written it to disk.
      
      Implement a new function `config_refresh_from_buffer` that will
      refresh the backend's config entries from a buffer instead of
      from the config file itself. Note that this will thus _not_
      update the backend's timestamp, which will cause us to re-read
      the buffer when performing a read operation on it. But this is
      still an improvement as we now lazily re-read the contents, and
      most importantly we will avoid constantly re-reading the contents
      if we perform multiple write operations.
      
      The following strace demonstrates this if we're re-writing a key
      multiple times. It uses our config example with `config_set`
      changed to update the file 10 times with different keys:
      
      	$ strace lg2 config x.x z |& grep '^open.*config'
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      
      And now with the optimization of `config_refresh_from_buffer`:
      
      	$ strace lg2 config x.x z |& grep '^open.*config'
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      	open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
      
      As can be seen, this is quite a lot of `open` calls less.
      Patrick Steinhardt committed
    • config_file: split out function that sets config entries · a0dc3027
      Updating a config file backend's config entries is a bit more
      involved, as it requires clearing of the old config entries as
      well as handling locking correctly. As we will need this
      functionality in a future patch to refresh config entries from a
      buffer, let's extract this into its own function
      `config_set_entries`.
      Patrick Steinhardt committed
    • config_file: split out function that reads entries from a buffer · 985f5cdf
      The `config_read` function currently performs both reading the
      on-disk config file as well as parsing the retrieved buffer
      contents. To optimize how we refresh our config entries from an
      in-memory buffer, we need to be able to directly parse buffers,
      though, without involving any on-disk files at all.
      
      Extract a new function `config_read_buffer` that sets up the
      parsing logic and then parses config entries from a buffer, only.
      Have `config_read` use it to avoid duplicated logic.
      Patrick Steinhardt committed
    • config_file: move refresh into `write` function · 3e1c137a
      We are quite lazy in how we refresh our config file backend when
      updating any of its keys: instead of just updating our in-memory
      representation of the keys, we just discard the old set of keys
      and then re-read the config file contents from disk. This refresh
      currently happens separately at every callsite of `config_write`,
      but it is clear that we _always_ want to refresh if we have
      written the config file to disk. If we didn't, then we'd run
      around with an outdated config file backend that does not
      represent what we have on disk.
      
      By moving the refresh into `config_write`, we are also able to
      optimize the case where the config file is currently locked.
      Before, we would've tried to re-read the file even if we have
      only updated its cached contents without touching the on-disk
      file. Thus we'd have unnecessarily stat'd the file, even though
      we know that it shouldn't have been modified in the meantime due
      to its lock.
      Patrick Steinhardt committed
    • config_file: implement stat cache to avoid repeated rehashing · d7f58eab
      To decide whether a config file has changed, we always hash its
      complete contents. This is unnecessarily expensive, as
      well-behaved filesystems will always update stat information for
      files which have changed. So before computing the hash, we should
      first check whether the stat info has actually changed for either
      the configuration file or any of its includes. This avoids having
      to re-read the configuration file and its includes every time
      when we check whether it's been modified.
      
      Tracing the for-each-ref example previous to this commit, one can
      see that we repeatedly re-open both the repo configuration as
      well as the global configuration:
      
      	$ strace lg2 for-each-ref |& grep config
      	access("/home/pks/.gitconfig", F_OK)    = -1 ENOENT (No such file or directory)
      	access("/home/pks/.config/git/config", F_OK) = 0
      	access("/etc/gitconfig", F_OK)          = -1 ENOENT (No such file or directory)
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	access("/tmp/repo/.git/config", F_OK)   = 0
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/home/pks/.gitconfig", 0x7ffd15c05290) = -1 ENOENT (No such file or directory)
      	access("/home/pks/.gitconfig", F_OK)    = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	access("/home/pks/.config/git/config", F_OK) = 0
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/home/pks/.gitconfig", 0x7ffd15c051f0) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/home/pks/.gitconfig", 0x7ffd15c05090) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/home/pks/.gitconfig", 0x7ffd15c05090) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/home/pks/.gitconfig", 0x7ffd15c05090) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      
      With the change, we only do stats for those files and open them a
      single time, only:
      
      	$ strace lg2 for-each-ref |& grep config
      	access("/home/pks/.gitconfig", F_OK)    = -1 ENOENT (No such file or directory)
      	access("/home/pks/.config/git/config", F_OK) = 0
      	access("/etc/gitconfig", F_OK)          = -1 ENOENT (No such file or directory)
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	access("/tmp/repo/.git/config", F_OK)   = 0
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/home/pks/.gitconfig", 0x7ffe70540d20) = -1 ENOENT (No such file or directory)
      	access("/home/pks/.gitconfig", F_OK)    = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	access("/home/pks/.config/git/config", F_OK) = 0
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	stat("/home/pks/.gitconfig", 0x7ffe70540ca0) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.gitconfig", 0x7ffe70540c80) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	stat("/home/pks/.gitconfig", 0x7ffe70540b40) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.gitconfig", 0x7ffe70540b20) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	stat("/home/pks/.gitconfig", 0x7ffe70540b40) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.gitconfig", 0x7ffe70540b20) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      	stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
      	stat("/home/pks/.gitconfig", 0x7ffe70540b40) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.gitconfig", 0x7ffe70540b20) = -1 ENOENT (No such file or directory)
      	stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
      
      The following benchmark has been performed with and without the
      stat cache in a best-of-ten run:
      
      ```
      
      int lg2_repro(git_repository *repo, int argc, char **argv)
      {
      	git_config *cfg;
      	int32_t dummy;
      	int i;
      
      	UNUSED(argc);
      	UNUSED(argv);
      
      	check_lg2(git_repository_config(&cfg, repo),
      			"Could not obtain config", NULL);
      
      	for (i = 1; i < 100000; ++i)
      		git_config_get_int32(&dummy, cfg, "foo.bar");
      
      	git_config_free(cfg);
      	return 0;
      }
      ```
      
      Without stat cache:
      
      	$ time lg2 repro
      	real    0m1.528s
      	user    0m0.568s
      	sys     0m0.944s
      
      With stat cache:
      
      	$ time lg2 repro
      	real    0m0.526s
      	user    0m0.268s
      	sys     0m0.258s
      
      This benchmark shows a nearly three-fold performance improvement.
      
      This change requires that we check our configuration stress tests
      as we're now in fact becoming more racy. If somebody is writing a
      configuration file at nearly the same time (there is a window of
      100ns on Windows-based systems), then it might be that we realize
      that this file has actually changed and thus may not re-read it.
      This will only happen if either an external process is rewriting
      the configuration file or if the same process has multiple
      `git_config` structures pointing to the same time, where one of
      both is being used to write and the other one is used to read
      values.
      Patrick Steinhardt committed
  2. 05 Jul, 2019 1 commit
    • w32_stack: convert buffer length param to `size_t` · 2f14c4fc
      In both `git_win32__stack_format` and `git_win32__stack`, we handle
      buffer lengths via an integer variable. As we only ever pass buffer
      sizes to it, this should be a `size_t` though to avoid loss of
      precision. As we also use it to compare with other `size_t` variables,
      this also silences signed/unsigned comparison warnings.
      Patrick Steinhardt committed
  3. 04 Jul, 2019 2 commits
    • attr_file: completely initialize attribute sessions · 1bbec26d
      The function `git_attr_session__init` is currently only initializing
      setting up the attribute's session key by incrementing the repo-global
      key by one. Most notably, all other members of the `git_attr_session`
      struct are not getting initialized at all. So if one is to allocate a
      session on the stack and then calls `git_attr_session__init`, the
      session will still not be fully initialized. We have fared just fine
      with that until now as all users of the function have allocated the
      session structure as part of bigger structs with `calloc`, and thus its
      contents have been zero-initialized implicitly already.
      
      Fix this by explicitly zeroing out the session to enable allocation of
      sessions on the stack.
      Patrick Steinhardt committed
    • attr: Don't fail in attr_setup if there exists a system attributes file · 18a6d9f3
      Regression introduced in commit 5452e49f on PR #4967.
      
      Signed-off-by: Sven Strickroth <email@cs-ware.de>
      Sven Strickroth committed
  4. 27 Jun, 2019 1 commit
    • hash: fix missing error return on production builds · 7fd3f32b
      When no hash algorithm has been initialized in a given hash context,
      then we will simply `assert` and not return a value at all. This works
      just fine in debug builds, but on non-debug builds the assert will be
      converted to a no-op and thus we do not have a proper return value.
      
      Fix this by returning an error code in addition to the asserts.
      Patrick Steinhardt committed
  5. 26 Jun, 2019 5 commits
  6. 25 Jun, 2019 1 commit
  7. 24 Jun, 2019 20 commits