1. 23 Jun, 2017 3 commits
    • generate.py: generate clar cache in binary directory · 8d22bcea
      The source directory should usually not be touched when using
      out-of-tree builds. But next to the previously fixed "clar.suite" file, we
      are also writing the ".clarcache" into the project's source tree,
      breaking the builds. Fix this by also honoring the output directory for
      the ".clarcache" file.
      Patrick Steinhardt committed
    • generate.py: enable overriding path of generated clar.suite · 9e240bd2
      The generate.py script will currently always write the generated
      clar.suite file into the scanned directory, breaking out-of-tree builds
      with read-only source directories. Fix this issue by adding another
      option to allow overriding the output path of the generated file.
      Patrick Steinhardt committed
    • generate.py: disallow generating test suites for multiple paths · 0a513a94
      Our generate.py script is used to extract and write test suite
      declarations into the clar.suite file. As is, the script accepts
      multiple directories on the command line and will generate this file for
      each of these directories.
      
      The generate.py script will always write the clar.suite file into the
      directory which is about to be scanned. This actually breaks
      out-of-tree builds with libgit2, as the file will be generated in the
      source tree instead of in the build tree. This is noticed especially in
      the case where the source tree is mounted read-only, rendering us unable
      to build unit tests.
      
      Due to us accepting multiple paths which are to be scanned, it is not
      trivial to fix though. The first solution which comes into mind would be
      to re-create the directory hierarchy at a given output path or use
      unique names for the clar.suite files, but this is rather cumbersome and
      magical. The second and cleaner solution would be to fold all
      directories into a single clar.suite file, but this would probably break
      some use-cases.
      
      Seeing that we do not ever pass multiple directories to generate.py, we
      will now simply retire support for this. This allows us to later on
      introduce an additional option to specify the path where the clar.suite
      file will be generated at, defaulting to "clar.suite" inside of the
      scanned directory.
      Patrick Steinhardt committed
  2. 21 Jun, 2017 1 commit
  3. 17 Jun, 2017 1 commit
  4. 13 Jun, 2017 7 commits
  5. 12 Jun, 2017 5 commits
    • remote: ensure we can create an anon remote on inmemory repo · fe9a5dd3
      Given a wholly in-memory repository, ensure that we can create an
      anonymous remote and perform actions on it.
      Edward Thomson committed
    • tests: odb: add tests with multiple backends · f148258a
      Previous to pulling out and extending the fake backend, it was quite
      cumbersome to write tests for very specific scenarios regarding
      backends. But as we have made it more generic, it has become much easier
      to do so. As such, this commit adds multiple tests for scenarios with
      multiple backends for the ODB.
      
      The changes also include a test for a very targeted scenario. When one
      backend found a matching object via `read_prefix`, but the last backend
      returns `GIT_ENOTFOUND` and when object hash verification is turned off,
      we fail to reset the error code to `GIT_OK`. This causes us to segfault
      later on, when doing a double-free on the returned object.
      Patrick Steinhardt committed
    • tests: odb: allow passing fake objects to the fake backend · 6e010bb1
      Right now, the fake backend is quite restrained in the way how it
      works: we pass it an OID which it is to return later as well as an error
      code we want it to return. While this is sufficient for existing tests,
      we can make the fake backend a little bit more generic in order to allow
      us testing for additional scenarios.
      
      To do so, we change the backend to not accept an error code and OID
      which it is to return for queries, but instead a simple array of OIDs
      with their respective blob contents. On each query, the fake backend
      simply iterates through this array and returns the first matching
      object.
      Patrick Steinhardt committed
    • tests: do not reuse OID from backend · 369cb45f
      In order to make the fake backend more useful, we want to enable it
      holding multiple object references. To do so, we need to decouple it
      from the single fake OID it currently holds, which we simply move up
      into the calling tests.
      Patrick Steinhardt committed
    • tests: odb: move fake backend into its own file · 2add34d0
      The fake backend used by the test suite `odb::backend::nonrefreshing` is
      useful to have some low-level tests for the ODB layer. As such, we move
      the implementation into its own `backend_helpers` module.
      Patrick Steinhardt committed
  6. 10 Jun, 2017 1 commit
  7. 08 Jun, 2017 1 commit
    • settings: rename `GIT_OPT_ENABLE_SYNCHRONOUS_OBJECT_CREATION` · 6c23704d
      Initially, the setting has been solely used to enable the use of
      `fsync()` when creating objects. Since then, the use has been extended
      to also cover references and index files. As the option is not yet part
      of any release, we can still correct this by renaming the option to
      something more sensible, indicating not only correlation to objects.
      
      This commit renames the option to `GIT_OPT_ENABLE_FSYNC_GITDIR`. We also
      move the variable from the object to repository source code.
      Patrick Steinhardt committed
  8. 06 Jun, 2017 5 commits
    • tests: index::version: improve write test for index v4 · 92d3ea4e
      The current write test does not trigger some edge-cases in the index
      version 4 path compression code. Rewrite the test to start off the an
      empty standard repository, creating index entries with interesting paths
      itself. This allows for more fine-grained control over checked paths.
      Furthermore, we now also verify that entry paths are actually
      reconstructed correctly.
      Patrick Steinhardt committed
    • tests: index::version: verify we write compressed index entries · 8fe33538
      While we do have a test which checks whether a written index of version
      4 has the correct version set, we do not check whether this actually
      enables path compression for index entries. This commit adds a new test
      by adding a number of index entries with equal path prefixes to the
      index and subsequently flushing that to disk. With suffix compression
      enabled by index version 4, only the last few bytes of these paths will
      actually have to be written to the index, saving a lot of disk space.
      For the test, differences are about an order of magnitude, allowing us
      to easily verify without taking a deeper look at actual on-disk
      contents.
      Patrick Steinhardt committed
    • tests: index::version: add test to read index version v4 · 82368b1b
      While we have a simple test to determine whether we can write an index
      of version 4, we never verified that we are able to read this kind of
      index (and in fact, we were not able to do so). Add a new repository
      which has an index of version 4. This repository is then read from a new
      test.
      Patrick Steinhardt committed
    • tests: index::version: move up cleanup function · fea0c81e
      The init and cleanup functions for test suites are usually prepended to
      our actual tests. The index::version test suite does not adhere to this
      stile. Fix this.
      Patrick Steinhardt committed
    • varint: fix computation for remaining buffer space · 8a5e7aae
      When encoding varints to a buffer, we want to remain sure that the
      required buffer space does not exceed what is actually available. Our
      current check does not do the right thing, though, in that it does not
      honor that our `pos` variable counts the position down instead of up. As
      such, we will require too much memory for small varints and not enough
      memory for big varints.
      
      Fix the issue by correctly calculating the required size as
      `(sizeof(varint) - pos)`. Add a test which failed before.
      Patrick Steinhardt committed
  9. 19 May, 2017 2 commits
    • indexer: name pack files after trailer hash · c0e54155
      Upstream git.git has changed the way how packfiles are named.
      Previously, they were using a hash of the contained object's OIDs, which
      has then been changed to use the hash of the complete packfile instead.
      See 1190a1acf (pack-objects: name pack files after trailer hash,
      2013-12-05) in the git.git repository for more information on this
      change.
      
      This commit changes our logic to match the behavior of core git.
      Chris Hescock committed
    • repository: make check if repo is a worktree more strict · 2696c5c3
      To determine if a repository is a worktree or not, we currently check
      for the existence of a "gitdir" file inside of the repository's gitdir.
      While this is sufficient for non-broken repositories, we have at least
      one case of a subtly broken repository where there exists a gitdir file
      inside of a gitmodule. This will cause us to misidentify the submodule
      as a worktree.
      
      While this is not really a fault of ours, we can do better here by
      observing that a repository can only ever be a worktree iff its common
      directory and dotgit directory are different. This allows us to make our
      check whether a repo is a worktree or not more strict by doing a simple
      string comparison of these two directories. This will also allow us to
      do the right thing in the above case of a broken repository, as for
      submodules these directories will be the same. At the same time, this
      allows us to skip the `stat` check for the "gitdir" file for most
      repositories.
      Patrick Steinhardt committed
  10. 17 May, 2017 2 commits
  11. 15 May, 2017 2 commits
  12. 05 May, 2017 3 commits
    • revparse: support open-ended ranges · 8b107dc5
      Support '..' and '...' ranges where one side is not specified.
      The unspecified side defaults to HEAD.
      
      Closes #4223
      William Bain committed
    • worktree: switch over worktree pruning to an opts structure · 883eeb5f
      The current signature of `git_worktree_prune` accepts a flags field to
      alter its behavior. This is not as flexible as we'd like it to be when
      we want to enable passing additional options in the future. As the
      function has not been part of any release yet, we are still free to
      alter its current signature. This commit does so by using our usual
      pattern of an options structure, which is easily extendable without
      breaking the API.
      Patrick Steinhardt committed
    • worktree: support creating locked worktrees · 8264a30f
      When creating a new worktree, we do have a potential race with us
      creating the worktree and another process trying to delete the same
      worktree as it is being created. As such, the upstream git project has
      introduced a flag `git worktree add --locked`, which will cause the
      newly created worktree to be locked immediately after its creation. This
      mitigates the race condition.
      
      We want to be able to mirror the same behavior. As such, a new flag
      `locked` is added to the options structure of `git_worktree_add` which
      allows the user to enable this behavior.
      Patrick Steinhardt committed
  13. 03 May, 2017 1 commit
    • tests: repo: fix repo discovery tests on overlayfs · ffd264d9
      Debian and Ubuntu often use schroot to build their DEB packages in a
      controlled environment. Depending on how schroot is configured, our
      tests regarding repository discovery break due to not being able to find
      the repositories anymore. It turns out that these errors occur when the
      schroot is configured to use an overlayfs on the directory structures.
      
      The reason for this failure is that we usually refrain from discovering
      repositories across devices. But unfortunately, overlayfs does not have
      consistent device identifiers for all its files but will instead use the
      device number of the filesystem the file stems from. So whenever we
      cross boundaries between the upper and lower layer of the overlay, we
      will fail to properly detect the repository and bail out.
      
      This commit fixes the issue by enabling cross-device discovery in our
      tests. While it would be preferable to have this turned off, it probably
      won't do much harm anyway as we set up our tests in a temporary location
      outside of the parent repository.
      Patrick Steinhardt committed
  14. 02 May, 2017 1 commit
    • worktree: introduce git_worktree_add options · a7aa73a5
      The `git_worktree_add` function currently accepts only a path and name
      for the new work tree. As we may want to expand these parameters in
      future versions without adding additional parameters to the function for
      every option, this commit introduces our typical pattern of an options
      struct. Right now, this structure is still empty, which will change with
      the next commit.
      Patrick Steinhardt committed
  15. 01 May, 2017 1 commit
  16. 28 Apr, 2017 4 commits
    • odb: add option to turn off hash verification · 35079f50
      Verifying hashsums of objects we are reading from the ODB may be costly
      as we have to perform an additional hashsum calculation on the object.
      Especially when reading large objects, the penalty can be as high as
      35%, as can be seen when executing the equivalent of `git cat-file` with
      and without verification enabled. To mitigate for this, we add a global
      option for libgit2 which enables the developer to turn off the
      verification, e.g. when he can be reasonably sure that the objects on
      disk won't be corrupted.
      Patrick Steinhardt committed
    • odb: verify object hashes · 28a0741f
      The upstream git.git project verifies objects when looking them up from
      disk. This avoids scenarios where objects have somehow become corrupt on
      disk, e.g. due to hardware failures or bit flips. While our mantra is
      usually to follow upstream behavior, we do not do so in this case, as we
      never check hashes of objects we have just read from disk.
      
      To fix this, we create a new error class `GIT_EMISMATCH` which denotes
      that we have looked up an object with a hashsum mismatch. `odb_read_1`
      will then, after having read the object from its backend, hash the
      object and compare the resulting hash to the expected hash. If hashes do
      not match, it will return an error.
      
      This obviously introduces another computation of checksums and could
      potentially impact performance. Note though that we usually perform I/O
      operations directly before doing this computation, and as such the
      actual overhead should be drowned out by I/O. Running our test suite
      seems to confirm this guess. On a Linux system with best-of-five
      timings, we had 21.592s with the check enabled and 21.590s with the
      ckeck disabled. Note though that our test suite mostly contains very
      small blobs only. It is expected that repositories with bigger blobs may
      notice an increased hit by this check.
      
      In addition to a new test, we also had to change the
      odb::backend::nonrefreshing test suite, which now triggers a hashsum
      mismatch when looking up the commit "deadbeef...". This is expected, as
      the fake backend allocated inside of the test will return an empty
      object for the OID "deadbeef...", which will obviously not hash back to
      "deadbeef..." again. We can simply adjust the hash to equal the hash of
      the empty object here to fix this test.
      Patrick Steinhardt committed
    • tests: object: test looking up corrupted objects · d59dabe5
      We currently have no tests which check whether we fail reading corrupted
      objects. Add one which modifies contents of an object stored on disk and
      then tries to read the object.
      Patrick Steinhardt committed
    • tests: object: create sandbox · 86c03552
      The object::lookup tests do use the "testrepo.git" repository in a
      read-only way, so we do not set up the repository as a sandbox but
      simply open it. But in a future commit, we will want to test looking up
      objects which are corrupted in some way, which requires us to modify the
      on-disk data. Doing this in a repository without creating the sandbox
      will modify contents of our libgit2 repository, though.
      
      Create the repository in a sandbox to avoid this.
      Patrick Steinhardt committed