- 22 Jun, 2018 11 commits
-
-
We strive to keep an options structure to many functions to be able to extend options in the future without breaking the API. `git_indexer_new` doesn't have one right now, but we want to be able to add an option for enabling strict packfile verification. Add a new `git_indexer_options` structure and adjust callers to use that.
Patrick Steinhardt committed -
When passing `--strict` to `git-unpack-objects`, core git will verify the pack file that is currently being read. In addition to the typical checksum verification, this will especially cause it to verify object connectivity of the received pack file. So it checks, for every received object, if all the objects it references are either part of the local object database or part of the pack file. In libgit2, we currently have no such mechanism, which leaves us unable to verify received pack files prior to writing them into our local object database. This commit introduce the concept of `expected_oids` to the indexer. When pack file verification is turned on by a new flag, the indexer will try to parse each received object first. If the object has any links to other objects, it will check if those links are already satisfied by known objects either part of the object database or objects it has already seen as part of that pack file. If not, it will add them to the list of `expected_oids`. Furthermore, the indexer will remove the current object from the `expected_oids` if it is currently being expected. Like this, we are able to verify whether all object links are being satisfied. As soon as we hit the end of the object stream and have resolved all objects as well as deltified objects, we assert that `expected_oids` is in fact empty. This should always be the case for a valid pack file with full connectivity.
Patrick Steinhardt committed -
The loop inside of `git_indexer_append` iterates over every object that is to be stored as part of the index. While the logic to retrieve every object from the packfile stream is rather involved, it currently just part of the loop, making it unnecessarily hard to follow. Move the logic into its own function `read_stream_object`, which unpacks a single object from the stream. Note that there is some subtletly here involving the special error `GIT_EBUFS`, which indicates to the indexer that no more data is currently available. So instead of returning an error and aborting the whole loop in that case, we do have to catch that value and return successfully to wait for more data to be read.
Patrick Steinhardt committed -
The `processed` variable local to `git_indexer_append` counts how many objects have already been processed. But actually, whenever it gets assigned to, we are also assigning the same value to the `stats->indexed_objects` struct member. So in fact, it is being quite useless due to always having the same value as the `indexer_objects` member and makes it a bit harder to understand the code. We can just remove the variable to fix that.
Patrick Steinhardt committed -
Now that we have implement functions to parse all git objects from raw data, we can implement a generic function `git_object__from_raw` to create a structure of type `git_object`. This allows us to parse and interpret objects from raw data without having to touch the ODB at all, which is especially useful for object verification prior to accepting them into the repository.
Patrick Steinhardt committed -
Currently, parsing objects is strictly tied to having an ODB object available. This makes it hard to parse an object when all that is available is its raw object and size. Furthermore, hacking around that limitation by directly creating an ODB structure either on stack or on heap does not really work that well due to ODB objects being reference counted and then automatically free'd when reaching a reference count of zero. Implement a function `git_tree__parse_raw` to parse a tree object from a pair of `data` and `size`.
Patrick Steinhardt committed -
Currently, parsing objects is strictly tied to having an ODB object available. This makes it hard to parse an object when all that is available is its raw object and size. Furthermore, hacking around that limitation by directly creating an ODB structure either on stack or on heap does not really work that well due to ODB objects being reference counted and then automatically free'd when reaching a reference count of zero. Implement a function `git_tag__parse_raw` to parse a tag object from a pair of `data` and `size`.
Patrick Steinhardt committed -
Currently, parsing objects is strictly tied to having an ODB object available. This makes it hard to parse an object when all that is available is its raw object and size. Furthermore, hacking around that limitation by directly creating an ODB structure either on stack or on heap does not really work that well due to ODB objects being reference counted and then automatically free'd when reaching a reference count of zero. Implement a function `git_commit__parse_raw` to parse a commit object from a pair of `data` and `size`.
Patrick Steinhardt committed -
Currently, parsing objects is strictly tied to having an ODB object available. This makes it hard to parse an object when all that is available is its raw object and size. Furthermore, hacking around that limitation by directly creating an ODB structure either on stack or on heap does not really work that well due to ODB objects being reference counted and then automatically free'd when reaching a reference count of zero. In some occasions parsing raw objects without touching the ODB is actually recuired, though. One use case is for example object verification, where we want to assure that an object is valid before inserting it into the ODB or writing it into the git repository. Asa first step towards that, introduce a distinction between raw and ODB objects for blobs. Creation of ODB objects stays the same by simply using `git_blob__parse`, but a new function `git_blob__parse_raw` has been added that creates a blob from a pair of data and size. By setting a new flag inside of the blob, we can now distinguish whether it is a raw or ODB object now and treat it accordingly in several places. Note that the blob data passed in is not being copied. Because of that, callers need to make sure to keep it alive during the blob's life time. This is being used to avoid unnecessarily increasing the memory footprint when parsing largish blobs.
Patrick Steinhardt committed -
Going forward, we will have to change how blob sizes are calculated based on whether the blob is a cahed object part of the ODB or not. In order to not have to distinguish between those two object types repeatedly when accessing the blob's data or size, encapsulate all existing direct uses of those fields by instead using `git_blob_rawcontent` and `git_blob_rawsize`.
Patrick Steinhardt committed -
The `git_walk_objects` structure is currently only being used inside of the pack-objects.c file, but being declared in its header. This has actually been the case since its inception in 04a36fef (pack-objects: fill a packbuilder from a walk, 2014-10-11) and has never really changed. Move the struct declaration into pack-objects.c to improve code encapsulation.
Patrick Steinhardt committed
-
- 18 Jun, 2018 5 commits
-
-
Fix last references to deprecated git_buf_free
Edward Thomson committed -
revwalk: avoid walking the entire history when output is unsorted
Edward Thomson committed -
Signed-off-by: Sven Strickroth <email@cs-ware.de>
Sven Strickroth committed -
Edward Thomson committed
-
Add mailmap support.
Edward Thomson committed
-
- 17 Jun, 2018 1 commit
-
-
Nika Layzell committed
-
- 16 Jun, 2018 1 commit
-
-
tree: remove unused functions
Edward Thomson committed
-
- 15 Jun, 2018 22 commits
-
-
Patrick Steinhardt committed
-
Patrick Steinhardt committed
-
Link `mbedTLS` libraries in when `SHA1_BACKEND` == "mbedTLS"
Patrick Steinhardt committed -
editorconfig: allow trailing whitespace in markdown
Patrick Steinhardt committed -
Nika Layzell committed
-
Nika Layzell committed
-
Nika Layzell committed
-
Nika Layzell committed
-
Nika Layzell committed
-
Nika Layzell committed
-
Nika Layzell committed
-
Nika Layzell committed
-
Nika Layzell committed
-
Emilio Cobos Álvarez committed
-
Nika Layzell committed
-
Nika Layzell committed
-
Nika Layzell committed
-
Nika Layzell committed
-
Nika Layzell committed
-
Emilio Cobos Álvarez committed
-
Also matches git.
Emilio Cobos Álvarez committed -
Emilio Cobos Álvarez committed
-