libfossil
pages.h
Go to the documentation of this file.
1/* -*- Mode: C; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
2/* vim: set ts=2 et sw=2 tw=80: */
3#if !defined(ORG_FOSSIL_SCM_PAGES_H_INCLUDED)
4#define ORG_FOSSIL_SCM_PAGES_H_INCLUDED
5/*
6 Copyright 2013-2021 The Libfossil Authors, see LICENSES/BSD-2-Clause.txt
7
8 SPDX-License-Identifier: BSD-2-Clause-FreeBSD
9 SPDX-FileCopyrightText: 2021 The Libfossil Authors
10 SPDX-FileType: Code
11
12 Heavily indebted to the Fossil SCM project (https://fossil-scm.org).
13
14 *****************************************************************************
15 This file contains only Doxygen-format documentation, split up into
16 Doxygen "pages", each covering some topic at a high level. This is
17 not the place for general code examples - those belong with their
18 APIs.
19*/
20
21/** @mainpage libfossil
22
23 Forewarning: this API assumes one is familiar with the Fossil SCM,
24 ideally in detail. The Fossil SCM can be found at:
25
26 https://fossil-scm.org
27
28 libfossil is an experimental/prototype library API for the Fossil
29 SCM. This API concerns itself only with the components of fossil
30 which do not need user interaction or the display of UI components
31 (including HTML and CLI output). It is intended only to model the
32 core internals of fossil, off of which user-level applications
33 could be built.
34
35 The project's repository and additional information can be found at:
36
37 https://fossil.wanderinghorse.net/r/libfossil/
38
39 This code is 100% hypothetical/potential, and does not represent
40 any Official effort of the Fossil project. It is up for any amount
41 of change at any time and does not yet have a stable API.
42
43 All Fossil users are encouraged to participate in its development,
44 but if you are reading this then you probably already knew that
45 :).
46
47 This effort does not represent "Fossil Version 2", but provides an
48 alternate method of accessing and manipulating fossil(1)
49 repositories. Whereas fossil(1) is a monolithic binary, this API
50 provides library-level access to (some level of) the fossil(1)
51 feature set (that level of support grows approximately linearly
52 with each new commit).
53
54 Current status: alpha. Some bits are basically finished but there
55 is a lot of work left to do. The scope is pretty much all
56 Fossil-related functionality which does not require a user
57 interface or direct user interaction, plus some range of utilities
58 to support those which require a UI/user.
59*/
60
61/** @page page_terminology Fossil Terminology
62
63 See also: https://fossil-scm.org/home/doc/trunk/www/concepts.wiki
64
65 The libfossil API docs normally assume one is familiar with
66 Fossil-internal terminology, which is of course a silly assumption
67 to make. Indeed, one of libfossil's goals is to make Fossil more
68 accessible, partly be demystifying it. To that end, here is a
69 collection of terms one may come across in the API, along with
70 their meanings in the context of Fossil...
71
72
73 - REPOSITORY (a.k.a. "repo) is an sqlite database file which
74 contains all content for a given "source tree." (We will use the
75 term "source tree" to mean any tree of "source" (documents,
76 whatever) a client has put under Fossil's supervision.)
77
78 - CHECKOUT (a.k.a. "local source tree" or "working copy") refers
79 to (A) the action of pulling a specific version of a repository's
80 state from that repo into the local filesystem, and (B) a local
81 copy "checked out" of a repo. e.g. "he checked out the repo," and
82 "the changes are in his [local] checkout."
83
84 - ARTIFACT is the generic term for anything stored in a repo. More
85 specifically, ARTIFACT refers to "control structures" Fossil uses
86 to internally track changes. These artifacts are stored as blobs
87 in the database, just like any other content. For complete details
88 and examples, see:
89 https://fossil-scm.org/home/doc/tip/www/fileformat.wiki
90
91 - A MANIFEST is a specific type of ARTIFACT - the type which
92 records all metadata for a COMMIT operation (which files, which
93 user, the timestamp, checkin comment, lineage, etc.). For
94 historical reasons, MANIFEST is sometimes used as a generic term
95 for ARTIFACT because what the fossil(1)-internal APIs originally
96 called a Manifest eventually grew into other types of artifacts
97 but kept the Manifest naming convention. In Fossil developer
98 discussion, "manifest" most often means what this page calls
99 ARTIFACT (probably because that how the C code is modelled). The
100 libfossil API calls uses the term "deck" instead of "manifest" to
101 avoid ambiguity/confusion (or to move the confusion somewhere
102 else, at least).
103
104 - CHECKIN is the term libfossil prefers to use for COMMIT
105 MANIFESTS. It is also the action of "checking in"
106 (a.k.a. "committing") file changes to a repository. A CHECKIN
107 ARTIFACT can be one of two types: a BASELINE MANIFEST (or BASELINE
108 CHECKIN) contains a list of all files in that version of the
109 repository, including their file permissions and the UUIDs of
110 their content. A DELTA MANFIEST is a checkin record which derives
111 from a BASELINE MANIFEST and it lists only the file-level changes
112 which happened between the baseline and the delta, recording any
113 changes in content, permisions, or name, and recording
114 deletions. Note that this inheritance of deltas from baselines is
115 an internal optimization which has nothing to do with checkin
116 version inheritance - the baseline of any given delta is normally
117 _not_ its direct checkin version parent.
118
119 - BRANCH, FORK, and TAG are all closely related in Fossil and are
120 explained in detail (with pictures!) at:
121 https://fossil-scm.org/home/doc/trunk/www/concepts.wiki
122 In short: BRANCHes and FORKs are two names for the same thing, and
123 both are just a special-case usage of TAGs.
124
125 - MERGE or MERGING: the process of integrating one version of
126 source code into another version of that source code, using a
127 common parent version as the basis for comparison. This is
128 normally fully automated, but occasionally human (and sometimes
129 Divine) intervention is required to resolve so-called "merge
130 conflicts," where two versions of a file change the same parts of
131 a common parent version.
132
133 - RID (Record ID) is a reference to the blob.rid field in a
134 repository DB. RIDs are used extensively throughout the API for
135 referencing content records, but they are transient values local
136 to a given copy of a given repository at a given point in
137 time. They _can_ change, even for the same content, (e.g. a
138 rebuild can hypothetically change them, though it might not, and
139 re-cloning a repo may very well change some RIDs). Clients must
140 never rely on them for long-term reference to SCM'd data - always use
141 the full UUID of such data. Even though they normally appear to be
142 static, they are most explicitly NOT guaranteed to be. Nor are
143 their values guaranteed to imply any meaning, e.g. "higher is
144 newer" is not necessarily true because synchronization can import
145 new remote content in an arbitrary order and a rebuild might
146 import it in random order. The API uses RIDs basically as handles
147 to arbitrary blob content and, like most C-side handles, must be
148 considered transient in nature. That said, within the db, records
149 are linked to each other exclusively using RIDs, so they do have
150 some persistence guarantees for a given db instance.
151*/
152
153
154/** @page page_APIs High-level API Overview
155
156 The primary end goals of this project are to eventually cover the
157 following feature areas:
158
159 - Provide embeddable SCM to local apps using sqlite storage.
160 - Provide a network layer on top of that for synchronization.
161 - Provide apps on top of those to allow administration of repos.
162
163 To those ends, the fossil APIs cover the following categories of
164 features:
165
166 Filesystem:
167
168 - Conversions of strings from OS-native encodings to UTF.
169 fsl_utf8_to_unicode(), fsl_filename_to_utf8(), etc. These are
170 primarily used internally but may also be useful for applications
171 working with files (as most clients will). Actually... most of
172 these bits are only needed for portability across Windows
173 platforms.
174
175 - Locating a user's home directory: fsl_find_home_dir()
176
177 - Normalizing filenames/paths. fsl_file_canonical_name() and friends.
178
179 - Checking for existence, size, and type (file vs directory) with
180 fsl_is_file() and fsl_dir_check(), or the more general-purpose
181 fsl_stat().
182
183
184 Databases (sqlite):
185
186 - Opening/closing sqlite databases and running queries on them,
187 independent of version control features. See fsl_db_open() and
188 friends. The actual sqlite-level DB handle type is abstracted out
189 of the public API, largely to simplify an eventual port from
190 sqlite3 to sqlite4 or (hypothetically) to other storage back-ends
191 (not gonna happen - too much work).
192
193 - There are lots of utility functions for oft-used operations,
194 e.g. fsl_config_get_int32() and friends to fetch settings from one
195 of several different configuration areas (global, repository,
196 checkout, and "versionable" settings).
197
198 - Pseudo-recusive transactions: fsl_db_txn_begin() and
199 fsl_db_txn_end(). sqlite does not support truly nested
200 transactions, but they can be simulated quite effectively so long
201 as certain conventions are adhered to.
202
203 - Cached statements (an optimization for oft-used queries):
204 fsl_db_prepare_cached() and friends.
205
206
207 The DB API is (as Brad Harder put so well) "very present" in the
208 public API. While the core API provides access to the underlying
209 repository data, it cannot begin to cover even a small portion of
210 potential use cases. To that end, it exposes the DB API so that
211 clients who want to custruct their own data can do so. It does
212 require research into the underlying schemas, but gives
213 applications the ability to do _anything_ with their repositories
214 which the core API does not account for. Historically, the ability
215 to create ad-hoc data structures as needed, in the form of SQL
216 queries, has accounted for much of Fossil's feature flexibility.
217
218
219 Deltas:
220
221 - Creation and application of raw deltas, using Fossil's delta
222 format, independent of version control features. See
223 fsl_delta_create() and friends. These are normally used only at
224 the deepest internal levels of fossil, but the APIs are exposed so
225 that clients can, if they wish, use them to deltify their own
226 content independently of fossil's internally-applied
227 deltification. Doing so is remarkably easy, but completely
228 unnecessary for content which will be stored in a repo, as Fossil
229 creates deltas as needed.
230
231
232 SCM:
233
234 - A "context" type (fsl_cx) which manages a repository db and,
235 optionally, a checkout db. Read-only operations on the DB are
236 working and write functionality (adding repo content) is
237 ongoing. See fsl_cx, fsl_cx_init(), and friends.
238
239 - The fsl_deck class assists in parsing, creating, and outputing
240 "artifacts" (manifests, control (tags), events, etc.). It gets its
241 name from it being container for "a collection of cards" (which is
242 what a Fossil artifact is).
243
244 - fsl_content_get() expands a (possibly) deltified blob into its
245 full form, and fsl_content_blob() can be used to fetch a raw blob
246 (possibly a raw delta).
247
248 - A number of routines exist for converting symbol names to RIDs
249 (fsl_sym_to_rid()), UUIDs to RIDs (fsl_uuid_to_rid(),
250 and similar commonly-needed lookups.
251
252
253 Input/Output:
254
255 - The API defines several abstractions for i/o interfaces, e.g.
256 fsl_input_f() and fsl_output_f(), which allow us to accept/emit
257 data from/to arbitrary streamable (as opposed to random-access)
258 sources/destinations. A fsl_cx instance is configured with an
259 output channel, the intention being that all clients of that
260 context should generate any output through that channel, so that
261 all compatible apps can cooperate more easily in terms of i/o. For
262 example, the s2 script binding for libfossil routes fsl_output()
263 through the script engine's i/o channels, so that any output
264 generated by libfossil-using code it links to can take advantage
265 of the script-side output features (such as output buffering,
266 which is needed for any non-trivial CGI output). That said: the
267 library-level code does not actually generate output to that
268 channel, but higher-level code like fcli does, and clients are
269 encouraged to in order to enable their app's output to be
270 redirected to an arbitrary UI element, be it a console or UI
271 widget.
272
273
274 Utilities:
275
276 - fsl_buffer, a generic buffer class, is used heavily by the
277 library. See fsl_buffer and friends.
278
279 - fsl_appendf() provides printf()-like functionality, but sends
280 its output to a callback function (optionally stateful), making it
281 the one-stop-shop for string formatting within the library.
282
283 - The fsl_error class is used to propagate error information
284 between the libraries various levels and the client.
285
286 - The fsl_list class acts as a generic container-of-pointers, and
287 the API provides several convenience routines for managing them,
288 traversing them, and cleaning them up.
289
290 - Hashing: there are a number of routines for calculating SHA1,
291 SHA3, and MD5 hashes. See fsl_sha1_cx, fsl_sha3_cx, fsl_md5_cx,
292 and friends.
293
294 - zlib compression is used for storing artifacts. See
295 fsl_data_is_compressed(), fsl_buffer_compress(), and friends.
296 These are never needed at the client level, but are exposed "just
297 in case" a given client should want them.
298*/
299
300/** @page page_is_isnot Fossil is/is not...
301
302 Through porting the main fossil application into library form,
303 the following things have become very clear (or been reinforced)...
304
305 Fossil is...
306
307 - _Exceedingly_ robust. Not only is sqlite literally the single
308 most robust application-agnostic container file format on the
309 planet, but Fossil goes way out of its way to ensure that what
310 gets put in is what gets pulled out. It cuts zero corners on data
311 integrity, even adding in checks which seem superfluous but
312 provide another layer of data integrity (i'm primarily talking
313 about the R-card here, but there are other validation checks). It
314 does this at the cost of memory and performance (that said, it's
315 still easily fast enough for its intended uses). "Robust" doesn't
316 mean that it never crashes nor fails, but that it does so with
317 (insofar as is technically possible) essentially zero chance of
318 data loss/corruption.
319
320 - Long-lived: the underlying data format is independent of its
321 storage format. It is, in principal, usable by systems as yet
322 unconceived by the next generation of programmers. This
323 implementation is based on sqlite, but the model can work with
324 arbitrary underlying storage.
325
326 - Amazingly space-efficient. The size of a repository database
327 necessarily grows as content is modified. However, Fossil's use of
328 zlib-compressed deltas, using a very space-efficient delta format,
329 leads to tremendous compression ratios. As of this writing (March,
330 2021), the main Fossil repo contains approximately 5.36GB of
331 content, were we to check out every single version in its
332 history. Its repository database is only 64MB, however, equating
333 to a 83:1 compression ration. Ratios in the range of 20:1 to 40:1
334 are common, and more active repositories tend to have higher
335 ratios. The TCL core repository, with just over 15 years of code
336 history (imported, of course, as Fossil was introduced in 2007),
337 is (as of September 2013) only 187MB, with 6.2GB of content and a
338 33:1 compression ratio.
339
340 Fossil is not...
341
342 - Memory-light. Even very small uses can easily suck up 1MB of RAM
343 and many operations (verification of the R card, for example) can
344 quickly allocate and free up hundreds of MB because they have to
345 compose various versions of content on their way to a specific
346 version. To be clear, that is total RAM usage, not _peak_ RAM
347 usage. Peak usage is normally a function of the content it works
348 with at a given time, often in direct relation to (but
349 significantly more than) the largest single file processed in a
350 given session. For any given delta application operation, Fossil
351 needs the original content, the new content, and the delta all in
352 memory at once, and may go through several such iterations while
353 resolving deltified content. Verification of its 'R-card' alone
354 can require a thousand or more underlying DB operations and
355 hundreds of delta applications. The internals use caching where it
356 would save us a significant amount of db work relative to the
357 operation in question, but relatively high memory costs are
358 unavoidable. That's not to say we can't optimize a bit, but first
359 make it work, then optimize it. The library takes care to re-use
360 memory buffers where it is feasible (and not too intrusive) to do
361 so, but there is yet more RAM to be optimized away in this regard.
362*/
363
364/** @page page_threading Threads and Fossil
365
366 It is strictly illegal to use a given fsl_cx instance from more
367 than one thread. Period.
368
369 It is legal for multiple contexts to be running in multiple
370 threads, but only if those contexts use different
371 repository/checkout databases. Though access to the storage is,
372 through sqlite, protected via a mutex/lock, this library does not
373 have a higher-level mutex to protect multiple contexts from
374 colliding during operations. So... don't do that. One context, one
375 repo/checkout.
376
377 Multiple application instances may each use one fsl_cx instance to
378 share repo/checkout db files, but must be prepared to handle
379 locking-related errors in such cases. e.g. db operations which
380 normally "always work" may suddenly pause for a few seconds before
381 giving up while waiting on a lock when multiple applications use
382 the same database files. sqlite's locking behaviours are
383 documented in great detail at https://sqlite.org.
384 */
385
386/** @page page_artifacts Creating Artifacts
387
388 A brief overview of artifact creating using this API. This is targeted
389 at those who are familiar with how artifacts are modelled and generated
390 in fossil(1).
391
392 Primary artifact reference:
393
394 https://fossil-scm.org/home/doc/trunk/www/fileformat.wiki
395
396 In fossil(1), artifacts are generated via the careful crafting of
397 a memory buffer (large string) in the format described in the
398 document above. While it's relatively straightforward to do, there
399 are lots of potential gotchas, and a bug can potentially inject
400 "bad data" into the repo (though the verify-before-commit process
401 will likely catch any problems before the commit is allowed to go
402 through). The libfossil API uses a higher-level (OO) approach,
403 where the user describes a "deck" of cards and then tells the
404 library to save it in the repo (fsl_deck_save()) or output it to
405 some other channel (fsl_deck_output()). The API ensures that the
406 deck's cards get output in the proper order and that any cards
407 which require special treatment get that treatment (e.g. the
408 "fossilize" encoding of certain text fields). The "deck" concept
409 is equivalent to Artifact in fossil(1), but we use the word deck
410 because (A) Artifact is highly ambiguous in this context and (B)
411 deck is arguably the most obvious choice for the name of a type
412 which acts as a "container of cards."
413
414 Ideally, client-level code will never have to create an artifact
415 via the fsl_deck API (because doing so requires a fairly good
416 understanding of what the deck is for in the first place,
417 including the individual Cards). The public API strives to hide
418 those levels of details, where feasible, or at least provide
419 simpler/safer alternatives for basic operations. Some operations
420 may require some level of direct work with a fsl_deck
421 instance. Likewise, much read-only functionality directly exposes
422 fsl_deck to clients, so some familiarity with the type and its
423 APIs will be necessary for most clients.
424
425 The process of creating an artifact looks a lot like the following
426 code example. We have elided error checking for readability
427 purposes, but in fact this code has undefined behaviour if error
428 codes are not checked and appropriately reacted to.
429
430 ```
431 fsl_deck deck = fsl_deck_empty;
432 fsl_deck * d = &deck ; // for typing convenience.
433 // Doxygen bug ^^^^^^^ requires space before semicolon!
434 fsl_deck_init( fslCtx, d, FSL_SATYPE_CONTROL ); // must come first
435 fsl_deck_D_set( d, fsl_julian_now() );
436 fsl_deck_U_set( d, "your-fossil-name", -1 );
437 fsl_deck_T_add( d, FSL_TAGTYPE_ADD, "...uuid being tagged...",
438 "tag-name", "optional tag value");
439 ...
440 // Now output it to stdout:
441 fsl_deck_output( f, d, fsl_output_f_FILE, stdout );
442 // See also: fsl_deck_save(), which stores it in the db and
443 // "crosslinks" it.
444 fsl_deck_finalize(d);
445 ```
446
447 The order the cards are added to the deck is irrelevant - they
448 will be output in the order specified by the Fossil specs
449 regardless of their insertion order. Each setter/adder function
450 knows, based on the deck's type (set via fsl_deck_init()), whether
451 the given card type is legal, and will return an error (probably
452 FSL_RC_TYPE) if an attempt is made to add a card which is illegal
453 for that deck type. Likewise, fsl_deck_output() and
454 fsl_deck_save() confirm that the decks they are given contain (A)
455 only allowed cards and (B) have all required
456 cards. fsl_deck_output() will "unshuffle" the cards, making sure
457 they're in the correct order.
458
459 Sidebar: normally outputing a structure can use a const form of
460 that structure, but the traversal of F-cards in a deck requires
461 (for the sake of delta manifests) using a non-const cursor. Thus
462 outputing a deck requires a non-const instance. If it weren't for
463 delta manifests, we could be "const-correct" here.
464*/
465
466/** @page page_transactions DB Transactions
467
468 The fsl_db_txn_begin() and fsl_db_txn_end()
469 functions implement a basic form of recursive transaction,
470 allowing the library to start and end transactions at any level
471 without having to know whether a transaction is already in
472 progress (sqlite3 does not natively support nested
473 transactions). A rollback triggered in a lower-level transaction
474 will propagate the error back through the transaction stack and
475 roll back the whole transaction, providing us with excellent error
476 recovery capabilities (meaning we can always leave the db in a
477 well-defined state).
478
479 It is STRICTLY ILLEGAL to EVER begin a transaction using "BEGIN"
480 or end a transaction by executing "COMMIT" or "ROLLBACK" directly
481 on a fsl_db instance. Doing so bypasses internal state which needs
482 to be kept abreast of things and will cause Grief and Suffering
483 (on the client's part, not mine).
484
485 Tip: implementing a "dry-run" mode for most fossil operations is
486 trivial by starting a transaction before performing the
487 operations. Many operations run in a transaction, but if the
488 client starts one of his own they can "dry-run" any op by simply
489 rolling back the transaction he started. Abstractly, that looks
490 like this pseudocode:
491
492 ```
493 db.begin();
494 fsl.something();
495 fsl.somethingElse();
496 if( dryRun ) db.rollback();
497 else db.commit();
498 ```
499*/
500
501/** @page page_code_conventions Code Conventions
502
503 Project and Code Conventions...
504
505 Foreword: all of this more or less evolved organically or was
506 inherited from fossil(1) (where it evolved organically, or was
507 inherited from sqilte (where it evol...)), and is written up here
508 more or less as a formality. Historically i've not been a fan of
509 coding conventions, but as someone else put it to me, "the code
510 should look like it comes from a single source," and the purpose
511 of this section is to help orient those looking to hack in the
512 sources. Note that most of what is said below becomes obvious
513 within a few minutes of looking at the sources - there's nothing
514 earth-shatteringly new nor terribly controversial here.
515
516 The Rules/Suggestions/Guidelines/etc. are as follows...
517
518
519 - C99 is the basis. It was C89 until 2021-02-12.
520
521 - The canonical build environment uses the most restrictive set of
522 warning/error levels possible. It is highly recommended that
523 non-canonical build environments do the same. Adding -Wall -Werror
524 -pedantic does _not_ guaranty that all C compliance/portability
525 problems can be caught by the compiler, but it goes a long way in
526 helping us to write clean code. The clang compiler is particularly
527 good at catching subtle foo-foo's such as uninitialized variables.
528
529 - API docs (as you may have already noticed), does not (any
530 longer) follow Fossil's comment style, but instead uses
531 Doxygen-friendly formatting. Each comment block MUST start with
532 two or more asterisks, or '*!', or doxygen apparently doesn't
533 understand it
534 (https://www.stack.nl/~dimitri/doxygen/manual/docblocks.html). When
535 adding code snippets and whatnot to docs, please use doxygen
536 conventions if it is not too much of an inconvenience. All public
537 APIs must be documented with a useful amount of detail. If you
538 hate documenting, let me know and i'll document it (it's what i do
539 for fun).
540
541 - Public API members have a fsl_ or FSL_ prefix (fossil_ seems too
542 long). For private/static members, anything goes. Optional or
543 "add-on" APIs (e.g. ::fcli) may use other prefixes, but are
544 encouraged use an "f-word" (as it were), simply out of deference
545 to long-standing software naming conventions.
546
547 - Internal APIs, especially non-static ones, start with `fsl__` or
548 `FSL__`, with two underscores. Such APIs must never be used in
549 client-side code.
550
551 - Public-API structs and functions use lower_underscore_style().
552 Static/internal APIs may use different styles. It's not uncommon
553 to see UpperCamelCase for file-scope structs.
554
555 - Function parameters and function-scope vars have no set
556 conventions - implementors are free to name those however they
557 like.
558
559 - Overall style, especially scope blocks and indentation, should
560 follow Fossil's. We are _not at all_ picky about whether or not
561 there is a space after/before parens in if( foo ), and similar
562 small details, just the overall code pattern and two-space
563 indentation. Hard tabs are verboten.
564
565 - Structs and enums all get the optional typedef so that they do
566 not need to be qualified with 'struct' resp. 'enum' when
567 used. Because of how doxygen tracks those, the typedef should be
568 separate from the struct declaration, rather than combinding
569 those into a single declaration.
570
571 - Function typedefs are named fsl_XXX_f. Implementations of such
572 typedefs/interfaces are typically named fsl_XXX_f_SUFFIX(), where
573 SUFFIX describes the implementation's
574 specialization. e.g. fsl_output_f() is a callback
575 typedef/interface and fsl_output_f_FILE() is a concrete
576 implementation for FILE handles.
577
578 - Enums tend to be named fsl_XXX_e.
579
580 - Functions follow the naming pattern prefix_NOUN_VERB(), rather
581 than the more C-conventional prefix_VERB_NOUN(),
582 e.g. fsl_foo_get() and fsl_foo_set() rather than fsl_get_foo() and
583 fsl_get_foo(). The primary reasons are (A) sortability for
584 document processors and (B) they more naturally match with OO API
585 conventions, e.g. `noun.verb()`. A few cases knowingly violate
586 this convention for the sake of readability or sorting of several
587 related functions (e.g. fsl_db_get_TYPE() instead of
588 fsl_db_TYPE_get()).
589
590 - Structs intended to be creatable on the stack are accompanied by
591 a const instance named fsl_STRUCT_NAME_empty, and possibly by a
592 macro named fsl_STRUCT_NAME_empty_m, both of which are
593 "default-initialized" instances of that struct. This is superiour
594 to using `memset()` for struct initialization because we can
595 define (and document) arbitrary default values and all clients who
596 copy-construct them are unaffected by many types of changes to the
597 struct's signature (though they may need a recompile). The
598 intention of the fsl_STRUCT_NAME_empty_m macro is to provide a
599 struct-embeddable form for use in other structs or
600 copy-initialization of const structs, and the `_m` macro is always
601 used to initialize its const struct counterpart. e.g. the library
602 guarantees that fsl_cx_empty_m (a macro representing an empty
603 fsl_cx instance) holds the same default values as fsl_cx_empty (a
604 const fsl_cx value).
605
606 - Returning int vs fsl_int_t vs fsl_size_t: int is used as a
607 conventional result code. fsl_int_t is often used as a signed
608 length-style result code (e.g. printf() semantics). Unsigned
609 ranges use fsl_size_t. Ints are (also) used as a "triplean" (3
610 potential values, e.g. <0, 0, >0). fsl_int_t also guarantees that
611 it will be 64-bit if available, so can be used for places where
612 large values are needed but a negative value is legal (or handy),
613 e.g. the final arguments for fsl_strndup() and
614 fsl_buffer_append(). The use of the fsl_xxx_t typedefs, rather
615 than (unsigned) int, is primarily for readability/documentation,
616 e.g. so that readers can know immediately that the function uses a
617 given argument or return value following certain API-wide
618 semantics. It also allows us to better define platform-portable
619 printf/scanf-style format modifiers for them (analog to C99's
620 PRIi32 and friends), which often come in handy.
621
622 - Signed vs. unsigned types for size/length arguments: use the
623 fsl_int_t (signed) argument type when the client may legally pass
624 in a negative value as a hint that the API should use fsl_strlen()
625 (or similar) to determine a byte array's length. Use fsl_size_t
626 when no automatic length determination is possible (or desired),
627 to "force" the client to pass the proper length. Internally
628 fsl_int_t is used in some places where fsl_size_t "should" be used
629 because some ported-in logic relies on loop control vars being
630 able to go negative. Additionally, fossil internally uses negative
631 blob lengths to mark phantom blobs, and care must be taken when
632 using fsl_size_t with those.
633
634 - Functions taking elipses (...) are accompanied by a va_list
635 counterpart named the same as the (...) form plus a trailing
636 'v'. e.g. fsl_appendf() and fsl_appendfv(). We do not use the
637 printf()/vprintf() convention because that hoses sorting of the
638 functions in generated/filtered API documentation.
639
640 - Error handling/reporting: please keep in mind that the core code
641 is a library, not an application. The main implication is that
642 all lib-level code needs to check for errors whereever they can
643 happen (e.g. on every single memory allocation, of which there are
644 many) and propagate errors to the caller, to be handled at his
645 discretion. The app-level code (::fcli) is not particularly strict
646 in this regard, and installs its own allocator which abort()s on
647 allocation error, which simplifies app-side code somewhat
648 vis-a-vis lib-level code. When reporting an error can be improved
649 by the inclusion of an error string, functions like
650 fsl_cx_err_set() can be used to report the error. Several of the
651 high-level types in the API have fsl_error object member which
652 contains such error state. The APIs which use that state take care
653 to use-use the error string memory whenever possible, so setting
654 an error string is often a non-allocating operation.
655*/
656
657
658/** @page page_fossil_arch Fossil Architecture Overview
659
660 An introduction to the Fossil architecture. These docs
661 are basically just a reformulation of other, more detailed,
662 docs which can be found via the main Fossil site, e.g.:
663
664 - https://fossil-scm.org/home/doc/trunk/www/concepts.wiki
665
666 - https://fossil-scm.org/home/doc/trunk/www/fileformat.wiki
667
668 Fossil's internals are fundamentally broken down into two basic
669 parts. The first is a "collection of blobs." The simplest way to
670 think of this (and it's not far from the full truth) is a
671 directory containing lots of files, each one named after a hash of
672 its contents. This pool contains ALL content required for a
673 repository - all other data can be generated from data contained
674 here. Included in the blob pool are so-called Artifacts. Artifacts
675 are simple text files with a very strict format, which hold
676 information regarding the idententies of, relationships involving,
677 and other metadata for each type of blob in the pool. The most
678 fundamental Artifact type is called a Manifest, and a Manifest
679 tells us, amongst other things, which of the hash-based file names
680 has which "real" file name, which version the parent (or parents!)
681 is (or are), and other data required for a "commit" operation.
682
683 The blob pool and the Manifests are all a Fossil repository really
684 needs in order to function. On top of that basis, other forms of
685 Artifacts provide features such as tagging (which is the basis of
686 branching and merging), wiki pages, and tickets. From those
687 Artifacts, Fossil can create/calculate all sorts of
688 information. For example, as new Artifacts are inserted it
689 transforms the Artifact's metadata into a relational model which
690 sqlite can work with. That leads us to what is conceptually the
691 next-higher-up level, but is in practice a core-most component...
692
693 Storage. Fossil's core model is agnostic about how its blobs are
694 stored, but libfossil and fossil(1) both make heavy use of sqlite
695 to implement many of their features. These include:
696
697 - Transaction-capable storage. It's almost impossible to corrupt a
698 Fossil db in normal use. sqlite3 offers literally the most robust
699 general-purpose file format on the planet.
700
701 - The storage of the raw blobs.
702
703 - Artifact metadata is transformed into various DB structures
704 which allow libfossil to traverse historical data much more
705 efficiently than would be possible without a db-like
706 infrastructure (and everything that implies). These structures are
707 kept up to date as new Artifacts are stored in a repository,
708 either via local edits or synching in remote content. These data
709 are incrementally updated as changes are made to a repo.
710
711 - A tremendous amount of the "leg-work" in processing the
712 repository state is handled by SQL queries, without which the
713 library would easily require 5-10x more code in the form of
714 equivalent hard-coded data structures and corresponding
715 functionality. The db approach allows us to ad-hoc structures as
716 we need them, providing us a great deal of flexibility.
717
718 All content in a Fossil repository is in fact stored in a single
719 database file. Fossil additionally uses another database (a
720 "checkout" db) to keep track of local changes, but the repo
721 contains all "fossilized" content. Each copy of a repo is a
722 full-fledged repo, each capable of acting as a central copy for
723 any number of clones or checkouts.
724
725 That's really all there is to understand about Fossil. How it does
726 its magic, keeping everything aligned properly, merging in
727 content, how it stores content, etc., is all internal details
728 which most clients will not need to know anything about in order
729 to make use of fossil(1). Using libfossil effectively, though,
730 does require learning _some_ amount of how Fossil works. That will
731 require taking some time with _other_ docs, however: see the
732 links at the top of this section for some starting points.
733
734
735 Sidebar:
736
737 - The only file-level permission Fossil tracks is the "executable"
738 (a.k.a. "+x") bit. It internally marks symlinks as a permission
739 attribute, but that is applied much differently than the
740 executable bit and only does anything useful on platforms which
741 support symlinks.
742
743*/
744
745#endif
746/* ORG_FOSSIL_SCM_PAGES_H_INCLUDED */