bk-Howto-BAM - (unknown subject)
NAME
bk Howto-BAM − configuring the Binary Asset Management
(BAM) subsystem
OVERVIEW
Binary Asset Management, or BAM, is a storage management
system for versioning larger binary assets, such as tool
chains, libraries, photos, music, videos, etc.
The BAM system is a hybrid, combining the best of the traditional centralized models with advantages of the distributed model developed by BitKeeper. When BAM is enabled, and a BAM server is set, then BAM files in BitKeeper repositories are passed by reference rather than copied. BAM files appear to be the same as regular BitKeeper files but they contain no data, only a pointer which names the data. The data typically resides in the BAM server and is fetched on demand as needed.
What this means to a user is that a 20GB repository full of BAM files can be cloned in a few seconds and if the user needs to work on only a small portion of the data, only that data needs to be fetched from the BAM server.
CONFIGURATION
In order to have larger binaries be stored in BAM format,
the configuration file (BitKeeper/etc/config) will need to
have BAM enabled:
BAM: on
In order to fetch BAM data on demand a server must be configured:
$ bk bam server bk://MyBigBox/MyRepo
Note that the BAM server is per repository construct that is inherited on clone, much like the repository level is inherited on clone.
CONVERTING
REPOS
If you have a repository that contains larger binaries
(minimum checked out size of 64KB in revision 1.1), then you
may do a conversion to BAM.
The conversion should be done in a clone because the conversion changes the identity of the repository, i.e., the converted repository will no longer be able to synchronize with the unconverted repositories. Note that while the conversion does change the package identity, it is idempotent so other repositories may also do the conversion and they will end up with the same package identity and be able to synchronize.
The conversion process will add a "BAM:on" variable to your config file. If you wish to minimize the number of those changesets, turn that variable on in your main tree and tell your team to pull that changeset before converting.
The conversion process tends to be longer in projects with more changesets and files (for example, the MySQL 5.2 tree with 51,000 changesets and 14,000 files took 5 minutes on a 2Ghz Opteron running Linux).
The conversion process is simple:
bk clone
my_repo my_repo.BAM
cd my_repo.BAM
bk bam convert
USAGE
Once the repository is converted, usage is as it was in the
past, you clone, pull, push, etc., as normal. In order to
check out BAM files the BAM server listed in the config file
must be accessible. At the end of a "bk -U get"
command you may see lines like:
Fetching 18 BAM
files from bk://MyBigBox/MyRepo...
BitKeeper/BAM/62/62af0d9c.d1 (5.41M of 6.29M)
followed by the normal check out messages for the associated files.
CHECKOUT
There is a BAM_checkout configuration variable which
is specific to BAM files. If this variable is not set then
BAM files use the checkout configuration variable.
For large collections of large files partitioned by
directories, the BitKeeper support team suggests
BAM_checkout:last as the best setting. That
setting will not pull BAM data into a clone until a user
requests an edit or a checkout. After that, the file will
remain in whatever mode it was, even across deltas.
SEE ALSO
bk-Howto, bk-bam, bk-newroot
CATEGORY
Overview