Carmel - CPAN Artifact Repository Manager

NAME SYNOPSIS DESCRIPTION WORKFLOW Development Production Deployments HOW IT WORKS SNAPSHOT SUPPORT CAVEATS / KNOWN ISSUES COMPARISONS WITH SIMILAR TOOLS Carton cpm COMMUNITY AUTHOR COPYRIGHT LICENSE SEE ALSO

NAME

Carmel − CPAN Artifact Repository Manager

# Run with a directory with cpanfile
carmel install
# list all the modules to be loaded
carmel list
# list all the modules in a tree
carmel tree
# show a location where a module is installed
carmel show Plack
# update Plack to the latest
carmel update Plack
# update all the modules in the snapshot
carmel update
# pin modules tp specific versions
carmel update [email protected] [email protected]
# show diffs for cpanfile.snapshot in a nice way
carmel diff
# Runs your perl script with modules from artifacts
carmel exec perl ...
# Requires all your modules in cpanfile in one shot
carmel exec perl −e 'use Carmel::Preload;'
# Roll out the currently selected modules into ./local
carmel rollout
# package modules tarballs and index into ./vendor/cache
carmel package
# use Carmel packages inside a script (without carmel exec)
perl −e 'use Carmel::Setup; ...'
# prints export PATH=... etc for shell scripting
carmel export
# find a module in a repository
carmel find DBI
# find a module matching the version query
carmel find Plack ">= 1.0000, < 1.1000"

DESCRIPTION

Carmel is yet another CPAN module manager.

Unlike traditional CPAN module installer, Carmel keeps the build of your dependencies in a central repository, then select the library paths to include upon runtime in development.

Carmel also allows you to rollout all the files in a traditional perl INC directory structure, which is useful to use in a production environment, such as containers.

WORKFLOW

Development

During the development, run "carmel install" when you setup a new environment, and any time you make changes to "cpanfile". This will update your build artifacts, and saves the changes to "cpanfile.snapshot". Commit the snapshot file in version control system so that you can reproduce the exact same versions across machines.

"carmel exec" makes it easy to run your application using the versions in "cpanfile" and "cpanfile.snapshot" dynamically.

# On your development environment
> cat cpanfile
requires 'Plack', '0.9980';
requires 'Starman', '0.2000';
> carmel install
> echo /.carmel >> .gitignore
> git add cpanfile cpanfile.snapshot .gitignore
> git commit −m "add Plack and Starman"
# On a new setup, or another developer's machine
> git pull
> carmel install
> carmel exec starman −p 8080 myapp.psgi
# Add a new dependency
> echo "requires 'Try::Tiny';" >> cpanfile
> carmel install
> git commit −am 'Add Try::Tiny'
# Update Plack to the latest
> carmel update Plack

Production Deployments

Carmel allows you to manage all the dependencies the same way across development environments and production environments. However, there might be cases where you want to avoid running your application with "carmel exec" in production, to avoid the overhead with large number of include paths, or to avoid installing Carmel in the production hosts.

Carmel provides two easy ways to avoid depending on Carmel on the deploy target environments. First, "carmel rollout" rolls out the build artifacts into a regular perl5 library path in "local". Once the rollout is complete, you can include the path just like a regular local::lib directory.

# Production environment: Roll out to ./local
> carmel rollout
> perl −Ilocal/lib/perl5 local/bin/starman −p 8080 myapp.psgi

You can run "carmel rollout"> in a CI system to create the "local" directory next to your application code for a linux package (e.g. deb package), or Docker containers.

Second, "carmel package" (similar to "carton bundle") creates a directory with tarballs and CPAN−style package index files, which you can pass to cpanm on a target machine. This way, you only need "cpanm", which is available as a self−contained single executable, to bootstrap the installation on a host with a stock perl.

# Vendor all the packages to vendor/cache
> carmel package
> git add vendor/cache
> git commit −m 'Vendor all the tarballs'
# Remote environment (CI etc.)
> git clone https://.../myapp.git && cd myapp
> cpanm −L /path/to/lib −−from $PWD/vendor/cache −nq −−installdeps .

HOW IT WORKS

Carmel will keep the build directory (artifacts) after a cpanm installation in a repository, which defaults to "$HOME/.carmel/{version}−{archname}/builds", and your directory structure would look like:

$HOME/.carmel/5.20.1−darwin−2level/builds
Plack−1.0033/
blib/
arch/
lib/
URI−1.64/
blib/
arch/
lib/
URI−1.63/
blib/
arch/
lib/

Carmel scans this directory and creates the mapping of which version of any package belongs to which build directory.

Given the list of modules and requirements from "cpanfile", "carmel install" computes which versions satisfy the requirements best, and if there isn't, installs the modules from CPAN to put it to the artifact repository. The computed mappings are preserved as a snapshot in "cpanfile.snapshot".

Once the snapshot is created, each following "carmel" command runs uses both "cpanfile" and "cpanfile.snapshot" to determine the best versions to satisfy the requirements. When you update "cpanfile" to bump a version or add a new module, "carmel" will install the new dependencies and update the snapshot accordingly.

"carmel exec" command, like "install" command, lists the build directories and ".pm" files you need from the repository, and then prepend the mappings of these files in the @INC hook. This is a handy way to run a perl program using the dependencies pinned by Carmel, without changing any include path.

"carmel update" command allows you to selectively update a dependency while preserving other dependencies in the snapshot. "carmel update Plack" for example pulls the latest version of Plack from CPAN (and its dependencies, if it needs a newer version than pinned in the snapshot), and updates the snapshot properly. Running "carmel update" without any arguments would update all the modules in "cpanfile", including its dependencies.

On a production environment, you might want to use the "carmel rollout" command, which saves all the files included in the "cpanfile", pinned with "cpanfile.snapshot", to the "local" directory. This directory can be included like a regular perl's library path, with "PERL5LIB=/path/to/local/lib/perl5", or with "use lib", and you don't need to use "carmel" command in production this way.

SNAPSHOT SUPPORT

As of v0.1.29, Carmel supports saving and loading snapshot file in "cpanfile.snapshot", in a compatible format with Carton. Versions saved in the snapshot file will be preserved across multiple runs of Carmel across machines, so that versions frozen in one environment can be committed to a source code repository, and can be reproduced in another box, so long as the perl version and architecture is the same.

CAVEATS / KNOWN ISSUES

	•		If you run multiple instances of "carmel", or hit Ctrl−C to interrupt the cpanm install session, Carmel might get into a state where some modules have been installed properly, while some modules in the dependency chain are missing. Make sure you don't run multiple instances of "carmel" at the same time, and let it finish the installation to get the full builds properly.
	•		There're certain dependencies that get missed during the initial "carmel install", and you'll see the error message "Can't find an artifact for Foo".

Please report it to the issue tracker if you can reliably reproduce this type of errors.

Usually you run "carmel install" again and the error will be gone.

•

In some situation, you might encounter conflicts in version resolutions, between what's pinned in the snapshot and a new version that's needed when you introduce a new module.

For example, suppose you have:

# cpanfile
requires 'Foo';
requires 'Bar'; # which requires Foo >= 1.001

Without the snapshot, Carmel has no trouble resolving the correct versions for this combination. But if you have:

# cpanfile.snapshot
Foo−1.000

The first time you run "carmel install", Carmel will try to install Foo−1.000, because that's the version pinned in the snapshot, while trying to pull the module Bar, which would conflict with that version of Foo.

This can happen 50% of the time, because if cpanm (called internally by Carmel) installs Bar first, then the resolution is done correctly and the version in the snapshot would be skipped, and the snapshot will be updated accordingly. This is due to perl's hash randomization after Perl 5.18.

To avoid this, you're recommended to run "carmel install" before making any changes to cpanfile. That will put the build caches to satisfy what's in cpanfile and the snapshot. After that, adding a new dependency will likely reuse what's in the build cache, while adding a new dependency can update the transient dependency (for Foo) without having conflicts.

If you encounter conflicts like this, you can work around it by:

	•		Run "carmel update Foo" to pull the latest version of Foo from CPAN, ignoring what's in the snapshot.
	•		Update "cpanfile" to explicitly update the version requirement for "Foo".

COMPARISONS WITH SIMILAR TOOLS

Carton

Carmel shares the same goal with Carton, where you can manage your dependencies by declaring them in "cpanfile", and pinning them in "cpanfile.snapshot". Most of the commands work the same way, so Carmel can most effectively be a drop−in replacement for Carton, if you're currently using it.

Here's a few key differences between Carmel and Carton:

•

Carton does not manage what's currently being installed in "local" directory. It just runs "cpanm" command with "−L local", with a hope that nothing has changed the directory except Carton, and whatever is in the directory won't conflict with the snapshot file. This can easily conflict when "cpanfile.snapshot" is updated by multiple developers or when you continuously update the dependencies across multiple machines.

Carmel manages all the dependencies for your project in the Carmel repository under "$HOME/.carmel", and nothing is installed under your project directory on development. The "local" directory is only created when you request it via "carmel rollout" command, and it's safe to run multiple times. Running "carmel install" after pulling the changes to the snapshot file will always install the correct dependencies from the snapshot file, as compared to Carton, which doesn't honor the snapshot on a regular install command, if whatever version in "local" already satisfies the version in "cpanfile".

•

Carton has no easy way to undo a change once you update a version of a module in "local", because which version is actually selected is only preserved as a file inside the directory, that's not managed by Carton. To undo a change you have to remove the entire "local" directory to start over.

Carmel preserves this information to the "cpanfile.snapshot" file, and every invocation of Carmel resolves the dependencies declared in "cpanfile" and pinned in "cpanfile.snapshot" dynamically, to create a stable dependency tree, without relying on anything in a directory under your project other than the snapshot file. Undoing the change in "cpanfile.snapshot" file immediately reverts the change.

cpm

App::cpm is an excellent standalone CPAN installer.

	•		Like Carton, cpm installs the dependencies declared in "cpanfile" to "local". Carmel installs them into a build cache, and doesn't use "local" directory for state management. You can run "carmel rollout" to copy the dependencies to "local" directory.
	•		cpm installs the modules in parallel, which makes the installation very fast. Like Carmel, cpm also manages its build artifacts cache, so a module that has previously been installed would be very fast to install, since there's no build process.
	•		Unlike Carton and Carmel, cpm doesn't have the ability to manage "cpanfile.snapshot" file on its own. It can read the snapshot however, so it's possible to use Carmel in a development environment, and then use "cpm install" instead of "carmel install" and "carmel rollout", if all you need is to build out a perl5 library path out of "cpanfile" and "cpanfile.snapshot" in the source code repository.