Mercurial
Mercurial is a free, distributed source control management tool. It efficiently handles projects of any size and offers an easy and intuitive interface.
1. Why Mercurial?
Git and Mercurial are two different tools with the same job, it has just been my experience that Mercurial does them a little bit better for my workflow, and I am able to build things faster using it.
- Mercurial is designed to provide tools for building a DVCS, and encourages you to expand your knowledge about the system to reap the benefits it provides. With Git I found myself often avoiding the implementation details in favor of quick-fixes and ugly scripts. On the other hand, I feel much more comfortable designing a solution with the tools Mercurial provides.
- the Mercurial system has features such as immutable commits and branches, which can be quite dangerous. There are ways to 'fix' changes made by these commands, but they go against the grain of the rest of the system. Git on the other hand lets you seamlessly edit commits and close branches, which can be life-savers when Billy the Intern commits to master.
- Mercurial also has superior features for exporting native repository to different DVCS (Git, Darcs, SVN), making it more useful in scenarios where a repo needs to be shared with users or tools that are unfamiliar with the 'hg' command.
- Overall, the choice is a personal one. Over time I have felt less constrained, as if there were less rules to follow with Mercurial. This is a far-cry from my first experience with it, where I felt the complete opposite.
2. Web Hosting
Hosting with vanilla Mercurial is quite similar to vanilla Git. hg serve
is basically
the equivalent to git instaweb
. Where they differ is in the out-of-box solutions - Git
has cgit, GitLab, Gitolite, git.sr.ht, etc. Mercurial has hg.sr.ht, and just recently
Heptapod, which is not production-ready.
My experience with self-hosting the sr.ht eco-system was far from a good one due to conflicting dependencies, package manager incompatibility, and some classic PHP craziness. The Heptapod docker container took eons just to build tests, so I just gave up on that for now but will be on the look out for new developments with that project. So what we're really left with is the built-in tools. Lucky for us, we have the hgweb scripts at our disposal.
Here's the relevant docs covering all topics in this section. They are all must-reads if you plan on exposing a Mercurial server to the public.
- PublishingRepositories - Mercurial
- SecuringRepositories - Mercurial
- AuthorizingUsers - Mercurial
hgweb + wsgi + nginx
The hgweb script is used for deployment of the server via CGI or WSGI. The WSGI setup is a bit more involved, but according to the docs:(!) Much better performance can be achieved by using WSGI instead of CGI.
This section covers the WSGI (pronounced whis-gee) setup, specifically for Nginx. The docs have better examples for Apache servers, so you do need to go off the beaten path to find just the right values to set in Nginx. The following setup worked for https://hg.rwest.io running Arch Linux.
- dependencies
uwsgi
anduwsgi-plugin-python
are required for the hgweb.wsgi script.nginx
is used here, but…apache
package is needed if you want to usehtpasswd
while setting up HTTP Authentication. SSH auth only or public hosts don't need this.
/etc/uwsgi/hgweb.ini
This is a UWSGI service configuration file - thehgweb.wsgi
script needs to be located in the directory specified bychdir
. If you are using systemd, try enabling the service withsystemctl enable uwsgi@hgweb.service
, and check the logs for any errors. The service calls the hgweb.wsgi script, which serves all repos configured viahgweb.conf
.[uwsgi] master = true ; max-requests = 1000 ; logto = {log file path}/hgweb-uwsgi.log uid = hgweb ; set process owner gid = hgweb stats = /run/uwsgi/stats.sock chmod-socket = 666 cap = setgid,setuid ; https://www.mercurial-scm.org/wiki/PublishRepositoriesOnNginx plugins = python socket = /run/uwsgi/hgweb.sock chdir = /home/hgweb/hg wsgi-file = hgweb.wsgi ; https://stackoverflow.com/questions/15878176/uwsgi-invalid-request-block-size ; http://uwsgi-docs.readthedocs.io/en/latest/ThingsToKnow.html ; buffer-size = 65535
hgweb.wsgi
The config path must be a byte string, and an absolute path. This script needs to be in the same directory ashgweb.conf
.#!/usr/bin/env python3 config = b"/home/hgweb/hgweb.conf" # Uncomment to send python tracebacks to the browser if an error occurs: #import cgitb; cgitb.enable() # enable demandloading to reduce startup time from mercurial import demandimport; demandimport.enable() from mercurial.hgweb import hgweb application = hgweb(config)
hgweb.conf
Mercurial web server configuration file. Setting staticurl to/static
allows us to pass serving of static content to Nginx, which is faster and has better caching controls.[web] encoding = UTF-8 baseurl = https://hg.rwest.io contact = some_dude templates = theme style = spartan logourl = https://rwest.io staticurl = /static descend = True collapse = True [paths] / = src/*
/etc/nginx/sites-enabled/hg.conf
Note that none of the extra uwsgi params from mercurial docs are used here, only the default uwsgiparams files provided by Nginx.server { server_name hg.rwest.io; listen 443 ssl; ssl_certificate fullchain.pem; ssl_certificate_key privkey.pem; ssl_protocols TLSv1.1 TLSv1.2 TLSv1.3; ssl_ciphers HIGH:!aNULL:!MD5; root /home/hgweb/hg; location / { limit_except GET HEAD { auth_basic "Mercurial userspace"; auth_basic_user_file /home/hgweb/hg/hg.htpasswd; } include uwsgi_params; uwsgi_pass unix:/run/uwsgi/hgweb.sock; } location /static { alias /home/hgweb/hg/static; expires 30d; } }
- dependencies
3. Bundles
Hg Bundles are a more powerful version of git bundles and come in two flavors:
Clonebundles and Pullbundles. They share a common format, support the bundle
and
unbundle
commands, but are also used to provision their own commands: hg clone
and
hg pull
respectively. Bundles are advertised via manifest files located in the .hg
directory of a repo, and support a variety of compression backends via the BUNDLESPEC
value specified in manifest.
- Bundlespec
- none-v2
- gzip-v2
- zstd-v2
- stream-v2
- Manifests
- clonebundles.manifest
- pullbundles.manifest
- manifest.json
- Hosting
4. Mercurial <-> Git interop
as of 2022 I've been using hg-fast-export to convert my hg repos to git so they can be mirrored on github.
It is quite easy to integrate this process with CI:
# >> make mirror # root path ROOT=$(dir $(abspath $(firstword $(MAKEFILE_LIST)))) # this is wherever you pulled hg-fast-export to STASH=~/stash # path to hg-fast-export binary FE=$(STASH)/fast-export/hg-fast-export.sh mirror:$(FE) $(ROOT) out mkdir -p out/$@; git init out/$@; cd out/$@ && \ git config core.ignoreCase false && git config push.followTags true && \ # run with '-M default' for default branch $(FE) -r $(ROOT) -M default && git checkout HEAD && \ git remote add gh git@github.com:richardwesthaver/mpk.git && \ git push gh --all; @rm -rf out/$@;
# config.nims -- >> nim mirror --hints:off import std/distros import std/os import std/sequtils from std/strutils import join const stash {.strdefine.}: string = expandTilde("~/stash") fastexport {.strdefine.}: string = stash / "fast-export/hg-fast-export.sh" proc getVcRoot(): string = ## Try to get the path to the current VC root directory. ## Return ``projectDir()`` if a ``.hg`` or ``.git`` directory is not found. const maxAttempts = 10 var path = projectDir() attempt = 0 while (attempt < maxAttempts) and (not (dirExists(path / ".hg") or (dirExists(path / ".git")))): path = path / "../" attempt += 1 if dirExists(path / ".hg"): result = path elif dirExists(path / ".git"): result = path else: echo "no VC root found, defaulting to projectDir" result = projectDir() task mirror, "push code to github mirror": withDir stash: exec "git init mpk" withDir "mpk": exec "git config core.ignoreCase false" exec "git config push.followTags true" exec fastexport & " -r " & getVcRoot() & " -M default" exec "git checkout HEAD" exec "git remote add gh git@github.com:richardwesthaver/mpk.git" var args: seq[string] when defined(f): args.add("--force") exec "git push gh --all --force " & args.join rmDir("mpk")
- Two different plugins - git and hggit, different features
- 'git' usage will treat a cloned repo as a git repo and just pull branch heads into mercurial (.git directory and .hg directory present at root)
- 'hggit' will convert all changesets, branches, etc (.hg directory only)
- pulling changesets can be incredibly slow at times
- unlike vanilla hg, pulling from a git repo with hggit does NOT support revision CLI flag ('-r 100' will not work), so the standard method of doing an incremental pull is not possible
- expect unexpected behaviors. always have backups.
- hggit will often pull the 'github-pages' branch from repos using GitHub Sites and treat it as the default branch in the resulting clone. To make the repo usable you need to update to the 'master' bookmark, and use that as default
5. Scripts
hg-pull.sh
#!/bin/bash # store the current dir CD=$(pwd) echo "Pulling in latest changes for all local repositories..." # Find all mercurial repositories, pull and update for i in $(find . -name ".hg" | cut -c 3-); do echo ""; echo $i; # We have to go to the .hg parent directory to call the pull command cd "$i"; cd ..; # pull and update hg pull -u; # go back to the CUR_DIR cd $CD done echo "Done."
hg-bundle.sh
#!/bin/bash # bundle a tar.zst archive of Mercurial repositories. CD=$(pwd) WD=$HOME/stash/tmp OUT=$WD/bundle SRC_PATH=$HOME/src BUNDLE_NAME=bundle-$(date "+%Y%m%d").tar.zst echo "Building $BUNDLE_NAME in $WD..." mkdir -pv $OUT rm -rf $OUT/* rm -rf $WD/$BUNDLE_NAME cd $SRC_PATH # Find all mercurial repositories, create bundles and dump them to $OUT dir for i in $(find . -name ".hg" | cut -c 3-); do echo ""; echo $i; cd "$i"; cd ..; hg bundle -a -t gzip-v2 $OUT/$(basename $(hg root)).hg.gz; hg bundle -a -t zstd-v2 $OUT/$(basename $(hg root)).hg.zst; hg bundle -a -t none-v2 $OUT/$(basename $(hg root)).hg; hg debugcreatestreamclonebundle $OUT/$(basename $(hg root)).hg.stream; echo "... Done."; cd $SRC_PATH done cd $WD # this will take a while with ultra mode tar -I 'zstd --ultra -22' -cf $BUNDLE_NAME bundle/ echo "Done."
hg-unbundle.sh
#!/bin/sh # unbundle a tar.zst archive of Mercurial repositories. # this will generate a directory name 'bundle' in '~/pkg/hg/' WD=$HOME/stash/tmp BUNDLE_NAME=bundle-$(date "+%Y%m%d") PKG_DIR=$HOME/pkg/hg echo "unbundling $i to $PKG_DIR/bundle" # the zstd options for tar no work for me, decompress archive (this should be MacOS only, maybe Win. need to add checks) unzstd $WD/$BUNDLE_NAME.tar.zst tar -xvf $WD/$BUNDLE_NAME.tar -C $PKG_DIR rm -rf $WD/$BUNDLE_NAME.tar.zst $WD/$BUNDLE_NAME.tar echo "Done."
6. Further Reading
- Misconceptions about Monorepos: Monorepo != Monolith - Victor Savkin 2019
- Why Google Stores Billions of Lines of Code in a Single Repository - Josh Levenberg, 2016