Binaries-have-conflict

Nov 25, 2015
DebianDebian diary

Here are some notes on how to deal with a certain problem in Debian packaging, which might be useful for people coming across the same issue. In certain cases you want to have two binary packages co-installable, which normally wouldn’t be because of the restriction that two packages can’t be installed at the same time which contain the same files: every single file in the filesystem have to belong to a single package¹, and couldn’t be overwritten by another package.

This problem arises e.g. when packaging Python libraries which are shipped with executables like helper scripts or CLI tools. Usually, two packages for the two Python stacks resp. runtime systems are being build out of the source package, python-foo and python3-foo². The included modules are going to be installed into different paths³, and there’s also other stuff in the packages which doesn’t conflict. No problem with that.

But, in both binary packages the same helper scripts usually are going to be installed into /usr/bin, which is a problem. Although the content of scripts might differ⁴, the two packages couldn’t installed at the same time because there is a binary conflict:

$ sudo debi citeproc-py_0.3.0-1_amd64.changes 
Selecting previously unselected package python-citeproc.
(Reading database ... 442052 files and directories currently installed.)
Preparing to unpack python-citeproc_0.3.0-1_all.deb ...
Unpacking python-citeproc (0.3.0-1) ...
Selecting previously unselected package python3-citeproc.
Preparing to unpack python3-citeproc_0.3.0-1_all.deb ...
Unpacking python3-citeproc (0.3.0-1) ...
dpkg: error processing archive python3-citeproc_0.3.0-1_all.deb (--install):
 trying to overwrite '/usr/bin/csl_unsorted', which is also in package python-citeproc 0.3.0-1
Setting up python-citeproc (0.3.0-1) ...
Processing triggers for man-db (2.7.2-1) ...
Errors were encountered while processing:
 python3-citeproc_0.3.0-1_all.deb
debi: debpkg -i failed

The all-around Lintian⁵ could be used to detect also this problem⁶ on the source package:

$ lintian citeproc-py_0.3.0-1_amd64.changes 
W: citeproc-py source: binaries-have-file-conflict python-citeproc python3-citeproc usr/bin/csl_unsorted

If two packages including the same file or files (meaning: filename and path) are supposed to be alternatively installable they need to have being declared as conflicting against each other⁷. But concerning Python libraries it’s definitely wanted that it is possible to have both runtime packages being installed together: people just want to run and develop things on both Python stacks on the same system. But how to handle the conflicting executables⁸?

There are comparatively easy ways to prevent rivaling binaries, but which all have some disadvantages. To begin with that, the scripts could be renamed (like by adding suffixes like -python2/-python3) to prevent filename duplication. But users which know the software from other setups might not find them, and that also probably does not correspond to what’s said in the documentation. And, this solution also needs duplicated manpages.

Another solution would be to build the scripts into /usr/share/⁹ and to require the user to run the stuff from here or to add that to $PATH, but this isn’t really convenient for the user. However, if scripts are of minor importance, that approach saves the maintainer to write manpages (which are related to stuff in /usr/bin).

When it doesn’t matter on which Python it runs it’s also possible to contribute scripts only by one of the two packages. I’ve chosen this solution for Citeproc-py¹⁰, where I’ve removed csl_unsorted from the Python 2 package. This solutions means that one have to install also the whole, probably otherwise unused Python 3 stuff only for running the shipped helper script. But here it’s o.k. because this library isn’t that big for which it’s justified not to have set up a more complex solution for the binaries-have-conflict problem.

Most intricate to set up, but also most convenient for the user is to install an alternatives mechanism to handle conflicting executables for co-installable packages. I’ve did that for my package of Python-afl¹¹. These two packages contain scripts (wrapper for afl), and Cython generated extensions in the public import paths¹². The user has either one or both packages installed and could run the script /usr/bin/py-afl-fuzz in any case: that is very much like it’s wanted¹³.

Alternatives were created for the case that two different programs in two different binary packages, which aren’t handled by declaring “Conflicts” and therefore could be installed at the same time, offering the same functionality by different implementations¹⁴. Using the alternative system, the user can choose which implementation her prefers, and switch between them every time that’s wanted without installing or removing packages. I’ve got alternatives installed e.g. for Pybtex¹⁵, which is an alternative implementation of the bibliography preprocessor bibtex (package: texlive-binaries)¹⁶:

$ update-alternatives --display pybtex
pybtex - auto mode
  link best version is /usr/bin/bibtex.pybtex
  link currently points to /usr/bin/bibtex.pybtex
  link pybtex is /usr/bin/pybtex
  slave pybtex.1.gz is /usr/share/man/man1/pybtex.1.gz
/usr/bin/bibtex.pybtex - priority 200
slave pybtex.1.gz: /usr/share/man/man1/bibtex.pybtex.1.gz

$ sudo update-alternatives --config bibtex
There are 2 choices for the alternative bibtex (providing /usr/bin/bibtex).

  Selection    Path                      Priority   Status
------------------------------------------------------------
  0            /usr/bin/bibtex.original   100       auto mode
  1            /usr/bin/bibtex.original   100       manual mode
* 2            /usr/bin/bibtex.pybtex     10        manual mode

Press  to keep the current choice[*], or type selection number: 

For Python-afl, alternatives have been set up like this: during build, the script(s) are put into /usr/share/ by dh_auto_install using custom settings for Pybuild. Thus, in the binary packages there are the files /usr/share/python-afl/py-afl-fuzz and /usr/share/python3-afl/py-afl-fuzz, which do not conflict.

The alternatives are put into action during the installation procedure of the packages by the postinst maintainer shell scripts¹⁷ in debian/, which run update-alternatives after the packages have been unpacked:

$ cat debian/python-afl.postinst
#!/bin/sh
set -e
if [ "$1" = configure ]
then
    update-alternatives \
    --install /usr/bin/py-afl-fuzz py-afl-fuzz /usr/share/python-afl/py-afl-fuzz 20
fi
#DEBHELPER#

The corresponding postinst script in the Python 3 package (both reduced here) installs another alternative for /usr/bin/py-afl-fuzz¹⁸:

$ cat debian/python3-afl.postinst 
#!/bin/sh
set -e
if [ "$1" = configure ]
then
    update-alternatives \
    --install /usr/bin/py-afl-fuzz py-afl-fuzz /usr/share/python3-afl/py-afl-fuzz 10
fi
#DEBHELPER#

The priorities (“20” and “10”) could have been chosen differently, this way the alternatives just remain unaltered if the Python 2 package has been installed first¹⁹. The alternatives are getting removed during the uninstall procedure by the corresponding prerm scripts, which trigger update-alternatives --remove²⁰. Alternatives are handled by symbolic links, /usr/bin/py-afl-fuzz links to /etc/alternatives/py-afl-fuzz, and that again to /usr/share/python-afl/py-afl-fuzz, or the other one. Like said, with a setup like this the user always has py-afl-fuzz in $PATH, either if one of the two packages is installed, or both (to switch between the alternatives with --config isn’t really needed here because it’s the same script). A hint: the package installation testing tool Piuparts²¹ could be used to check over if maintainer scripts are running as they are supposed to²².

Manpages like /usr/share/man/man1/py-afl-fuzz.1.gz are also conflicting, but this problem could also be solved using alternatives: for Python-afl, two manpages having different file names are generated with txt2man for the two binary packages, py-afl-fuzz.py2.1 and py-afl-full.py3.1. And for update-alternatives, slaves could be used with --install²³.

update-alternatives \
--install /usr/bin/py-afl-fuzz py-afl-fuzz /usr/share/python-afl/py-afl-fuzz 20 \
--slave /usr/share/man/man1/py-afl-fuzz.1.gz py-afl-fuzz.1.gz /usr/share/man/man1/py-afl-fuzz.py2.1.gz

update-alternatives \
--install /usr/bin/py-afl-fuzz py-afl-fuzz /usr/share/python3-afl/py-afl-fuzz 10 \
--slave /usr/share/man/man1/py-afl-fuzz.1.gz py-afl-fuzz.1.gz /usr/share/man/man1/py-afl-fuzz.py3.1.gz

Though, not so elegant is the fact that if both packages are installed at the same time, there are even three identical manpages on the system, which could be confusing for the user:

$ man py-afl-fuzz
py-afl-fuzz   py-afl-fuzz.py2   py-afl-fuzz.py3

This isn’t really optimal, and I’m seeking out for a solution on this point of solving binaries-have-conflict with alternatives. I’m thinking about putting the manpages also into /usr/share, but that’s not so refined, either. Thus, this solution has its downside with manpages.