The SafeRepo Initiative
Contents
Update as of 2009-10-19
This wiki page is the original concept around SafeRepo, however since then we have added a site for this:
Introduction
The SafeRepo Initiative is a proposal for a standard in 3rd party Yum repositories. Currently there is none.
Terminology
Distro or Stock Distro: Refers to a Linux distribution as it comes from the vendor, such as Redhat, CentOS, Fedora, Etc.
Repo or 3rd Party Repo: Refers to any 3rd party yum repository that is a system can subscribe to in order to install additional packages that the Stock Distro doesn't provide.
3rd Party Package: Refers to an rpm package that a 3rd Party Repo provides.
Stock Package: Refers to an rpm package that a Stock Distro provides.
The Current Problem with 3rd Party Repos
3rd party yum repositories introduce a level of unknown compatibility, as well as unknown usage. One repo might offer 3rd Party Packages that are meant to replace existing Stock Packages within a Stock Distro, and others might explicitly only add packages to a distribution. However there are many that you just don't know what they will do when you subscribe, and often they simply override Stock Packages from the Distro which is unsafe in a production environment.
Some examples of safe repositories:
- Fedora EPEL: Extra Packages for Enterprise Linux is a repository geared towards adding packages to RHEL, but have strict guidelines that packages will never replace or conflict with existing RHEL packages. Therefore, subscribing to EPEL is safe in that nothing will happen unless you explicitly install a package from the repo.
- IUS Community: The IUS Community Project provides packages for RHEL that follow the latest upstream stable source versions of specific software such as PHP, MySQL, Python, Etc. IUS is a safe repo because it uses alternate package names that provide the same software (for example php52 provides php) but do not obsolete. IUS packages explicitly conflict with stock packages, therefore preventing anything from IUS automatically installing on/over RHEL.
Examples of unsafe repositories:
- Any 3rd Party Repo that offers 3rd Party Packages that obsolete or directly replace Stock Packages because they have the same package name. For example, if a repository has 'php-5.2.11' and you subscribe to it, your server will automatically upgrade PHP to that version. Additionally, if you don't have software installed but attempt to install something that has a dependency, should the 3rd Party Repo provide that dependency overriding the Stock Disto, it is unsafe. Meaning, if you don't have php installed at all and attempt to install 'cacti' (requires php, php-mysql, etc), no 3rd Party Repo should resolve that dependency over the Stock Distro.
SafeRepo Package Requirements
Package requirements to provide a SafeRepo are simple.
- The 3rd Party Package is an addon and does not currently exist in the Stock Distro. Should the Stock Distro later add the package, the 3rd Package Package becomes a Stock Package and should be removed from the 3rd Pary Repo.
OR
The 3rd Party Package provides another Stock Package (i.e. php53-mysql provides php-mysql) but does not obsolete it. The 3rd Party Package must explicitly conflict with the Stock Package (i.e. php53-mysql conflicts with php-mysql < 5.3). Such packages will not automatically upgrade or replace Stock Packages.
So What's Stopping Us?
For repositories that explicitly only provide addon packages (EPEL) there is nothing in the way. However for repositories that provide optional replacement packages (IUS), Yum does not properly resolve dependencies when mixing the 3rd Party Repo with the Stock Distro.
Take the following example:
[root@centos5-2 ~]# rpm -qa | grep php php52-common-5.2.11-1.1.ius.el5 php52-cli-5.2.11-1.1.ius.el5 php52-5.2.11-1.1.ius.el5 [root@centos5-2 ~]# yum repolist Loaded plugins: fastestmirror, priorities Loading mirror speeds from cached hostfile * epel: mirrors.tummy.com * base: mirror.5ninesolutions.com * updates: mirror.raystedman.net * addons: www.cyberuse.com * ius: dl.iuscommunity.org * extras: www.cyberuse.com epel | 3.4 kB 00:00 base | 1.1 kB 00:00 updates | 951 B 00:00 addons | 951 B 00:00 ius | 1.9 kB 00:00 primary.sqlite.bz2 | 105 kB 00:00 extras | 1.1 kB 00:00 repo id repo name status addons CentOS-5 - Addons enabled : 0 base CentOS-5 - Base enabled : 3272 epel Extra Packages for Enterprise Linux 5 - enabled : 4469 extras CentOS-5 - Extras enabled : 288 ius IUS Community Packages for Enterprise Li enabled : 187 updates CentOS-5 - Updates enabled : 700 repolist: 8916 [root@centos5-2 ~]# yum install cacti ...output snipped... Resolving Dependencies --> Running transaction check ---> Package cacti.noarch 0:0.8.7e-1.el5 set to be updated --> Processing Dependency: php-mysql for package: cacti --> Processing Dependency: php-snmp for package: cacti --> Running transaction check ---> Package php-mysql.x86_64 0:5.1.6-23.el5 set to be updated --> Processing Dependency: php-common = 5.1.6-23.el5 for package: php-mysql --> Processing Dependency: php-pdo for package: php-mysql ---> Package php-snmp.x86_64 0:5.1.6-23.el5 set to be updated --> Running transaction check ---> Package php-common.x86_64 0:5.1.6-23.el5 set to be updated ---> Package php-pdo.x86_64 0:5.1.6-23.el5 set to be updated --> Finished Dependency Resolution Dependencies Resolved ==================================================================================================== Package Arch Version Repository Size ==================================================================================================== Installing: cacti noarch 0.8.7e-1.el5 epel 2.1 M Installing for dependencies: php-common x86_64 5.1.6-23.el5 base 154 k php-mysql x86_64 5.1.6-23.el5 base 87 k php-pdo x86_64 5.1.6-23.el5 base 64 k php-snmp x86_64 5.1.6-23.el5 base 29 k Transaction Summary ==================================================================================================== Install 5 Package(s) Update 0 Package(s) Remove 0 Package(s) Total download size: 2.5 M Is this ok [y/N]:
As you can see, this CentOS 5.2 box already has php52 installed which replace php. The IUS php52 packages provides php, but does not obsolete it. When we attempt to install cacti (requires php-mysql, php-snmp, etc) Yum does not resolve the dependency properly. Because php52 is installed, which replaced php, Yum should know that php52-mysql/php52-snmp are preferred rather than the actual php-mysql/php-snmp packages. But even though php52-mysql provides php-mysql, Yum considers the actual Stock Package a better provider (this has to do with yum compare_providers).
That said, if you explicitly add the desired packages resolution is fine:
[root@centos5-2 ~]# yum install cacti php52-mysql php52-snmp php52-pdo ...output snipped... Resolving Dependencies --> Running transaction check ---> Package cacti.noarch 0:0.8.7e-1.el5 set to be updated ---> Package php52-mysql.x86_64 0:5.2.11-2.ius.el5 set to be updated --> Processing Dependency: php-common = 5.2.11-2.ius.el5 for package: php52-mysql ---> Package php52-snmp.x86_64 0:5.2.11-2.ius.el5 set to be updated ---> Package php52-pdo.x86_64 0:5.2.11-2.ius.el5 set to be updated --> Running transaction check --> Processing Dependency: php52-common = 5.2.11-1.1.ius.el5 for package: php52 --> Processing Dependency: php52-common = 5.2.11-1.1.ius.el5 for package: php52-cli ---> Package php52-common.x86_64 0:5.2.11-2.ius.el5 set to be updated --> Running transaction check ---> Package php52-cli.x86_64 0:5.2.11-2.ius.el5 set to be updated ---> Package php52.x86_64 0:5.2.11-2.ius.el5 set to be updated --> Finished Dependency Resolution Dependencies Resolved ==================================================================================================== Package Arch Version Repository Size ==================================================================================================== Installing: cacti noarch 0.8.7e-1.el5 epel 2.1 M php52-mysql x86_64 5.2.11-2.ius.el5 ius 94 k php52-pdo x86_64 5.2.11-2.ius.el5 ius 74 k php52-snmp x86_64 5.2.11-2.ius.el5 ius 35 k Updating: php52 x86_64 5.2.11-2.ius.el5 ius 1.4 M php52-cli x86_64 5.2.11-2.ius.el5 ius 2.6 M php52-common x86_64 5.2.11-2.ius.el5 ius 246 k Transaction Summary ==================================================================================================== Install 4 Package(s) Update 3 Package(s) Remove 0 Package(s) Total download size: 6.6 M Is this ok [y/N]:
And before you ask, Priorities does not ship with RHEL and Protect Base does not matter because the names of the packages are different (php52 vs. php).
A Simple Fix For Yum Compare Providers
A simple fix to provide this functionality is to add an additional check to Yum's compare_providers which would properly provide the best match for dependency resolution in this situation. Yep, the request has been sent upstream:
To show how this situation would look should the patch in the above ticket be applied:
[root@centos5-2 ~]# cat depsolve.patch | patch /usr/lib/python2.4/site-packages/yum/depsolve.py patching file /usr/lib/python2.4/site-packages/yum/depsolve.py
Please note, this is a dev box (do not patch files directly like this).
And now it works:
[root@centos5-2 ~]# yum install cacti Loaded plugins: fastestmirror ...output snipped... Resolving Dependencies --> Running transaction check ---> Package cacti.noarch 0:0.8.7e-1.el5 set to be updated --> Processing Dependency: php-mysql for package: cacti --> Processing Dependency: php-snmp for package: cacti --> Running transaction check ---> Package php52-mysql.x86_64 0:5.2.11-2.ius.el5 set to be updated --> Processing Dependency: php52-pdo for package: php52-mysql ---> Package php52-snmp.x86_64 0:5.2.11-2.ius.el5 set to be updated --> Running transaction check ---> Package php52-pdo.x86_64 0:5.2.11-2.ius.el5 set to be updated --> Finished Dependency Resolution Dependencies Resolved ==================================================================================================== Package Arch Version Repository Size ==================================================================================================== Installing: cacti noarch 0.8.7e-1.el5 epel 2.1 M Installing for dependencies: php52-mysql x86_64 5.2.11-2.ius.el5 ius 94 k php52-pdo x86_64 5.2.11-2.ius.el5 ius 74 k php52-snmp x86_64 5.2.11-2.ius.el5 ius 35 k Transaction Summary ==================================================================================================== Install 4 Package(s) Update 0 Package(s) Remove 0 Package(s) Total download size: 2.3 M Is this ok [y/N]:
As you can see, with the additional check we can properly resolve that php52-mysql and others are the better match because their source rpm is already installed (implying that this package set is preferred). Because php52 and php conflict, it would not be possible for both php52's source rpm, and php's source rpm to both be installed (making php52-mysql and others a better match).
But does it interrupt RHEL?
Nope.
[root@centos5-2 ~]# rpm -e php52 php52-common php52-cli --nodeps [root@centos5-2 ~]# yum install php php-common php-cli ...snipped output... Installed: php.x86_64 0:5.1.6-23.2.el5_3 php-cli.x86_64 0:5.1.6-23.2.el5_3 php-common.x86_64 0:5.1.6-23.2.el5_3 Complete! [root@centos5-2 ~]# yum install cacti ...output snipped... Resolving Dependencies --> Running transaction check ---> Package cacti.noarch 0:0.8.7e-1.el5 set to be updated --> Processing Dependency: php-mysql for package: cacti --> Processing Dependency: php-snmp for package: cacti --> Running transaction check ---> Package php-mysql.x86_64 0:5.1.6-23.2.el5_3 set to be updated --> Processing Dependency: php-pdo for package: php-mysql ---> Package php-snmp.x86_64 0:5.1.6-23.2.el5_3 set to be updated --> Running transaction check ---> Package php-pdo.x86_64 0:5.1.6-23.2.el5_3 set to be updated --> Finished Dependency Resolution Dependencies Resolved ==================================================================================================== Package Arch Version Repository Size ==================================================================================================== Installing: cacti noarch 0.8.7e-1.el5 epel 2.1 M Installing for dependencies: php-mysql x86_64 5.1.6-23.2.el5_3 updates 88 k php-pdo x86_64 5.1.6-23.2.el5_3 updates 65 k php-snmp x86_64 5.1.6-23.2.el5_3 updates 29 k Transaction Summary ==================================================================================================== Install 4 Package(s) Update 0 Package(s) Remove 0 Package(s) Total size: 2.3 M Total download size: 2.3 M Is this ok [y/N]:
Such an approach gives us the ability to safely subscribe 3rd Party Repositories that do replace Stock Distro Packages, however only if you explicitly install them. Once you do, dependency resolution continues to work as it should.
Advancing Yum
With these capabilities in place we can now start to offer a standard for 3rd Party Repos that replace existing Stock Distro Packages. The idea of 'replaces' rather than 'obsoletes' comes into place. Obsoletes means a newer 3rd Party Package will automatically replace a Stock Distro Package. Provides is great for RPM, but not for Yum dependency resolution. Replaces is explicit. Nothing will install unless you explicitly replace an existing package. If you try to install it, Conflict!
[root@centos5-2 ~]# yum install php53 ... output snipped ... Resolving Dependencies --> Running transaction check ---> Package php53.x86_64 0:5.3.0-5.ius.el5 set to be updated --> Processing Dependency: php53-cli = 5.3.0-5.ius.el5 for package: php53 --> Processing Dependency: php53-common = 5.3.0-5.ius.el5 for package: php53 --> Running transaction check ---> Package php53-cli.x86_64 0:5.3.0-5.ius.el5 set to be updated ---> Package php53-common.x86_64 0:5.3.0-5.ius.el5 set to be updated --> Processing Conflict: php53 conflicts php < 5.3 --> Finished Dependency Resolution php53-5.3.0-5.ius.el5.x86_64 from ius has depsolving problems --> php53 conflicts with php Error: php53 conflicts with php
As expected, php and subpackages are installed already... so attempting to install php53 will conflict. But what if there was a '--replace-existing' flag allowing the end user to say, "You know what, I'm aware this package conflicts, but I want to replace what it conflicts with... with this package"?
[root@centos5-2 ~]# yum install php53 --replace-existing ... output snipped ... Resolving Dependencies --> Running transaction check ---> Package php53.x86_64 0:5.3.0-5.ius.el5 set to be updated --> Processing Dependency: php53-cli = 5.3.0-5.ius.el5 for package: php53 --> Processing Dependency: php53-common = 5.3.0-5.ius.el5 for package: php53 --> Running transaction check ---> Package php53-cli.x86_64 0:5.3.0-5.ius.el5 set to be updated ---> Package php53-common.x86_64 0:5.3.0-5.ius.el5 set to be updated --> Processing Replacement: php53 replaces php --> Processing WhatRequires php --> Package cacti requires php, Provided by php53 --> Finished Dependency Resolution php53-5.3.0-5.ius.el5.x86_64 from ius has depsolving problems --> php53 conflicts with php Error: php53 conflicts with php Installed: php53.x86_64 0:5.3.0-5.ius.el5.x86_64 php53-cli.x86_64 0:5.3.0-5.ius.el5.x86_64 php53-common.x86_64 0:5.3.0-5.ius.el5.x86_64 Complete!
Note: This is mocked up output to support the idea.
Making The Community Aware
SysAdmins and end users need to know if a 3rd Party Repo they are subscribing to is safe. The idea of having a 'SafeRepo Aware' stamp of approval that packagers can put on their site comes to mind. It would be quite a stretch to consider a 'certification' but 'SafeRepo Aware!' to me says, "This packager attempts to follow the standards of the SafeRepo Initiative". It gets the word about about what SafeRepo means and why the end user should care about it, and encouraging better packaging practices and standards for 3rd Party Repos.
