| How to Run Binary-Only Application Packages on Various Versions of Linux 
Posted: 3 Mar 2005
The purpose of this document is to
provide a migration path for users who have invested in proprietary application
packages that were certified for specific versions of various basic libraries
(such as glibc) but who now intend to run them on different versions.
The
reader of this document should be able to install and configure the files
required for such a task. (The document will also provide several hints.) All
examples are based on x86 hardware; other Linux architecture changes may require
changes to both to the example code and the recipes.
This document is the
result of using the described steps in a real project.
A Linux program usually consists of
machine code to perform a specific set of functions. Normally, some of these
functions are provided by libraries. A library can exist as a static version
included in an archive, or it can exist as a dynamic, shared object. When a
program is created, the developer decides whether static or shared libraries
will be used. There are only a few instances in which a static library should be
used; such instances include binary incompatibilities between different versions
of a library or evolving changes in the infrastructure of such a
library.
A typical example of the latter is what occurred while the
programming language C++ matured but still wasn't commonly standardized.
Implementation details such as how virtual functions in derived inheritance
lattices are addressed resulted in different, incompatible versions of the same
library, depending on the version of the compiler that was used to create the
library.
A shared library has several benefits: its machine code is
loaded only once into the computer's memory, it is shared among several programs
using this library and only the memory for program-specific data needs to be
allocated. Another benefit is that a system administrator can easily upgrade a
shared library without having to touch any programs using that library; this is
especially important for security fixes.
Of course, none of this is
Linux-specific; all of these descriptions also apply to other modern operating
systems that support shared libraries.
Remember when you wanted to execute a
newly downloaded program on your brand-new, leading-edge Linux system only to
find that it didn't work? Most often this happened during the transition phase
from libc4 to libc5 or from libc5 to libc6--the latter of which is also known as
glibc (the GNU C library).
The C library is probably the most important
library on every UNIX/Linux system because it acts as the interface between each
application and the kernel. Changes in such a central library can easily render
a system useless if such changes are not backward compatible.
There are
other scenarios as well: a program may have been built on a system that was
using newer versions of libraries than you have installed on your system. It is
likely that such programs will not even start or work properly on your system.
This happens if the program is actively using the new or changed features in the
updated versions of the libraries.
It is common sense that versions of a
library with functionality not included in former versions (and that carry a
version number indicating that programs link to an older version) cannot be
loaded and executed. This version number is called the major revision of a
shared library. The following diagram explains this by using glibc as an
example:
| GLIBC Version |
Filename |
| libc5 |
/lib/libc.so.5 |
| libc6 |
/lib/libc.so.6 |
To maintain certain minor
revisions of a shared library, the filename can be augmented by additional
suffixes such as libc.so.6.3.3.
As described in the "Background" section,
there are additional complications caused by binary incompatibilities of C++
libraries compiled with GNU C++ compilers prior to version 3.3. Even with a
currently standardized environment including consistent Application Programming
Interface (API) and Application Binary Interface (ABI) definitions, it is not
yet clear if future versions of C++ compilers and libraries will remain
completely compatible. To address such issues, the Linux Standards Base (LSB)
recommends that every software vendor creating software written in C++
statically link the C++ parts. Unfortunately, very few vendors actually have
followed this recommendation. To make things worse, a Linux distributor created
its own version of a C++ compiler but failed to ensure that its shared libraries
carried different version numbers than those used for the official C++
compiler.
In short, if you want to start a program and it fails to do so
(and emits error messages similar to the following), your problem can likely be
solved by following the suggestions described in "The Solution" section of this
white paper.
Error Message When Starting a C
Program
./a.out: relocation error: ./a.out: symbol errno, version
GLIBC_2.0 not defined in file libc.so.6 with link time reference
If
you happen to see the above error message about the symbol errno, your program
has been linked with a version of glibc older than 2.3. Newer versions of glibc
no longer provide errno as a global variable to allow its thread-safe usage. In
this situation, you would have to set up a compatibility environment with an
older version of glibc, such as version 2.2.5.
If, however, you don't see
any error message at all and the program starts but doesn't work as you expect
it to, you might be experiencing the C++ problem described above. The runtime
linker, which is loading your program into memory and resolving any dependencies
with shared libraries, is able to locate the required shared libraries on your
system and to load them into memory. The shared libraries may not, however, fit
with the actual program.
The first thing to do in
identifying problem sources is to run the command ldd on your programs. The
purpose of this program is to print the shared libraries required by each
program or the shared library specified on the command line. Running ldd on
/bin/bash will show output similar to the following:
| ldd /bin/bash |
|
linux-gate.so.1 => (0xffffe000) |
|
libreadline.so.4 => /lib/libreadline.so.4 (0x40036000) |
|
libhistory.so.4 => /lib/libhistory.so.4 (0x40062000) |
|
libncurses.so.5 => /lib/libncurses.so.5 (0x40069000) |
|
libdl.so.2 => /lib/libdl.so.2 (0x400af000) |
|
libc.so.6 => /lib/tls/libc.so.6 (0x400b2000) |
|
/lib/ld-linux.so.2 => /lib/ld-linux.so.2
(0x40000000) |
If a shared library cannot be found,
an error message will be shown:
libfoo.so.1 => not
found
This error message indicates that the program you are trying to
execute actually needs the shared library libfoo.so.1 but cannot find it in the
predefined locations that shared libraries look for. These predefined locations
normally contain the directories /usr/lib and /lib. In addition, a system
administrator can add additional directories to a file /etc/ld.so.conf to
instruct the runtime linker where to search for shared libraries. As a last
resort, an environment variable LD_LIBRARY_PATH can be defined to list
additional directories. On Linux, this variable will not be used for programs
that run under Ids and that do not belong to the user.
After you have identified the source of
the problem, you can set up a compatibility environment to provide the required
functionality to execute your program properly. We strongly advise installing
additional libraries and other files outside of the standard system directories
/lib and /usr/lib, as shown in the following example:
| Program name to be migrated |
zoo |
| Prefix ($prefix) directory location to install the compatibility
files |
/opt/compat-env/zoo |
| List of required libraries |
glibc-2.3.2-95.27 |
| libgcc-3.2.3-42 |
libstdc++-3.2.3-42 |
| Source ($srcdir) directory where required files are located
initially |
/root/zoo-src |
Step
1
Create directories /bin and /lib under the prefix
directory:
% prefix=/opt/compat-env/zoo
% mkdir -p $prefix/bin $prefix/lib
Step
2
Place copies of the required files into their respective
locations:
% srcdir=/root/zoo-src
% cd $prefix/bin
% cp -p $srcdir/zoo .
% cd $prefix/lib
% rpm2cpio $srcdir/glibc-2.3.2-95.27*.rpm | cpio -idvm '*libc*.so*'
% find . -name '*libc*.so*' -exec mv -v '{}' . ;
% rpm2cpio $srcdir/glibc-2.3.2-95.27*.rpm | cpio -idvm '*ld*.so*'
% find . -name '*ld*.so*' -exec mv -v '{}' . ;
% rpm2cpio $srcdir/libgcc-3.2.3-42*.rpm | cpio -idvm '*libgcc_so*.so*'
% find . -name '*libgcc_so*.so*' -exec mv -v '{}' . ;
% rpm2cpio $srcdir/libstdc++-3.2.3-42*.rpm | cpio -idvm '*libstdc*.so*'
% find . -name '*libstdc*.so*' -exec mv -v '{}' . ;
% rm -rf lib usr
Step 3
Create a wrapper script to
set up the appropriate environment variables and start the program:
% cd $prefix/bin
% mv zoo zoo.exec
% cat > zoo << EOF
#! /bin/bash
prefix=/opt/compat-env/zoo
LD_LIBRARY_PATH="$prefix/lib${LD_LIBRARY_PATH+:$}"
export LD_LIBRARY_PATH
$prefix/lib/ld-linux.so.2 $.exec "$@"
EOF
% chmod 755 zoo
Now the program zoo can be executed, and the
versions of glibc, libgcc and libstdc++ will be used as per the
requirement.
After you have set up a
compatibility environment as described in the previous section, it is still
possible that the program cannot be executed. In this instance, an an error
messages such as the following is printed:
symbol __libc_wait, version
GLIBC_2.0 not defined in file libc.so.6 with link time
reference
Newer versions of the GNU C library assign version numbers
not only to the library itself as a whole but also to individual symbols. This
usually happens to conform with certain language standards. In the example, the
global symbol __libc_wait with version number GLIBC_2.0 is not defined anymore.
This could be solved by using an older version of the C library or by using
another feature of glibc and its runtime linker. It is possible to preload
shared objects to override symbols defined in shared objects that would be
loaded in the course of the runtime linker's normal dependency resolution. This
can also be used to define symbols that are otherwise not defined, such as the
__libc_wait function in the example.
The proper fix for this problem
would be to create a small shared object containing the required implementation
and to load it in the wrapper script:
% cd $prefix/lib
% cat > libcwait.c << EOF
#include
#include
#include
#include
pid_t __libc_wait (int *status)
{
int res;
asm volatile ("pushl %%ebx
"
"movl %2, %%ebx
"
"movl %1, %%eax
"
"int $0x80
"
"popl %%ebx"
: "=a" (res)
: "i" (__NR_wait4), "0" (WAIT_ANY), "c" (status), "d" (0),
"S" (0));
return res;
}
EOF
% gcc -fpic -O2 -shared libcwait.c -o libcwait.so
Now make the
required modifications to the wrapper script:
% cd $prefix/bin
% cat > zoo << EOF
#! /bin/bash
prefix=/opt/compat-env/zooetc.
LD_LIBRARY_PATH="$prefix/lib${LD_LIBRARY_PATH+:$}"
export LD_LIBRARY_PATH
LD_PRELOAD=$prefix/lib/libcwait.so
export LD_PRELOAD
$prefix/lib/ld-linux.so.2 $.exec "$@"
EOF
% chmod 755 zoo
Using the suggestions outlined in this
white paper, you can create a compatible environment to execute programs that
your vendor has not yet ported to or built on a specific version of a Linux
operating system. For example, you can easily set up a compatible environment to
run programs from a Red Hat Enterprise Linux* system on a SUSE LINUX Enterprise
Server and vice-versa.
Additional information can be found in the GNU C
library documentation, which should be included with your particular Linux
distribution, and at the LSB
Web site.
|
