Open-Ended Systems Corporation
1000 East Dominguez Street
Carson, CA 90746
Phone: (310) 957-2200
Fax: (310) 329-9853
cfinley@oesc.com
I. The Porting and Migration Challenge
Introduction
Although the HP 3000 continues to remain popular with most users, some organizations may have a business need to migrate some or all of their existing applications from the MPE platform to UNIX and/or NT. It is not only possible, but in many cases it is wise and prudent to port applications from an MPE/iX computing environment to either HP-UX or NT. Applications software is increasingly becoming recognized as an extremely valuable asset and porting is one of the best ways to preserve it.
What Makes Porting Challenging
Porting an application from one computing environment to another might at first seem to be an overwhelming task. However, with the right planning and the right tools, it is often the least expensive option for obtaining needed application functionality in a new computing environment.
The terms porting, migrating, converting, reengineering and rearchitecting are sometimes incorrectly used interchangeably. This can lead to confusion. This paper is about porting in the sense that an end-user who is experienced in using the application in the source environment should be able to use the application in the target environment with little or no retraining. It also means that there should be minimal changes to source program logic. This paper is not intended to provide a exhaustive, comprehensive treatment of all aspects of the porting and application enhancement processes. Rather its goal is to provide an understanding of why porting can be challenging and to help the reader understand the key tools and techniques needed to effectively port an application from one computing environment to another. It is a intended to be a primer directed at a user who has applications software that now runs on the HP MPE operating system who is considering porting that application to either UNIX or NT.
Although most programs nowadays are written in what might at first appearance appear to be "standard" 3rd GL or 4th GL languages, most programs are not compatible across computing platforms. Differences in user interfaces, databases, systems interfaces, etc. between systems can make moving an application from one computer to another (porting) difficult. Fourth generation language based applications are less challenging to port than third generation language based applications. For that reason, the focus of this paper will be third-generation languages such as COBOL, FORTRAN or C. The fourth generation language user will find that many of the same issues mentioned in this paper pertain to their situation as well only less so. In other words, porting 4GL-based applications to UNIX and/or NT will generally speaking be easier than porting 3GL-based applications. This is particularly true when the 4GL vendor supports both the source and target environments.
We will start by discussing what factors and features make programs incompatible across platforms. Then we will show a model that can be used as a framework to investigate areas of incompatibility. Finally, we will show a list of what features need to be replicated on HP-UX and NT in order to make MPE/iX programs execute in those environments with little or no change. (For example, the additional of MPE-like operating system calls and a VPlus-like screen handler make porting can greatly simplify the amount of effort needed to port applications that use those features).
Opportunities for Incompatibility
VPlus, TurboIMAGE and MPE Intrinsics are all unique, proprietary aspects of MPE. If one took a program that uses any or all of these features and tried to compile or execute them on an HP-UX and/or NT system without any other tools or preparation, it simply would not work. There are several functions that an MPE environment provides that are not provided in exactly the same fashion in any given target environment. For example, VPlus screen handling and TurboIMAGE databases are unique features of the MPE environment. The first goal of porting is to first identify what features are provided to programs, programmers and computer users in the source MPE environment. The next step is to somehow make the source and target versions of the application sources behave the same from an end-user and an executing program point of view. One way to do this is to find either exact or functionally compatible replacement functions in the target environment. A second way to make the source and target versions compatible is to translate from one to the other. For example, if the application is written in BASIC and there is no compatible version of BASIC on the target platform, the BASIC code can be translated into a language that is available such as "C".
No matter what approach one adopts to make the source and target environments compatible, the first step is to analyze the source environment in order to determine needed functionality. Dividing the functions provided by an MPE environment into sub-groups makes analysis easier. The next section will start by defining an ideal target environment model, one that supports both ported and native applications on the same host. For example. If our goal is to make our existing MPE application run on HP-UX, our target computing environment would allow us to run both our MPE application and native HP-UX applications on the same host computer. Next in the same section we will subdivide the functionality usually encountered in an MPE/iX environment into sub-environments. This will serve as an outline for a specification - that will be used to identify required functionality in the target computing environment. Our goal is, in essence, create a checklist of functions needed in UNIX and/or NT that will allow our originally MPE-based applications to run on the target environment with little or no modification.
Once the ideal target computing environment has been described in this section, Section I, the porting and migration process examined. Section II, Overview of the Porting and Migration Process, describes in general terms what is involved in porting and migrating applications. Finally, in Section III, Third-Party Porting Tools , the models developed in Section I will be used as a framework to describe some real-world tools that are available to assist in the porting of HP MPE applications to UNIX and/or NT.
Overview of Requirements for Multiple Portable Architectures (MPA)
In porting an application from MPE to another environment what we are really trying to do is create a portable MPE-like environment that allows us to run the MPE application with little or no alteration. Ideally, our target computing environment also allows us to run native applications (applications written with either UNIX or NT in mind) on the same computer. Figure 1 illustrates this point. More succinctly, Our goal is to be able to have an environment wherein we can run both MPE and either HP-UX or NT applications on the same computer. We call this MPE-like environment Multiple Portable Architectures or MPA for short.
In order to gain a better understanding of what is required create such an environment such as that in Figure 1 we start by looking at the MPE functionality in manner suggested by Figure 2. In essence, what this shows is that, from a functional point of view, the MPE environment can be divided into four sub environments: The four sub-environments are shown in Figure 2. They are:
The execution environment is the computer system from the point of view of the executing program. The user or operations environment represents the pieces of the operating system and environment that the programmer and/or computer user uses to interact with the computer system. The development environment represents the compilers, linkers and debuggers that the programmer uses to get programs ready for execution. Finally, the administration environment represents those programs used by administrators to perform operations and administration tasks.
From an application porting point of view, the most interesting of environments are the execution environment and the user (operations) environment.
Execution (Runtime) Sub-Environment
The most important sub-environment is called the runtime, or execution environment. This environment is made up of all of the programs that are used when the program is actually doing useful work, that is, executing.
This sub-environment consists of: the runtime User Interface (screens), Operating System, Interfaces, Data Management and Transaction Management
This sub-environment must be extremely compatible with MPE/iX because its goal is to allow most programs ported from an MPE/iX environment to run on MPA with little or no alteration. It consists of an MPE-like systems intrinsic library and compiler libraries. Here our goal is to provide a rich set of supported MPE/iX system functions that cover most of documented MPE/iX intrinsics.
The User (Operations) Sub-Environment
The second sub-environment is the User (Operations) sub-environment. The programs in this sub-environment are used before, during and after program execution to either cause the programs to enter the execution environment (run or execute), stop a program, alter its behavior with human intervention or to do something with its input or output. That is it consists of the programs that a computer user (end-user, operator or systems manager) uses to actually operate the computer. This sub-environment consists of: Files and Data Commands and Operations and Control programs such as Utilities, Editors and Mail.
This sub-environment does not need to be as compatible with MPE/iX as the execution sub-environment. In fact, there is an argument for having users make use of native features of the operating system on which the application happens to be executing as opposed to creating an artificial MPE like environment for them. Therefore, the MPA philosophy is to use the target operating system features when available. This means that MPE features implemented in the target user sub-environment are to be implemented only if they support the porting of programs without alteration or if a functional equivalent does not exist on the target operating system. However, if a feature is needed only for the purpose of avoiding user training, implementing them is of low priority and is not likely to be implemented.
The Development Sub-Environment
Software in the Development Sub-environment is used to create programs and prepare them for execution. This sub-environment consists of software used for Editing, 3rd Generation Languages, Linking Programs and 4th Generation Programs.
The Administration Sub-Environment
The Administration Sub-environment covers many areas such as security, high availability, software updates and distribution and managing the network, and backup software and strategies.
Administrative software is generally not expected to play a major role in portability of an application. However, it does affect the ease of application portability and the ability for the user to support the same software application on multiple operating system type. If, for example, the same backup software is available on MPE, UNIX and NT, employee training becomes easier for the user organization.
General Porting Design Objectives
For all sub-environments of the MPA environment the following are our objectives:
II. Overview of the Porting and Migration Process
Introduction
Differences in the software development culture, tools, and the API can make porting and migration a difficult transition for both programs and programmers. The systems offer very different services, organized around different design philosophies. Additionally, as a separate issue, legacy code is at times outdated in terms of modern coding standards. Older MPE applications may have processor-specific calls, or dependencies on MPE system services or library routines. One of the goals of migration approach suggested in this paper is to create an environment in which both MPE-based applications and HP-UX applications can coexist. Figure 1 in section I illustrates this principal. Under this scenario, the ported application can then be maintained and enhanced as either an MPE application or a UNIX or NT application.
This section describes the overall porting process in terms of products and services that can be used to first facilitate the port and to help operate in the new environment.
Overview of Application Porting and Migration
This section discusses the various options for porting applications to the HP UNIX and/or NT platform.
Every organization has its own specific issues and problems concerning migrating code from one platform to another. In general, however, migrating, or converting an application that runs on MPE to the HP UNIX and/or NT platform can be accomplished using the following methods:
One method to convert a program from MPE to UNIX and/or NT is to rewrite MPE specific code and recompile the resultant source code using a native UNIX and/or NT compiler. This method produces a native load module that takes full advantage of the HP UNIX and/or NT environments. If such an application is converted this way to HP-UX initially it will port relatively easily to other UNIX systems platforms and provide you with the full performance that the HP UNIX system offers. An NT port performed this way may also create some performance advantage over other methods.
Rewriting code can be a major undertaking, depending on the processor-specific assumptions or outdated language constructs. If an MPE application is written to be strictly POSIX or XPG4 compliant, then it should be possible to port it to HP UNIX and/or NT or other platforms relatively easily. The problem is that a majority of legacy code was written before industry programming standards were available and most MPE developers do not develop in the POSIX domain.
HP provides the same basic compilers for C and Fortran, for both HP-UX and MPE. In fact, we are told that the same compilers are used in both environments. After moving code from MPE to HP UNIX and compiling the code, the resulting compiler error messages can point out where many of the portability problems may lie. Under HP UNIX, and especially under MPE, porting in-house Fortran and C applications from HP 3000 to the HP 9000 is often a straightforward recompile. After moving the code to HP UNIX, the programmer can compile the program, then examine compiler messages to isolate specific problems. Although C and FORTRAN compilers do exists on NT, they are not as similar as the HP-UX to MPE compilers. Porting therefore is more difficult.
HP provides COBOL II, PASCAL, BASIC, Business BASIC and RPG compilers for MPE. For COBOL, HP makes available Micro Focus COBOL. This is available for both HP-UX and NT. Third-party solutions exist for the other languages as well. Porting applications written in those languages are not as simple as the C and FORTRAN ports, however, and tools may help take the drudgery out of such an effort. This is discussed below in Tools Based Migration.
The two most common 4th generation languages in use on the HP 3000 are Speedware and Powerhouse. Both of these firms offer versions that run on HP-UX and NT and this makes porting of the application relatively easy.
Moreover, both of these vendors offer a migration path to client/server. Cognos has a product called Axiant that is used for migrating Powerhouse applications and Speedware has a product called Visual Speedware.
Although HP 3000 and HP 9000 share a common architecture, the Intel architecture computers such as the HP Netserver have different data organization, addressing schemes, and other differences. For example, byte ordering is reversed on Intel architecture computers (little endian), this may result in differences in program behavior on the Intel computer that would not occur on the HP-UX based 9000.
HP and number of other companies offer professional services that help organizations move from MPE to HP UNIX and/or NT. Among these companies is OESC. These companies provide specialized migration skills and tools. They can provide full migration services, assessment, and customized training.
Porting tools fall into two principal categories, translators and emulators. In cases where the application's source language is not supported on HP UNIX and/or NT, or where you wish to move your application from a proprietary programming language to an open language such as C, you may want to consider translating the application code. Also, in some cases the easiest way to convert software with proprietary extensions for use on another platform is by translation. Emulators are used to simulate such functions as system interfaces and command languages. Section III presents details on migration tools available for use in MPE porting.
Introduction
Very few third-party companies provide tools for porting applications from MPE to HP UNIX and/or NT. This section discusses the tools available from OESC and its partners that allow us to implement the MPA environment show in Figure 1 in Section I.In some cases it may be possible to move a few MPE applications to an HP UNIX and/or NT system by merely recompiling and re-linking the application. This allows you to maintain a single pool of source code to run on both operating systems. Although there have been other vendors in past that have, the Only third-party vendor that now provides software that emulates the MPE run-time library and system services routines is OESC. Section 3 provides more information on these emulation products.
You can use this emulation software to speed the port of your application because you will have far less code to rewrite than you would have in a native port. Compared to performing a native port, using emulation software to speed porting an application also lowers the cost of porting that application.
One disadvantage of using emulation software instead of performing a native port is that the application's performance might suffer in comparison to a native HP UNIX and/or NT application.
Some installations prefer to treat porting their applications with the emulation software as an intermediate step. Once an application is ported using the emulation software, you can begin a full native port of the application. This strategy gives your application users relatively quick access to your application on their HP UNIX and/or NT system. If you perform a native port later, that port provides a version of the application that takes full advantage of the speed of the HP UNIX and/or NT system.
Description of a Porting Toolset
Appendix A expands the requirements identified in the above section and provides examples of tools that meet those requirements. This section provides an explanation of what features exists in those categories.
The four sub-environments are:
The execution environment is the computer system from the point of view of the executing program. The user or operations environment represents the pieces of the operating system and environment that the programmer and/or computer user uses to interact with the computer system. The development environment represents the compilers, linkers and debuggers that the programmer uses to get programs ready for execution. Finally, the administration environment represents those programs used by administrators to perform operations and administration tasks.
The Execution Sub-Environment
The most important sub-environment is called the runtime, or execution environment. This environment is made up of all of the programs that are used when the program is actually doing useful work, that is, executing.
This sub-environment must be extremely compatible with MPE/iX because its goal is to allow most programs ported from an MPE/iX environment to run on MPA with little or no alteration. It consists of an MPE like systems intrinsic library and compiler libraries. Here our goal is to provide a rich set of supported MPE/iX system functions that cover most of documented MPE/iX intrinsics.
This sub-environment, consists of:
The runtime User Interface (screens)
Operating System Interfaces
Data Management
Transaction Management
The User (Operations) Sub-Environment
The second sub-environment is the User (Operations) sub-environment. The programs in this sub-environment are used before, during and after program execution to either cause the programs to enter the execution environment (run or execute), stop a program, alter it’s behavior with human intervention or to do something with its input or output. That is, it consists of the programs that a computer user (end-user, operator or systems manager) uses to actually operate the computer.
This sub-environment does not need to be as compatible with MPE/iX as the execution sub-environment. In fact, there is an argument for having users make use of native features of the operating system on which the application happens to be executing as opposed to creating an artificial MPE like environment for them. Therefore, the MPA philosophy is to use the target operating system features when available. This means that MPE features implemented in the target user sub-environment are to be implemented if they support the porting of programs without alteration or if a functional equivalent does not exist on the target operating system. However, if a feature is needed only for the purpose of avoiding user training, implementing them is of low priority and is not likely to be implemented.
The User (Operations) environment, for example MPA commands do not provide an exact emulation of the MPE Command Interpreter. It is based on UNIX implementation of the POSIX shell, which is "instructed" how to simulate MPE CI shell. Only a basic set of MPE CI commands is supplied, those which are essential for MPE trained user, more or less often used from MPE programs through COMMAND() and HPCICOMMAND() intrinsics, or which perform some operation on easier way than existing UNIX commands. Some others have been added mostly just as one more shell scripts. Most of MPA-implemented MPE commands are POSIX shell or awk scripts. They create an MPE command-like result using UNIX primitives. In other cases, where no function is provided by MPA, virtually any task of missing MPE commands can be performed using corresponding UNIX commands and utilities.
This environment consists of:
Files and Data
Commands
Operations Control
Utilities
The Development Sub-Environment
Software in the Development Sub-environment is used to create programs and prepare them for execution.
This sub-environment consists of:
Editing
3rd GL Compiling
4GL Development
Linking Programs
The Administration Sub-Environment
The Administration Sub-environment covers many areas such as security, high availability, software updates and distribution and managing the network, and backup software and strategies.
Administrative software is generally not expected to play a major role in portability of an application. However, it does affect the ease of application portability and the ability for the user to support the same software application on multiple operating system type. If, for example, the same backup software is available on MPE, UNIX and NT, employee training becomes easier for the user organization.
Backup
Security
Appendix A:
Detailed Description of Porting Tool Requirements and Examples
Source Environment |
Solution |
Product Source |
||||
The Execution Sub-Environment |
||||||
The runtime User Interface (screens) |
||||||
VPlus to Character Terminal |
Wingspan, Terminal Emulator and Blockmode Driver |
Denkart/Wagner |
||||
VPlus to GUI |
NewFACE FRONTMAN |
HP-Germany Minisoft |
||||
Proprietary Screen Library |
Wingspan and Terminal Emulator and Blockmode Driver |
Denkart/Wagner |
||||
Direct I/O To Screens |
Screen Capture Process Wingspan and Terminal Emulator and Blockmode Driver |
OESC/Denkart |
The Execution Sub-Environment (Continued) |
Source Environment |
Solution |
Product Source |
|||||||
Operating System Interfaces |
||||||||||
The File System |
MPE_FS |
MPA/RX, MPA_FS |
Denkart/Marxmeier |
|||||||
System Information |
MPE_SINFO |
MPA/RX, MPA_SINFO |
Denkart/Marxmeier |
|||||||
Process Management |
MPE_PM |
MPA/RX, MPA_PM |
Denkart/Marxmeier |
|||||||
User Information |
MPE_UINFO |
MPA/RX, MPA_UINFO |
Denkart/Marxmeier |
|||||||
Program Information |
MPE_PINFO |
MPA/RX, MPA_PINFO |
Denkart/Marxmeier |
|||||||
Interprocess Communication |
MPE_MSGFILE |
MPA/RX, MPA_MSGFILE |
Denkart/Marxmeier |
|||||||
Process Synchronization |
MPE_RINS |
MPA/RX, MPA_RINS |
Denkart/Marxmeier |
|||||||
Miscellaneous OS Interfaces |
MPE_MISC |
MPA/RX, MPA_MISC |
Denkart/Marxmeier |
|||||||
Data Management |
||||||||||
MPE Flat Files |
MPE Flat Files |
Through intrinsics |
Denkart |
|||||||
KSAM |
KSAM |
E-ISAM |
Marxmeier and OESC |
|||||||
IMAGE Clone |
IMAGE Clone |
HP-Eloquence |
HP Germany |
|||||||
SQL Access |
SQL Access |
HP-Eloquence |
Marxmeier |
|||||||
Transaction Management |
||||||||||
IMAGE Transaction Management |
MPE Transaction Manager |
HP-Eloquence |
Marxmeier |
The User (Operations) Sub-Environment |
Source Environment |
Solution |
Product Source |
|||
Files and Data |
||||||
MPE File Usage |
MPE File Usage |
Emulator/Intrinsics |
Denkart/Allegro |
|||
Directories |
Directories |
Unix Directories |
Denkart/Allegro |
|||
Tape Files |
Tape Files |
Emulator/Intrinsics |
Denkart/Allegro |
|||
Commands |
||||||
Commands |
Commands |
M-Shell, K-Shell |
Denkart/Allegro |
|||
Command Files |
Command Files |
M-Shell, K-Shell |
Denkart/Allegro |
|||
UDC's |
UDC's |
Unix Shell K, C, Bourne |
Denkart/Allegro |
|||
Jobs and Job Files |
Jobs and Job Files |
Unix Shell K, C, Bourne |
Denkart/Allegro |
|||
Operations Control |
||||||
Production Scheduling |
Production Scheduling |
Maestro |
Unison/IBM |
|||
Output Spooling |
Output Spooling |
Spoolmate |
Unison/IBM |
|||
Utilities |
||||||
Sort/Merge |
Sort/Merge |
Sort/Merge |
HP |
|||
Editors |
Editior |
VI and Qedit |
UNIX, Robelle |
|||
|
n/a |
n/a |
||||
Source Environment |
Solution |
Product Source |
|||
The Development Sub-Environment |
|||||
Editing |
|||||
Editing |
See User Sub-Environment |
||||
3rd GL Compiling |
|||||
FORTRAN |
FORTRAN/iX |
FORTRAN/UX or C++ |
HP or Denkart Translator |
||
ANSI/C |
C/iX |
C/ANSI |
HP |
||
COBOL |
COBOL II/iX |
Microfocus COBOL |
Microfocus |
||
PASCAL |
PASCAL/iX |
PASCAL |
Various |
||
BASIC |
BASIC |
C |
Denkart Translator |
||
4GL Development |
|||||
Speedware |
Speedware |
Speedware. |
Speedware |
||
Powerhouse |
Speedware |
Powerhouse |
Cognos |
||
Linking Programs |
MPE Linker |
HP-UX or NT Linker |
HP |
||
The Administration Sub-Environment |
|||||
Backup |
Turbostore |
Tar, HiBack |
HP, HiComp |
||
Security |
Security 3000 |
Security UX |
VeSoft |