Package Manager (in 3.1)

 

Overview

The goal of this functionality is to enable a set of functionally related updates that are not part of the xTuple core application to be loaded and updated together as a single cohesive package. Elements of the package may include screens, scripts, reports, and other functionality required to support a specific functional module.

Functional Requirements

All modular functionality should be database driven so that it may be loaded following the same basic principles used in xTuple Updater utility version 1.1.0. The Updater can currently load tables, functions, triggers, views and records in the database. The notion of the "Package Manager" is really simply the xTupleERP interface and Updater modified together to support the following logic:

  • A concept of a functional module set
    • Includes a Name, Description, Developer and Version
  • The module set may include the following elements
    • Database Objects
      • Schemas
      • Tables
      • Views
      • Functions
      • Triggers
    • System Records
      • Reports
      • Screens
      • Scripts
      • Privileges
      • Menus
    • Dependencies
  • Tables will be included to keep a record of all database objects.
  • System records will include additional keys that reference the package they belong to.
  • Users should be able to delete a package.
  • The same package file methodology can be used to create installation and update files.

New Terms and Definitions

    Package

    • Formerly an upgrade bundle for xTupleERP, now may be a modular extension of xTupleERP

    System Tables

    • Tables that are shared by core xTupleERP and functionality used by modules, such as report, where rows are identified as belonging to the core or a package.

Related Existing Functionality

The current version 1.1.0 of the xTuple Update utility already handles most of what is required for this functionality. Specifically, it runs SQL scripts to insert and update objects and records in the database and imports report XML files as report records. It also has the capability to run developer defined pre-checks to look for conflicts before running an update. If any are found, it will display a message and not allow the end user to proceed until the conflicts are resolved.

Similar and Related Requests

This is being specifically developed for the first "Third Party" package being created for xTupleERP which is the CashRegister module for Highlander. It will allow the various screens and database objects to be loaded and managed as a unit.

User-Level Functionality

UI files used for mock ups pictured below may be found attached to this page.

Updater

Module installation packages would be nearly identical in format to xTuple core update packages that exist today. To install a module, users simply launch the updater, select the module package and run it. The Updater would be modified to process additional files including .UI (screen) files into the uiforms table and .JS (script) files in scripts table the same way it imports report definitions today. There should be a facility added in that allows these imported files to be associated with the package they belong to.

An upgrade package and an installation package for a module will be identical. An installation package will simply be a series of all the upgrades from the initial release to the current.

Window Changes

List Packages

Under the System > Design menu a new listing will be added called "Packages."

packages1.png

Packages will list all the module extensions that have been installed on the xTuple database. Buttons on the right side will do the following:

  • New: Launches the Updater. Users will select and run a package much the same way they would today for an upgrade.

  • View: Launches a screen, shown below, where users can see package details.

  • Delete: Allows users to delete a package. User should be warned with a critical message, then asked by a second warning/question if they have backed up their database first.

Package

The package screens pictured below will be view only and simply display information about the package for auditing/troubleshooting purposes.

package1.png

The above screen shows information stored on the package header table.

package2.png

The above screen shows database object information stored on the package item table.

package3.png

The above screen shows database system record information obtained by joins and unions to report, priv, cmd, uiforms and script tables. Optionally, it may be useful to add a package name column on other displays and reports that show these records so they can be easily differentiated from "core" functionality.

package4.png

The above screen lists other packages on which this package is dependent. Parent packages may not be deleted if child dependencies exist.

Tamper Prevention

Support for packages may be exceedingly difficult if users attempt to modify package contents. If they do so and break the functionality it will not be readily apparent that the modules were tampered with to the developer supporting the package. To reduce this problem, we should prevent users from modifying system records. Triggers should be added to the report, commands, uiforms and scripts tables that prevent users from updating or deleting them. The xTuple Updater should be modified to disable these triggers before running scripts, and re-enable them afterward. The report designer should not allow a user to save a report to the database that exists with the same grade and name as another and has a key pointing to a parent package. This will force users to save a modified report as another grade. Screens, scripts and commands that are linked to a package should be view only, but users should be able to copy them so they may make their own modified versions if necessary. This paradigm is similar to the financial reporting engine that has "System" reports that can not be modified, but can be copied as a baseline for a new user report.

It will still be possible for technical users to modify database objects or even disable the tamper prevention trigger on the tables to allow changing of system records. The idea here, however, is not to make it impossible for users to change the system, but to prevent less technical users using the GUI tools from accidentally causing problems for themselves that are difficult to trace and resolve for the supporting developer.

Usability Considerations

The package manager will be easy for end users to run, but challenging for developers to make packages for unless they have detailed technical knowledge. We have an internal guide that describes how to build update packages. We can use portions of this to publish a new general use guide for developers to package modules.

Some framework exists in Updater for a GUI package builder utility. That would be extremely helpful for the xTuple community. However, since it is not absolutely essential at this time, it will be deferred until a later release.

Problems and Alternatives

Developer Responsibilities

It will be up to the developer creating a package to make sure appropriate records are created for each package item that references a new database object. There might be a way to automate this at some point to reduce error, particularly if the graphical package builder was completed.

It is possible that modules could conflict with existing functionality by creating objects or records that have already been created by another module. A thorough developer could run pre-checks to make sure none of the objects or records already exists. This adds significant work to the developer that is likely to be overlooked. Again, a GUI package builder could reduce this problem. Also, we should encourage third parties to create schemas for themselves to reduce the likelihood of overlap and conflict among database objects.

Pre-checks for compatible xTuple versions and pre-requisite packages on which a given package is dependent should be added by the developer. However, keeping track of dependencies on core functionality or other packages may be problematic for developers as these changes over time. In fact, the methodology of aggregating updates as installation packages may be untenable in the long run if xTuple core technology changes significantly. At some point it will probably be useful, if not necessary, to create an export function on the Package List window that generates a "clean" installation package based on the installed contents of an existing package. This would be similar in convenience to the practice today of backing up a database to create a new "install" database.

User Responsibilities

If an end user was to add a new grade of report based on package generated report, it would no longer be associated with the package. This may cause confusion if the package is deleted, because the user's report will remain even though the facility and data required to run it no longer exist. However this behavior is desirable because unlike standard package reports, there would be no way to reproduce the user's report if it was deleted. Once a user modifies a report, the modified version is their responsibility.

End User License Agreement (EULA)

It is standard practice when installing or updating software to present a user with screen showing a EULA that they must agree to before proceeding. It may be necessary to include the ability for the xTuple Updater to be able to do this. The topic warrants further discussion.

Upgrade Compatibility

As mentioned above, situations may arise where a parent package is updated in such a way that child package functionality no longer works. How can we mitigate this problem when the developer of the parent package may know nothing about child packages that depend on it? One solution may be to host a web service that maintains a compatibility matrix between packages. The Updater could check for compatibility problems before updating a parent package and warn the user that incompatible packages will be disabled until they are updated to compatible versions. If no compatible versions are available, users would be advised to wait on the parent upgrade until they are.

It would be up to developers of packages to maintain the compatibility references for their software. If they do not do so, the Updater would assume child packages are incompatible and default to disabling them.

This solution would require a substantial amount of infrastructure work that is beyond the scope of this specification, but should be kept in mind as a possibility later as packages and versions proliferate.

Internal Design

Basic Algorithms

Delete Package

Look for packages dependent on this package
If any are found
  Return Error Message

Select Package Items for Type View
For each View
  If no other packages depend on this view
    Drop the View

...Repeat for Functions, Triggers, Tables and Schema...

Delete Package Header

Custom Widget Changes

Schema Changes

The following two tables will be added:

Package Header - pkghead

Column

Type

Description

Contsraints

pkghead_id

serial

Id

Primary Key

pkghead_name

text

Name of Package

Unique, Not Null

pkghead_descrip

text

Description

 

pkghead_version

text

Version number

Not Null

pkghead_developer

text

Developer

Not Null

pkghead_notes

text

Notes

 

Package Item - pkgitem

Column

Type

Description

Contsraints

pkgitem_id

serial

Id

Primary Key

pkgitem_pkghead_id

integer

Package Head Id

Foreign Key Cascade Delete, Not Null

pkgitem_type

bpchar

Type

Not Null, Check S = Schema, T = Table, V= View, R = Trigger, F = Function

pkgitem_name

text

DB object name

Not Null, Type and Name together should be unique

pkgitem_descrip

text

Description

 

Package Dependencies - pkgdep

Column

Type

Description

Constraints

pkgdep_id

serial

Id

PrimaryKey

pkgdep_pkghead_id

integer

Package Head Id

Foreign Key, Not Null

pkgdep_parent_pkghead_id

integer

Parent Package Head Id

Foreign Key, Not Null

Cascade Delete System Records

System records associated with a package will be deleted automatically when a package is deleted. This will be accomplished with foreign keys. A new column with a cascade delete foreign key relationship to the pkghead_id column on the pkghead table will be added to the following system tables:

  • cmd
  • report
  • priv
  • uiform
  • script

Likewise cascade delete foreign keys should be added between cmd and cmd arg, and usr and usrpriv to ensure all child records are removed when the package is removed.

Privileges for feature

Name

Description

MaintainPackages

Can Add/DeletePackages

ViewPackages

Can View Packages

Stored Procedure Changes

A helper function getPackageId([PackagName]) should be added that returns the primary key of a package record based on a name passed. This will allow the creation of insert statements for package item records and system records with foreign keys that refer back to the package header.

Triggers would be added to system records to prevent alteration of package generated records.

Error Handling

See Problems and Alternatives

QA Considerations

In addition to unit testing, functionality developed for the CashRegister module should be used as a prototype package.

Once we start down this path, we should in practice test core xTupleERP functionality with no packages installed. Packages should be tested independently.

Documentation Considerations

Aside from standard user reference documentation, the internal documentation for the xTuple release process should be modified to suit third party developers who would be interested in building packages, and released on the xTuple wiki.

Release Considerations

The Package Manager should be made available on version 3.1 of xTuple ERP so that functionality developed using this methodology, specifically Highlander features, can be developed and deployed independently of the xTupleERP core release cycle.

AttachmentSize
packageManager.zip3.16 KB