Sponsor Link: EAS Training - Get training in the Essential toolset. Register your interest now. Read more
     
Home Getting Started Using the Essential Integration Server
Using the Essential Integration Server PDF Print E-mail

 

This tutorial introduces the Essential Integration Server and describes how imports from external data sources are performed with Essential Architecture Manager.

The Essential Integration Server is released at a pre-alpha version, currently. This article describes how this version is to be used and documents the main components of the integration solution.

Details of how the Essential Integration Server should be installed are available in the install guide.

 

Introduction

Information about your architecture can be imported into Essential Architecture Manager from any source repository, e.g. a configuration management repository, an Excel Spreadsheet or relational database, enabling you to quickly populate your architecture model using information that is already captured in your organisation. Additionally, this approach can be used for ‘mass data entry’ of elements to the model via, e.g. a spreadsheet, for repetitive task or where the information is being captured by someone who has no direct access to model.

 

Approach

The Essential Integration Server is a web-based application that drives the import process. It provides a Java Servlet-based engine for importing data from external sources with a simple index page that presents a menu of import options and invokes the integration server.
Currently, the Essential Integration Server provides the following services:

  • Repository Import Service - a service for importing elements from external sources into the Essential repository.

 

The server runs XSLT on the source data files to produce a script (contained in one or more text files) that uses the Protégé API to import information into the Essential repository. Importing directly to the underlying Protégé datastore is not recommended.


Performing an Integration or Import

The simple web application provides on-line instructions for how to run the integration processes. Completing the process requires manual steps in the current release and these are as follows:

Details of how to complete these steps are described in the sections below.

1. Go to the Essential Integration Server home page

Use the following URL, where is the hostname of the machine on which you have deployed the Essential Integration Server to.

http://:8080/EssentialIntegrationServer/
Essential Integration Server default homepage screenshot
You will then see a web page as shown, here.

2. Select the repository to import from via the links available

Click on the link for the repository that you wish to import data from. You can import full extracts or partial updates as required, provided the overall structure of the source data file remains the same. When the server has processed the source data file, the success page will be returned to you. Any errors that may have occurred during the import processing will be displayed via an alternative page.

Out-of-the box, two links are provided.

  • The Essential Architecture Manager repository import, which imports information in XML documents in the format of the Essential Architecture Manager reporting snapshots. This is used to import information from other Essential Architecture Manager repositories or other external repositories that have been transformed to represent the information in this format. To use this import, deploy your source XML (that is to be imported) as EssentialIntegrationServer/IntegrationScripts/XML/reportXML.xml
  • My External Source import, which imports information in any format, using your own XSL (which should generate the import script) operating on your own format XML document. Use this to import information in any other format (other than the Essential Architecture Manager report snapshot format). To use this, deploy your XSL as EssentialIntegrationServer/IntegrationScripts/XSL/import.xsl and your XML as EssentialIntegrationServer/IntegrationScripts/XML/importSource.xml

The index.html provides hyperlinks that invoke the Repository Import Service REST-ful Web Service. You can easily create additional imports by creating new links on this page to invoke the required XSL on your XML sources. For more information on the Repository Integration Service and the parameters for it, see below.Import Script Generation Success screenshot

A set of links to the import scripts produced by the Repository Integration Service is shown and these scripts are available to download. The scripts interact with the Essential repository through the Protégé API, rather than directly to the underlying datastore to ensure that the integrity of the repository is maintained. All integrations and imports should operate in this manner and directly reading or writing to the underlying Protégé datastore is not recommended. The scripts can be downloaded to your local workstation via your browser (e.g. right-click->Save Target As…). Once downloaded, you can execute them from your Protégé client, as described below. It is recommended that you download these to the default location, which is currently:

C:/EssentialAM/RepositoryIntegration
(Note Unix-style directory separators)

 

This default location is controlled by a Servlet parameter on the RepositoryIntegrationService Servlet and is used to specify the path to the import scripts and the Standard_Functions file. Simply update the 'clientDefaultLocation' parameter in the WebContent/WEB-INF/web.xml file to change the default location.

You can run these from an alternative location without changing the default but the path must be updated in the _all.txt and _0.txt files (where is the name of your external repository).

For large imports, where the Repository Import Service will create many script files, you can copy the generated scripts to the default location. The script files are written to the WebContent/IntegrationScripts folder.

3. Backup the target Essential Repository (recommended)

If an automatic backup of the repository database has not been performed recently, it is advisable to make an unscheduled backup. This is particularly important if the source data has not been imported before or its structure could have changed. Depending on the complexity of the integration that has been defined, un-doing an import that has not completed as expected can be impractical and restoring the database to the state before the import was run is the safest roll-back approach.

To perform this backup:

  • For file-based repositories (Stand Alone mode), make a copy of the PPRJ, PINS and PONS files of your Protege project.
  • For database-based repositories (Multi-User mode), take a backup of the essentialdb database using your database tools.

4. Execute the integration scripts from Protégé

Having downloaded or copied the integration / import scripts to the correct location and performed a backup of the repository, you are ready to import the external information into the Essential repository, as follows: Execute Import Scripts screenshot

  • Open Protégé and log into the live repository as normal.
  • Select the ‘Script Console’ tab
  • Enter the command: execfile(“c:/EssentialAM/RepositoryIntegration/_all.txt”)
  • Use the Unix-style ‘/’ directory characters as Protégé is a Java system
  • Replace with the name of the repository that you are importing from, e.g. essentialImport, and that matches the name of the _all.txt file that you downloaded.
  • If you are using an alternative location to the default for your scripts, replace the above path to the _all.txt file with your actual path.
  • Press the ‘Enter’ key on your keyboard to execute the import. See 'Execute Import Script' screenshot.


If you have saved the script file in an alternative location to the default, make sure that you update the path in _0.txt and _all.txt

Import Completed Successfully screenshotThe import process may take some time, depending on the current number of users logged into the Essential modelling environment and the size and complexity of the integration that is being performed. Once the import has completed, the script will provide a summary update of what it did via the Script Console, see screenshot. If any error messages are reported or it appears that the content of the repository (viewed via the ‘Instances’ tab) is not as expected, you may wish to stop the Essential Server and restore the repository from your last backup (or just restart Protege without saving your project in Stand-Alone mode). In such cases, the first thing to investigate is the source data file.

Running the script again will demonstrate the update functionality of the integration solution. This shows that attempting to import an instance that is already in the repository will cause it to be updated not duplicated.

Import Updates screenshot

 

Adding new imports

To add a new import to the Essential Integration Server, update the index.html file in the root directory of the Essential Integration Server web application to add additional menu items with links configured to call the integration servlet – RepositoryImportService - with the correct parameters as follows:

  • xslFile – the name of the XSL file that processes the source data file into the import script. These are deployed to the IntegrationScripts/XSL directory of the web application by default. This default is controlled by servlet parameters for the Repository Import Service.
  • scriptFile – the name of the resulting import script file to use (this will have numbered suffixes automatically added when the scripts are created)
  • externalRepository – the name of the source data file that is to be imported. This should be specified with a relative path, where the root is the ‘IntegrationScripts’ directory of the web application. Normally, these are deployed to IntegrationScripts/XML.
  • externalRepositoryName[optional] the of the external repository that is being imported. It is import to make sure that this is unique within your Essential Architecture Manager repository as it is used for synchronisation or updates (to avoid duplicate instances).
  • externalRepositoryDesc - [optional] a brief description of the external repository that is being imported. Stored within your Essential Architecture Manager repository with the name.

 

 

Additional Support

  • For additional support in the development of custom imports from your existing information sources, professional services from the Essential Project Team are available. Please use the Services Enquiry form for more information.

 

 

 
Related Articles