Earth Observation Mission CFI Software
FILEHANDLING SOFTWARE USER MANUAL

Code	EO-MA-DMS-GS-011
Issue	4.27
Date	07/06/24

	Name	Function	Signature
Prepared by:	EOCFI Team	Project Engineers
Checked by:	Inês Estrela	Project Manager
Approved by:	Antonio Gutierrez	Division Head

DOCUMENT INFORMATION

Contract Data		Classification
Contract Number:	4000102614/1O/NL/FF/ef	Internal
Contract Number:	4000102614/1O/NL/FF/ef	Public
Contract Issuer:	ESA / ESTEC	Industry	X
Contract Issuer:	ESA / ESTEC	Confidential

External Distribution
Name	Organization	Copies

Electronic handling
Document generator:	Doxygen 1.7.1
Electronic file name:	eo-ma-dms-gs-011-10

Document Status Log

Issue	Change Description	Date
1.0	These libraries corresponds to version 4.0 of C libraries.	27/03/09
4.1	These libraries corresponds to version 4.1 of C libraries.	07/05/10
4.2	These libraries corresponds to version 4.2 of C libraries.	31/01/11
4.3	These libraries corresponds to version 4.3 of C libraries.	06/02/12
4.4	These libraries corresponds to version 4.4 of C libraries.	05/07/12
4.5	These libraries corresponds to version 4.5 of C libraries.	01/03/13
4.6	These libraries corresponds to version 4.6 of C libraries.	03/10/13
4.7	These libraries corresponds to version 4.7 of C libraries.	28/03/14
4.8	These libraries corresponds to version 4.8 of C libraries.	29/10/14
4.9	These libraries corresponds to version 4.9 of C libraries.	23/04/15
4.10	These libraries corresponds to version 4.10 of C libraries.	29/10/15
4.11	These libraries corresponds to version 4.11 of C libraries.	15/04/16
4.12	These libraries corresponds to version 4.12 of C libraries.	03/11/16
4.13	These libraries corresponds to version 4.13 of C libraries.	05/04/17
4.14	These libraries corresponds to version 4.14 of C libraries.	16/11/17
4.15	These libraries corresponds to version 4.15 of C libraries.	20/04/18
4.16	These libraries corresponds to version 4.16 of C libraries.	09/11/18
4.17	These libraries corresponds to version 4.17 of C libraries.	10/05/19
4.18	These libraries corresponds to version 4.18 of C libraries.	08/11/19
4.19	These libraries corresponds to version 4.19 of C libraries.	29/05/20
4.20	These libraries corresponds to version 4.20 of C libraries.	30/11/20
4.21	These libraries corresponds to version 4.21 of C libraries.	23/06/21
4.22	These libraries corresponds to version 4.22 of C libraries.	22/12/21
4.23	These libraries corresponds to version 4.23 of C libraries.	23/06/22
4.24	These libraries corresponds to version 4.24 of C libraries.	29/11/22
4.25	These libraries corresponds to version 4.25 of C libraries.	10/05/23
4.26	These libraries corresponds to version 4.26 of C libraries.	31/10/23
4.27	These libraries corresponds to version 4.27 of C libraries.	07/06/24

ACRONYMS, NOMENCLATURE AND TERMINOLOGY

Acronyms

ANX	Ascending Node Crossing
AOCS	Attitude and Orbit Control Subsystem
ASCII	American Standard Code for Information Interchange
BOM	Beginning Of Mission
CFI	Customer Furnished Item
EOM	End Of Mission
ESA	European Space Agency
ESTEC	European Space Technology and Research Centre
GPL	GNU Public License
GPS	Global Positioning System
IERS	International Earth Rotation Service
I/F	Interface
LS	Leap Second
OBT	On-board Binary Time
OSF	Orbit Scenario File
SRAR	Satellite Relative Actual Reference
SUM	Software User Manual
TAI	International Atomic Time
UTC	Coordinated Universal Time
UT1	Universal Time UT1
WGS[84	World Geodetic System 1984

Nomenclature

CFI	A group of CFI functions, and related software and documentation. that will be distributed by ESA to the users as an independent unit
CFI function	A single function within a CFI that can be called by the user
Library	A software library containing all the CFI functions included within a CFI plus the supporting functions used by those CFI functions (transparently to the user)

Notes in Terminology

In order to keep compatibility with legacy CFI libraries, the Earth Observation Mission CFI Software makes use of terms that are linked with missions already or soon in the operational phase like the Earth Explorers. This may be reflected in the rest of the document when examples of Mission CFI Software usage are proposed or description of Mission Files is given.

APPLICABLE AND REFERENCE DOCUMENTS

Applicable Documents

[GEN_SUM]

Earth Observation Mission CFI Software. General Software User Manual. EO-MA-DMS-GS-017. Issue 4.27 07/06/24

Reference Documents

[MCD]	Earth Observation Mission CFI Software. Conventions Document. EO-MA-DMS-GS-0001. Issue 4.27 07/06/24.
[MSC]	Earth Observation Mission CFI Software. Mission Specific Customizations. EO-MA-DMS-GS-0018. Issue 4.27 07/06/24.
[EE_COMMON_SUM]	Earth Observation Mission CFI Software. EECommon Software User Manual. EO-MA-DMS-GS-010. Issue 4.27 07/06/24.

INTRODUCTION

The File Handling Library provides a simple programming interface for creating, modifying, writing and reading XML files tailored to the XML usage needs of the Earth Observation Missions Ground Segment files.

It is built on top of the GNOME libxml2 library but it hides most of the details associated to file parsing, in-memory representation and file writing.

Programming Usage Guide

The interface to an XML file represented as a tree

Any well-formed XML file may be viewed as a hierarchy of elements, attributes and values that can be represented as a tree. In fact this is the in-memory representation used by libxml when an XML file is read into memory.

Although the FILE_HANDLING library hides all these details to the programmer, a few basic concepts need to be presented here to understand how to use the library functions in order to achieve a given goal. We assume that the user has some basic knowledge of XML and understands the concepts of XML elements, attributes and text nodes.

An element B which is contained directly within an element A is referred to as child of A. Unsurprisingly, A is said to be the parent of B.
Two elements, B and C, that are children of a common parent A are said to be sibling elements.

The FILE_HANDLING library uses iterators for traversing the in-memory representation of an XML file as built by libxml. An iterator is an abstraction of a pointer to a specific element within a collection. The FILE_HANDLING library uses an iterator to XML elements since elements are the basic building blocks of XML files. All the operations related with xml files are grouped in the class XmlFile.

Opening a file for reading and/or modification

The first operation a user must perform in order to use an existing XML file for reading and/or modifying it is loading the file content into memory. This operation is performed by the method XmlFile::read. This method takes as an input the file name. The FILE_HANDLING library may handle in parallel up to XFCFI_MAX_FILES_NUMBER. When a file is read, XmlFile::read assigns a unique number (stored in XmlFile object). This unique number is used, thereon in order to access the in-memory representation of that file, in a transparent way for the user.

Reading a file sequentially

The in-memory representation of an XML file used by libxml allows an optimum traversal using a first-child-next-sibling algorithm. This algorithm ensures that all elements within the tree are visited only once using recursively the following rules:

If an element has children elements that have not been visited, the next element to visit is its first child.
If all element children have been visited or the element has no children, the next element to visit is its next sibling.
If an element has no next sibling, then go to the parent and apply the second rule.

Starting with an iterator pointing to the root element, all elements are visited once and only once until the iterator returns to the root element.

A file may be read sequentially using the family of methods XmlFile::getElementValueAs*.These methods will return the value of the first element found whose name matches exactly the element name provided as input typed as requested in the function name.

The FILE_HANDLING library uses a global iterator for each file (do not mistake the concept of global iterator with that of a global variable). When a file is read using the XmlFile::read function, this global iterator points automatically to the root element. Each function uses a local iterator for searching the requested element.

The global iterator behaves as follows:

If the requested element is found, the global iterator is set to point to the found element.
When an XmlFile::getElementValueAs* method is called several times, the search will start just after the point where the global iterator was left in the preceding call.
In case the requested element name is not found, the global iterator will not be moved
If the element is found and an error occurs, i.e., because of a type conversion error or an empty element, the iterator will point to the found element.
The XmlFile::root function allows to reset the iterator to the root element in order to start over from the beginning.

Sometimes, it is desirable to have a function that reads all elements with a given name and creates a vector of values. This functionality is achieved by the XmlFile::getElementArrayAs* family of methods. A call to any of these functions will first set the iterator pointing to the root element, then will traverse the file looking for the requested elements and will return the vector of values leaving the iterator pointing to the last element read.

Attributes can be read with the family of methods XmlFile::getAttributeValueAs* specifying the attribute name as well as the name of the element that contains it. If the global iterator is pointing to an element name different from the specified one, this element is first looked for, the iterator is moved to it and then the requested attribute is looked for within that element. If the iterator is already pointing to an element whose name matches the specified one, the attribute is directly searched for within that element.

Reading a file using path expressions

Traversing a file sequentially may not always be the optimum way of accessing information if we just need to read a few values. Random access to the in-memory representation of an XML file is achieved using XPath expressions. XPath expressions allow addressing XML elements and attributes using a syntax that resembles the one used for addressing files and directories in a hierarchical file system and is based on the following rules:

Element names are separated by the "/" character.
XPath expressions may be absolute (i.e., starting with the "/" character and listing all elements from the root to the requested one), or relative to the current position of the iterator (i.e., without the leading "/").
"." is used for the current element, and ".." for the parent element as it is used for directories.
If an element has a collection of children with the same name, individual elements may be accessed using their element name followed by an integer index between square brackets, being 1 the index to the first element (i.e., A[1], for the first occurrence of element A, A[2] for the second and so on)
Attribute names are preceded by the "@" character.

The XmlFile::getPathValueAs* family of methods implement random access file reading using XPath expressions. The global iterator has the same behaviour as in the XmlFile::getElementValueAs* familliy of methods.

Searching for elements with a given value

The FILE_HANDLING library provides two functions for searching the XPath expression of an element with a given value.

The XmlFile::findValueInElement function uses an element name and a string value and returns the first element found whose value matches the value provided as input.
The XmlFile::findValueInPath function uses an XPath expression and a string value. The XPath expression must contain the [*] character sequence in order to indicate the elements in a sequence (list) that are iterated.

The search functions set the file iterator pointing to the found element and return its absolute XPath. If the search operation returns an error, the iterator is not moved.

Creating new elements or removing existing elements

A file can be created in memory from scratch using the XmlFile::create function. Like the XmlFile::read that reads an existing XML file from disk, it returns a unique integer value that is stored in the XmlFile object.

When a file is created in memory, the first operation that must be performed is the creation of the root element, which is done by the XmlFile::createRoot function.

From that point, elements may be added using as input a reference element to which they are attached and the created element name. The element to which the new element is attached is referenced using an XPath expression. The following possibilities are available.

Add a child to the reference element using XmlFile::addChild . If the reference element already has children, the new element will be appended at the end of the children list.
Add a sibling after the reference element using XmlFile::addNext . This will insert the new element just after the reference one.
Add a sibling before the reference element using XmlFile::addPrevious . This will insert the new element just before the reference one.
Attributes are added with the XmlFile::addAttribute function using the name of the element that contains them as the reference element.

After a new element has been added to the in-memory representation of the XML file, the global iterator is pointing to the last added element. Therefore, relative XPath expressions used just after an element insertion are relative to the inserted element.

Existing elements are removed using the XmlFile::removeNode function with an XPath expression addressing them. If the requested element has children, all of them will be removed recursively.

Setting and/or modifying element values

When elements and attributes are created, they have no value. Element and attribute values may be set and/or modified with the XmlFile::setValue overcharged methods using an XPath expression for addressing the requested element or attribute. Format information as in the C standard library printf family of functions must be provided when setting an element or attribute value.

Writing an XML file to disk

An in-memory representation of an XML file can be written to a file on disk with the XmlFile::write function. Note that if the in-memory representation is loaded from a file, it may be written under a different file name.

Releasing memory

The in-memory representation of an XML file normally takes a significant amount of memory space, typically between 7 and 10 times the size of the file on disk. For this reason it is important to provide the means for releasing the memory allocated after being used. This can be done using the XmlFile::clean, or it is done when the object XmlFile is destroyed.

LIBRARY INSTALLATION

For a detailed description of the installation of any CFI library, please refer to [GEN_SUM].

LIBRARY USAGE

To use the FILE_HANDLING software library in a user application, that application must include in its source code the necessary header files for the concrete application:

FileDataHandlingData.h
XmlFile.h
CfiError.h

Third party libraries:

POSIX thread library: libpthread.so (Note: this library is normally pre-installed in Linux and MacOS platforms. For Windows platforms, pthread.lib is included in the distribution package, with license LGPL)
GEOTIFF, TIFF, PROJ, LIBXML2 libraries (these libraries are included in the distribution package. Their usage terms and conditions are available in the file "TERMS_AND_CONDITIONS.TXT" which is part of the distribution package).

To link correctly this application, the user must include in his linking command flags like (assuming cfi_lib_dir and cfi_include_dir are the directories where respectively all CFI libraries and include files have been installed, see [GEN_SUM] for installation procedures):

LINUX:
-Icfi_include_dir -Lcfi_lib_dir
-lCfiFileHandling -lCfiEECommon -lm -lc
MacOS:
-Icfi_include_dir -Lcfi_lib_dir
-lCfiFileHandling -lCfiEECommon -lm -lc
Note: the EOCFI libraries are built using the switch -mmacosx-version-min=10.9 for backward compatibility, so the libraries are built for STL library libstdc++.
Starting from Mavericks the default STL library is libc++, so link problems can appear if you do not use the same build switch as the EOCFI
WINDOWS (dynamic libraries):
/I "cfi_include_dir" /libpath:"cfi_lib_dir"
libCfiFileHandling.lib libCfiEECommon.lib
WINDOWS (static libraries):
/I "cfi_include_dir" /libpath:"cfi_lib_dir"
libCfiFileHandling.lib libCfiEECommon.lib libxml2.lib pthread.lib Ws2_32.lib

Note that, as Earth Observation CFI libraries are dynamic, cfi_lib_dir must be included in the path where the system looks for dynamic libraries (LD_LIBRARY_PATH in LINUX, DYLD_LIBRARY_PATH in MacOs). For WINDOWS sytem, the executable will look for .dll libraries, not the .lib ones that are used to create the executable, so they must be in the path described by PATH variable.
All the classes described in this document are defined in EECFI namespace, to avoid any possible conflict with classes of other libraries, so the user must choose one of the following two options in order to use the classes:

Inserting using namespace EECFI; in the code.
Using the classes with the namespace prefix (e.g.: EECFI::OrbitFile orbitFile;)

ERROR HANDLING

The error management in C++ FILE_HANDLING is made throw exceptions, that is, if any error is produced, an exception of type CfiError is thrown and it must be catched putting the code in a try-catch block.

See [GEN_SUM] to know more about how to handle the CFI errors. For a descripton about the CfiError class and its methods, see [EE_COMMON_SUM].